CN116680988A - Porous medium permeability prediction method based on Transformer network - Google Patents
Porous medium permeability prediction method based on Transformer network Download PDFInfo
- Publication number
- CN116680988A CN116680988A CN202310737138.9A CN202310737138A CN116680988A CN 116680988 A CN116680988 A CN 116680988A CN 202310737138 A CN202310737138 A CN 202310737138A CN 116680988 A CN116680988 A CN 116680988A
- Authority
- CN
- China
- Prior art keywords
- porous medium
- dimensional
- image
- sequence
- permeability
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 230000035699 permeability Effects 0.000 title claims abstract description 46
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 53
- 239000011159 matrix material Substances 0.000 claims abstract description 48
- 238000012549 training Methods 0.000 claims abstract description 31
- 239000011148 porous material Substances 0.000 claims abstract description 29
- 238000004088 simulation Methods 0.000 claims abstract description 16
- 238000012360 testing method Methods 0.000 claims abstract description 15
- 238000003062 neural network model Methods 0.000 claims abstract description 13
- 238000012795 verification Methods 0.000 claims abstract description 9
- 239000013598 vector Substances 0.000 claims description 85
- 239000010410 layer Substances 0.000 claims description 63
- 230000007246 mechanism Effects 0.000 claims description 12
- 230000006870 function Effects 0.000 claims description 10
- 238000012512 characterization method Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 5
- 230000004913 activation Effects 0.000 claims description 4
- 230000001965 increasing effect Effects 0.000 claims description 4
- 239000007790 solid phase Substances 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 4
- 238000012935 Averaging Methods 0.000 claims description 3
- 230000002776 aggregation Effects 0.000 claims description 3
- 238000004220 aggregation Methods 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 3
- 238000003709 image segmentation Methods 0.000 claims description 3
- 230000000704 physical effect Effects 0.000 claims description 3
- 239000002356 single layer Substances 0.000 claims description 3
- 230000000694 effects Effects 0.000 claims description 2
- 238000011478 gradient descent method Methods 0.000 claims description 2
- 238000010200 validation analysis Methods 0.000 description 8
- 238000013528 artificial neural network Methods 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 5
- 239000012530 fluid Substances 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000003344 environmental pollutant Substances 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 231100000719 pollutant Toxicity 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 239000011435 rock Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2119/00—Details relating to the type or aim of the analysis or the optimisation
- G06F2119/02—Reliability analysis or reliability optimisation; Failure analysis, e.g. worst case scenario performance, failure mode and effects analysis [FMEA]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Medical Informatics (AREA)
- Geometry (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a porous medium permeability prediction method based on a transducer network, and belongs to the technical field of pore scale numerical simulation. The method simulates a flow field by using a pore scale simulation method, and calculates the permeability of the three-dimensional porous medium; taking the three-dimensional structure image of the porous medium as a space sequence consisting of two-dimensional slice images, and extracting the physical attribute of each two-dimensional slice image; adding a corresponding physical parameter matrix into a three-dimensional structure image of the porous medium, constructing a sample set by taking the permeability of the porous medium as a label value corresponding to the image, determining a training set, a verification set and a test set, and constructing and training a PhyCNN-transducer neural network model by utilizing a convolutional neural network CNN and a transducer network, thereby realizing accurate prediction of the permeability of the porous medium. According to the invention, the three-dimensional image regression problem is converted into the two-dimensional image sequence regression problem, so that the understanding of the structural characteristics of the porous medium is deepened, and the essential characteristics of the porous medium are captured, thereby improving the prediction performance and generalization capability of the model.
Description
Technical Field
The invention relates to the technical field of pore scale numerical simulation, in particular to a porous medium permeability prediction method based on a Transformer network.
Background
The permeability is taken as an important parameter for measuring the difficulty of fluid passing through the porous medium, and plays a basic control role in establishing a mathematical model of porous medium seepage and pollutant migration. Accurate acquisition of permeability is helpful for more comprehensively knowing the detailed process of substance transfer in the porous medium, reduces the uncertainty of application and improves the application efficiency, so that the dynamic characteristics of the geologic body can be accurately and quantitatively evaluated and predicted.
The traditional permeability measurement method is a Darcy pressure gradient method, but the method has the problems of long test period and large environmental disturbance.
In recent twenty years, with the rapid development of computational fluid mechanics, pore scale numerical simulation has been widely applied to simulation of porous media fields and calculation of macroscopic parameters (such as permeability), mainly including direct simulation methods such as lattice boltzmann method, classical computational fluid mechanics (Computational Fluid Dynamics, CFD), and pore network model method. The direct simulation method carries out simulation calculation based on a three-dimensional image of the pore space, so that complex boundary conditions exist, and the calculation cost is high due to the nonlinear characteristics of a Navier-Stokes equation, so that the calculable size of the porous medium is limited. The pore network model method simplifies the geometric shape of the pore space, simultaneously reserves the necessary pore structure characteristics of substance transportation, has shorter calculation time and can simulate a porous medium rock core with larger size, but the accuracy of the simulation result is generally inferior to that of a direct simulation method.
With the vigorous development of computer technology and deep learning, the use of deep learning neural network models to predict the permeability of porous media has become a trend. The basic idea of the method is to establish a direct mapping relation between the porous medium digital image and the permeability value by using a neural network architecture. The most common of these is convolutional neural network CNN. From darcy's law, the permeability of a porous medium is determined by its pore space structure. The porous medium digital image is used as the input of a CNN model, the network compresses the extracted high-dimensional deep feature image into feature vectors through the transformation operations of convolution, pooling, residual connection and the like, and the mapping relation with the permeability value is further established through the full-connection layer.
As an alternative to the traditional numerical simulation method, the neural network model has short calculation time and high accuracy, and is a main stream direction for rapidly predicting the permeability parameter or other physical parameters of the porous medium in the future. However, training neural network models, particularly deep three-dimensional convolutional neural network models, is a difficult challenge for the memory of Graphics Processing Unit (GPU) because the memory size of the GPU often limits the predictive performance and training efficiency of the model.
Disclosure of Invention
Aiming at the defects existing in the prior art, the invention provides a porous medium permeability prediction method based on a transform network, which is used for establishing a PhyCNN-transform hybrid neural network, converting a three-dimensional image regression problem into a two-dimensional image sequence regression problem and solving the problems of prediction performance and training efficiency of a memory size limit model of a GPU to a certain extent.
In order to solve the technical problems, the invention provides the following technical scheme: a porous medium permeability prediction method based on a transducer network comprises the following steps:
s1, simulating a flow field by using a pore scale simulation method, and calculating the permeability of a three-dimensional porous medium;
s2, regarding the three-dimensional structure image of the porous medium as a space sequence consisting of two-dimensional slice images, and extracting the physical attribute of each two-dimensional slice image;
s3, adding a corresponding physical parameter matrix into the three-dimensional structure image of the porous medium to construct a sample set, and determining a training set, a verification set and a test set;
s4, constructing a PhyCNN-transducer neural network model based on the convolutional neural network CNN and the transducer network; the convolutional neural network CNN is used for extracting image features containing physical parameter information, and is different from a common CNN and expressed as PhyCNN;
combining the training set with a gradient descent method, and training and optimizing parameters of a neural network model PhyCNN-transducer; the verification set is used for evaluating the prediction capability of the PhyCNN-transducer model in the training process; the test set was used to examine the final predictive effect of the PhyCNN-transporter model.
According to the technical scheme, the pore scale simulation method comprises a pore network model PNM and a lattice Boltzmann method LBM.
According to the technical scheme, the three-dimensional structure image segmentation step of the porous medium comprises the following steps:
the three-dimensional structure image of the porous medium is cut into continuous two-dimensional images along any axis direction (generally, the water flow direction) of a Cartesian space rectangular coordinate system, the length of the porous medium in the cutting axis direction is the sequence length, and the thickness of the cut image is equal to the pixel size.
The axis along which the cutting is performed in any one axis direction of the cartesian space rectangular coordinate system is called a cutting axis.
According to the technical scheme, the physical properties of the two-dimensional slice image comprise porosity and specific surface area parameters;
the physical attribute extraction method comprises the following steps: the 1 value of the porous medium two-dimensional slice gray level image represents a solid phase, the 0 value pixel of the porous medium two-dimensional slice gray level image represents a pore structure, the statistics of the pixel points of the slice image is realized by using python programming, and the porosity and the specific surface area are calculated.
The porosity is defined as the ratio of the occupied area of the pores to the slice area, and the representation method is the ratio of the number of pore pixels to the total number of pixels of the slice image; the specific surface area is defined as the ratio of the total perimeter of all pores in the slice to the partial area of the pores, and the representation method is the ratio of the pixel number of the pore edge to the total pixel number of the pores;
according to the technical scheme, the corresponding physical parameter matrix is added into the three-dimensional structure image of the porous medium, and the specific method comprises the following steps of: and establishing a two-dimensional parameter matrix with the same size as the two-dimensional slice image, wherein the upper half elements of the matrix are the porosity values of the slice, and the lower half elements of the matrix are the specific surface area values of the slice, and adding the parameter matrix as a single-channel image into the original gray image of the three-dimensional structure of the porous medium, so that the three-dimensional structure image of the porous medium contains physical parameter information.
According to the technical scheme, the porous medium permeability calculated by using a pore network model PNM or a lattice Boltzmann method LBM is used as a label value corresponding to an image, so that the sample set is formed; the sample set is divided into a training set, a verification set and a test set according to a certain proportion, and can be divided into a 6:2:2 ratio or other similar proportions.
According to the technical scheme, the PhyCNN-transducer neural network model consists of a convolutional neural network CNN and a transducer network, and specifically comprises the following steps:
the convolutional neural network CNN is provided with 4 two-dimensional convolutional layers with different sizes, and a single-layer first full-connection layer is tightly connected after the 4 convolutional layers;
the first full-connection layer of the convolutional neural network CNN is connected with the coding layer of the transducer network;
the Transformer network has 6 identical coding layers, each coding layer having 2 sublayers; the first sub-layer is a multi-head self-attention layer, the second sub-layer is a second full-connection layer, and residual connection and regularization operations are included between the multi-head self-attention layer and the second full-connection layer;
the convolution layer is used for generating a multi-channel feature map for each two-dimensional image in the sequence; the first full-connection layer is used for compressing the multi-channel feature map into a feature vector with a fixed dimension; the size and the number of the two-dimensional convolution kernels are fixed.
The convolutional neural network CNN extracts image features containing physical parameter information, and is distinguished from common CNNs and expressed as PhyCNNs.
The Transformer network is a Seq2Seq model based on a multi-headed self-attention mechanism. The self-attention mechanism can correlate different positions in the sequence and re-characterize the sequence. Multi-headed self-attention allows the model to focus information from different subspaces at different locations, thereby enhancing the characterizability of the model.
According to the technical scheme, the regression prediction of the permeability comprises the following specific steps:
inputting a porous medium three-dimensional structure containing physical parameter information and represented by a two-dimensional image sequence into a two-dimensional convolution layer of a CNN model, and increasing the nonlinear characterization capability of a network by using a Batch Norm and ReLU activation function to generate a multi-channel feature map for each two-dimensional image in the sequence;
the multi-channel feature map is input to the fully connected layer, compressing it into a fixed-dimension feature vector. Repeating this operation for each image in the sequence, thereby forming a sequence of feature vectors;
determining positional information for each item in the sequence using sinusoidal positional codes;
adding position codes into the feature vector sequence, inputting the position codes into a multi-head self-attention layer of a transducer network, associating different positions in the feature vector sequence, and re-characterizing the positions;
and (3) averaging in the dimension of the sequence length of the characteristic vector sequence after characterization, and inputting the characteristic vector sequence into a full-connection layer to carry out regression prediction on the permeability of the three-dimensional porous medium.
Since the self-attention mechanism ignores the position information of each item in the sequence, it is necessary to add a position code to the input sequence;
according to the technical scheme, the sinusoidal position codes:
wherein 2i and 2i+1 represent the 2i and 2i+1 th elements embedded in the feature vector, pos is the global position of the feature vector in the sequence, PE represents the position code, d model Representing the number of features expected by the model, i.e. the dimension of the feature vectors in the sequence.
According to the technical scheme, the re-characterization step of the porous medium feature vector sequence comprises the following steps:
in order to select information related to porous medium permeability from an input sequence of feature vectors containing position codes, a multi-headed self-attention mechanism introduces a dimension d k Is a query vector and a key vector of (1) and has a dimension d v Is a vector of values of (a);
the correlation between each feature vector (represented by a key value pair) in the sequence and the query vector is calculated as a scoring function. The key vector is used for calculating the attention distribution, and the value vector is used for calculating the aggregation information;
the dot product of the query vector is calculated using all key vectors and divided byAnd then calculating the weight of the value vector by using a softmax function, wherein the matrix representation method comprises the following steps:
wherein Q is a query vector matrix, K is a key vector matrix, V is a value vector matrix, d k ＝d model /h, representing Q, K in the dimension of h subspaces (heads), K T Is the transpose of the key vector matrix;
multi-headed self-attention projects the query matrix Q, key matrix K, and value matrix V to d with different linear projections of the h subspaces (heads), respectively k 、d k And d v And (3) the output of each subspace (head) is spliced and projected again to obtain the final multi-head self-attention mechanism output:
wherein, the projections are all parameter matrixes,and +.>Weight matrix of linear transformation, i E [1, h]H represents the number of projections, i.e. the number of heads, subspaces of multi-head self-attention, d model Representing model expectationsFeature numbers of (a), i.e. the dimension of feature vectors in the sequence, head i Representing the ith "head" of the multi-head self-attention mechanism.
Compared with the prior art, the invention has the following beneficial effects: according to the PhyCNN-converter model provided by the invention, the three-dimensional image regression problem is converted into the regression problem of a two-dimensional image sequence, and aiming at the task of porous medium permeability prediction, two-dimensional physical parameter information is added into a porous medium digital image, so that the model can deepen understanding of structural characteristics of the porous medium, and the essential characteristics of the porous medium are captured, so that the prediction accuracy and generalization capability are improved. Compared with a common 3D CNN model, the PhyCNN-transducer model greatly reduces the parameter quantity of neural network training, and greatly improves the training speed, the prediction performance and the generalization capability of the model.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate the invention and together with the embodiments of the invention, serve to explain the invention. In the drawings:
FIG. 1 is a frame structure of a PhyCNN-transporter model;
FIG. 2 is a network structure of a transducer;
FIG. 3 is a three-dimensional structure of a porous media sample;
FIG. 4 is a data distribution of a porous medium;
FIG. 5 is a graph showing loss values of PhyCNN-transporter model at different batch sizes during training;
FIG. 6 is a prediction result of the PhyCNN-transporter model for the validation set at different batch sizes;
FIG. 7 is a prediction result of an optimal PhyCNN-transporter model;
fig. 8 is a prediction result of the 3D CNN model.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1-8, the present invention provides a method for predicting permeability of a porous medium based on a Transformer network, which comprises the following steps:
s1, simulating a flow field by using a PNM or LBM pore scale simulation method, and calculating the permeability of a three-dimensional porous medium;
s2, regarding the three-dimensional structure image of the porous medium as a space sequence consisting of two-dimensional slice images, and extracting the physical attribute of each two-dimensional slice image;
wherein the physical properties of the two-dimensional slice image include porosity and specific surface area parameters; the three-dimensional structure image segmentation step of the porous medium comprises the following steps: the three-dimensional structure image of the porous medium is cut into continuous two-dimensional images along any axis direction (generally, the water flow direction) of a Cartesian space rectangular coordinate system, the length of the porous medium on a cutting axis is the sequence length, and the thickness of the cut image is equal to the pixel size.
The physical attribute extraction method comprises the following steps: the 1 value of the porous medium two-dimensional slice gray level image represents a solid phase, the 0 value pixel of the porous medium two-dimensional slice gray level image represents a pore structure, the statistics of the pixel points of the slice image is realized by using python programming, and the porosity and the specific surface area are calculated.
S3, adding a corresponding physical parameter matrix into the three-dimensional structure image of the porous medium, constructing a sample set by taking the permeability of the porous medium as a label value corresponding to the image, and determining a training set, a verification set and a test set. The method for adding the corresponding physical parameter matrix into the three-dimensional structure image of the porous medium comprises the following steps of: and establishing a two-dimensional parameter matrix with the same size as the two-dimensional slice image, wherein the upper half elements of the matrix are the porosity values of the slice, and the lower half elements of the matrix are the specific surface area values of the slice, and adding the parameter matrix as a single-channel image into the original gray image of the three-dimensional structure of the porous medium, so that the three-dimensional structure image of the porous medium contains physical parameter information.
The sample set may be divided into a training set, a validation set, and a test set in a 6:2:2 or other similar ratio.
S4, constructing a PhyCNN-transducer neural network model based on a convolutional neural network CNN and a transducer network, wherein the PhyCNN-transducer neural network model specifically comprises the following steps: inputting a porous medium three-dimensional structure containing physical parameter information and represented by a two-dimensional image sequence into a two-dimensional convolution layer of a CNN model, and increasing the nonlinear characterization capability of a network by using a Batch Norm and ReLU activation function to generate a multi-channel feature map for each two-dimensional image in the sequence;
the multi-channel feature map is input to the fully connected layer, compressing it into a fixed-dimension feature vector. Repeating this operation for each image in the sequence, thereby forming a sequence of feature vectors;
determining positional information for each item in the sequence using sinusoidal positional codes; the sinusoidal position code:
wherein 2i and 2i+1 represent the 2i and 2i+1 th elements embedded in the feature vector, pos is the global position of the feature vector in the sequence, PE represents the position code, d model Representing the number of features expected by the model, i.e. the dimension of the feature vectors in the sequence.
Adding position codes into the feature vector sequence, inputting the position codes into a multi-head self-attention layer of a transducer network, associating different positions in the feature vector sequence, and re-characterizing the positions; the re-characterization step of the porous medium feature vector sequence comprises the following steps:
in order to select information related to porous medium permeability from an input sequence of feature vectors containing position codes, a multi-headed self-attention mechanism introduces a dimension d k Is a query vector and a key vector of (1) and has a dimension d v Is a vector of values of (a);
the correlation between each feature vector (represented by a key value pair) in the sequence and the query vector is calculated as a scoring function. The key vector is used for calculating the attention distribution, and the value vector is used for calculating the aggregation information;
the dot product of the query vector is calculated using all key vectors and divided byAnd then calculating the weight of the value vector by using a softmax function, wherein the matrix representation method comprises the following steps:
wherein Q is a query vector matrix, K is a key vector matrix, V is a value vector matrix, d k ＝d model /h, representing Q, K in the dimension of h subspaces (heads), K T Is the transpose of the key vector matrix;
multi-headed self-attention projects the query matrix Q, key matrix K, and value matrix V to d with different linear projections of the h subspaces (heads), respectively k 、d k And d v And (3) the output of each subspace (head) is spliced and projected again to obtain the final multi-head self-attention mechanism output:
wherein, the projections are all parameter matrixes,and +.>Weight matrix of linear transformation, i E [1, h]H represents the number of projections, i.e. the number of heads, subspaces of multi-head self-attention, d model Representing the number of features expected by the model, i.e. the dimension of the feature vector in the sequence, head i Representing the ith "head" of the multi-head self-attention mechanism.
Averaging the characteristic vector sequences in the dimension of sequence length after characterization, inputting the characteristic vector sequences into a full-connection layer, and carrying out regression prediction on the permeability of the three-dimensional porous medium;
the PhyCNN-transducer neural network model consists of a convolutional neural network CNN and a transducer network, and specifically comprises the following steps: the convolutional neural network CNN is provided with 4 two-dimensional convolutional layers with different sizes, and a single-layer first full-connection layer is tightly connected after the 4 convolutional layers; the first full-connection layer of the convolutional neural network CNN model is connected with the coding layer of the Transformer network; the Transformer network has 6 identical coding layers, each coding layer having 2 sublayers; the first sub-layer is a multi-head self-attention layer, the second sub-layer is a second full-connection layer, and residual connection and regularization operations are included between the multi-head self-attention layer and the second full-connection layer; the convolution layer is used for generating a multi-channel feature map for each two-dimensional image in the sequence; the first full connection layer is used for compressing the multi-channel feature map into a feature vector with a fixed dimension.
Training a PhyCNN-transporter neural network by using a training data set and optimizing parameters by using a reverse gradient method; using the validation set to evaluate the model, calculating a regression performance index, such as Root Mean Square Error (RMSE), of the model, determining coefficients (R 2 ) Etc.; adjusting the model according to the evaluation result, such as a network structure, a super parameter such as Batch size (Batch size), a learning rate, or using a regularization technology; permeability predictions were made for the test set (new porous media image) using a trained PhyCNN-transducer neural network.
Example 1
Randomly selecting 6250 semi-true three-dimensional porous media (shown in figure 3) generated by Arash Rabbani and corresponding permeability values calculated by PNM; three-dimensional porous media image volume of 256 3 Voxel, voxel size 5 μm. Permeability values are expressed in square units of pixels, and their data distribution in darcy units is shown in fig. 4;
considering the three-dimensional porous medium as 256 two-dimensional gray-scale images which are continuous along the x-axis direction, wherein the image size is 256 multiplied by 256, and the pixel with the value of 0 represents the pore structure and the pixel with the value of 1 represents the solid-phase skeleton; the porosity and specific surface area of the two-dimensional slice image can be calculated by python programming, and the two-dimensional parameter matrix is formed and added into the image, so that the two-dimensional parameter matrix becomes a two-channel image containing physical information and is input into a PhyCNN-transporter network;
constructing a PhyCNN-transform neural network architecture by using a Pytorch platform, wherein network hyper-parameter settings such as CNN convolution kernel size, number and the like are shown in a table 1;
TABLE 1
3500 (56%) samples in the sample set were selected as training sets, 1500 (24%) samples as validation sets, 1250 (20%) samples as test sets.
The PhyCNN-transporter neural network obtains a good prediction result after 100epoch training. Fig. 5 shows the loss curves of the PhyCNN-transducer model for the validation set at different batch sizes. R for verification set in connection with FIG. 6 2 And determining the optimal PhyCNN-transducer model by the characterization of the RMSE. The predicted results of the best PhyCNN-transporter model on the validation set and the test set are shown in FIG. 7.
Comparative example 1
The same training set, validation set and test set as PhyCNN-transducer were used.
A three-dimensional convolutional neural network (3D CNN) model based on pytorrch is established, and a three-dimensional porous medium digital image with a volume of 2563 voxels is compressed into a feature vector with 256 dimensions. Firstly, constructing an encoder through 4 3D convolution layers, a Batch normalization (Batch Norm) layer and a ReLU activation function, and extracting features of a digital image; then, compressing the extracted features into 256-dimensional feature vectors using a linear layer; finally, these feature vectors are input into 5 fully connected layers (including input and output layers) to predict permeability.
Wherein, the epoch used for 3D CNN model training is increased to 150, and the initial learning rate is 1e-3; at epoch of 20, 50, 80 and 110, respectively, the learning rate is set to 0.1, 0.05, 0.025 and 0.01 times the initial learning rate in this order;
r of 3D CNN on validation set when batch size is equal to 8 2 0.9844, rmse 0.0765;
r of training completed 3D CNN on test set 2 0.9670, rmse 0.1479;
comparing the predicted results of the 3D CNN model and the PhyCNN-transducer model on the verification set and the test set, the result shows that the predicted result of the 3D CNN model is relatively poor.
In addition, the training parameter number of the 3D CNN is 20.2M, and the training parameter number of the PhyCNN-transporter is 8.59M, so that the parameter number is reduced by 57.5%, and the required video memory is greatly saved. In terms of training speed, the PhyCNN-transporter model averages 29.0 minutes per epoch and the 3D CNN model averages 29.6 minutes per epoch when trained with the same GPU device. In addition, the 3D CNN model has the advantages that the training epoch required for the loss curve convergence is larger, and the training time is far longer than that of PhyCNN-transducer.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Finally, it should be noted that: the foregoing description is only a preferred embodiment of the present invention, and the present invention is not limited thereto, but it is to be understood that modifications and equivalents of some of the technical features described in the foregoing embodiments may be made by those skilled in the art, although the present invention has been described in detail with reference to the foregoing embodiments. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (9)
1. A porous medium permeability prediction method based on a transducer network is characterized by comprising the following steps of:
simulating a flow field by using a pore scale simulation method, and calculating the permeability of the three-dimensional porous medium;
the three-dimensional structure image of the porous medium is regarded as a space sequence consisting of two-dimensional slice images, and the physical attribute of each two-dimensional slice image is extracted;
adding a corresponding physical parameter matrix into the three-dimensional structure image of the porous medium, constructing a sample set by taking the permeability of the porous medium as a label value, and determining a training set, a verification set and a test set;
constructing a PhyCNN-transporter neural network model based on a convolutional neural network (Convolutional Neural Network, CNN) and a transporter network;
combining the training set with a gradient descent method, and training and optimizing parameters of a neural network model PhyCNN-transducer; the verification set is used for evaluating the prediction capability of the PhyCNN-transducer model in the training process; the test set was used to examine the final predictive effect of the PhyCNN-transporter model.
2. The method for predicting the permeability of the porous medium based on the Transformer network according to claim 1, wherein the method comprises the following steps: the pore scale simulation method includes a pore network model (Pore Network Model, PNM) and lattice boltzmann method (Lattice Boltzmann Method, LBM).
3. The method for predicting the permeability of a porous medium based on a Transformer network according to claim 1, wherein the three-dimensional structure image segmentation step of the porous medium is as follows:
and cutting the three-dimensional structure image of the porous medium into continuous two-dimensional images along any axis direction of a Cartesian space coordinate system, wherein the length of the porous medium on a cutting axis is the sequence length, and the thickness of the cut image is equal to the pixel size.
4. The method for predicting the permeability of the porous medium based on the Transformer network according to claim 1, wherein the method comprises the following steps:
the physical properties of the two-dimensional slice image comprise porosity and specific surface area parameters;
the physical attribute extraction method comprises the following steps: the 1 value of the porous medium two-dimensional slice gray level image represents a solid phase, the 0 value pixel of the porous medium two-dimensional slice gray level image represents a pore structure, the statistics of the pixel points of the slice image is realized by using python programming, and the porosity and the specific surface area are calculated.
5. The method for predicting the permeability of the porous medium based on the Transformer network according to claim 1, wherein the method comprises the following steps: the specific method for adding the corresponding physical parameter matrix into the three-dimensional structure image of the porous medium comprises the following steps: and establishing a two-dimensional parameter matrix with the same size as the two-dimensional slice image, wherein the upper half elements of the matrix are the porosity values of the slice, and the lower half elements of the matrix are the specific surface area values of the slice, and adding the parameter matrix as a single-channel image into the original gray image of the three-dimensional structure of the porous medium, so that the three-dimensional structure image of the porous medium contains physical parameter information.
6. The method for predicting the permeability of the porous medium based on the Transformer network according to claim 1, wherein the method comprises the following steps: the PhyCNN-transducer neural network model consists of a convolutional neural network CNN and a transducer network, and specifically comprises the following components:
the convolutional neural network CNN is provided with 4 two-dimensional convolutional layers, and a first full-connection layer of a single layer is tightly connected after the 4 convolutional layers;
the first full-connection layer of the convolutional neural network CNN is connected with the coding layer of the transducer network;
the Transformer network has 6 identical coding layers, each coding layer having 2 sublayers; the first sub-layer is a multi-head self-attention layer, the second sub-layer is a second full-connection layer, and residual connection and regularization operations are included between the multi-head self-attention layer and the second full-connection layer;
the convolution layer is used for generating a multi-channel feature map for each two-dimensional image in the sequence;
the first full connection layer is used for compressing the multi-channel feature map into a feature vector with a fixed dimension.
7. The method for predicting the permeability of a porous medium based on a Transformer network according to claim 1, wherein the regression prediction of the permeability comprises the following specific steps:
inputting a porous medium three-dimensional structure containing physical parameter information and represented by a two-dimensional image sequence into a two-dimensional convolution layer of a CNN model, and increasing the nonlinear characterization capability of a network by using a Batch Norm and ReLU activation function to generate a multi-channel feature map for each two-dimensional image in the sequence;
inputting the multi-channel feature map into a full connection layer, so as to compress the multi-channel feature map into a feature vector with fixed dimension; repeating this operation for each image in the sequence, thereby forming a sequence of feature vectors;
determining positional information for each item in the sequence using sinusoidal positional codes;
adding position codes into the feature vector sequence, inputting the position codes into a multi-head self-attention layer of a transducer network, associating different positions in the feature vector sequence, and re-characterizing the positions;
and (3) averaging in the dimension of the sequence length of the characteristic vector sequence after characterization, and inputting the characteristic vector sequence into a full-connection layer to carry out regression prediction on the permeability of the three-dimensional porous medium.
8. The method for predicting the permeability of the porous medium based on the Transformer network according to claim 7, wherein the method comprises the following steps: the sinusoidal position code:
wherein,,2i and 2i+1 represent the 2i and 2i+1 th elements embedded in the feature vector, pos is the global position of the feature vector in the sequence, PE represents the position code, d model Representing the number of features expected by the model, i.e. the dimension of the feature vectors in the sequence.
9. The method for predicting the permeability of a porous medium based on a Transformer network according to claim 7, wherein the step of re-characterizing the feature vector sequence is:
the multi-head self-attention mechanism introduces dimension d k Is a query vector and a key vector of (1) and has a dimension d v Is a vector of values of (a);
calculating the correlation between each feature vector in the sequence and the query vector by using a scoring function; the key vector is used for calculating the attention distribution, the value vector is used for calculating the aggregation information, and the characteristic vector adopts a key value representation;
the dot product of the query vector is calculated using all key vectors and divided byAnd then calculating the weight of the value vector by using a softmax function, wherein the matrix representation method comprises the following steps:
wherein Q is a query vector matrix, K is a key vector matrix, V is a value vector matrix, d k ＝d model /h, representing Q, K in the dimension of h subspaces (heads), K T Is the transpose of the key vector matrix;
multi-headed self-attention projects the query matrix Q, key matrix K, and value matrix V to d with different linear projections of the h subspaces (heads), respectively k 、d k And d v And (3) the output of each subspace (head) is spliced and projected again to obtain the final multi-head self-attention mechanism output:
wherein, the projections are all parameter matrixes,andweight matrix of linear transformation, i E [1, h]H represents the number of projections, i.e. the number of heads, subspaces of multi-head self-attention, d model Representing the number of features expected by the model, i.e. the dimension of the feature vector in the sequence, head i Representing the ith "head" of the multi-head self-attention mechanism. />
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310737138.9A CN116680988A (en) | 2023-06-21 | 2023-06-21 | Porous medium permeability prediction method based on Transformer network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310737138.9A CN116680988A (en) | 2023-06-21 | 2023-06-21 | Porous medium permeability prediction method based on Transformer network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116680988A true CN116680988A (en) | 2023-09-01 |
Family
ID=87781991
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310737138.9A Pending CN116680988A (en) | 2023-06-21 | 2023-06-21 | Porous medium permeability prediction method based on Transformer network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116680988A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118171548A (en) * | 2024-03-28 | 2024-06-11 | 南京工业大学 | Method for identifying equivalent permeability coefficient of cross section of flood plain aquifer based on convolutional neural network |
-
2023
- 2023-06-21 CN CN202310737138.9A patent/CN116680988A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118171548A (en) * | 2024-03-28 | 2024-06-11 | 南京工业大学 | Method for identifying equivalent permeability coefficient of cross section of flood plain aquifer based on convolutional neural network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109685743B (en) | Image mixed noise elimination method based on noise learning neural network model | |
| CN112541572B (en) | Residual oil distribution prediction method based on convolutional encoder-decoder network | |
| CN107784288B (en) | Iterative positioning type face detection method based on deep neural network | |
| CN110728219A (en) | 3D face generation method based on multi-column multi-scale graph convolution neural network | |
| CN111507357B (en) | Defect detection semantic segmentation model modeling method, device, medium and equipment | |
| CN112489164B (en) | Image coloring method based on improved depth separable convolutional neural network | |
| CN111381919A (en) | Forming datasets for inferring editable feature trees | |
| CN105260998A (en) | MCMC sampling and threshold low-rank approximation-based image de-noising method | |
| CN109657707B (en) | Image classification method based on observation matrix transformation dimension | |
| CN112381790A (en) | Abnormal image detection method based on depth self-coding | |
| CN112560966B (en) | Polarized SAR image classification method, medium and equipment based on scattering map convolution network | |
| CN112183742A (en) | Neural network hybrid quantization method based on progressive quantization and Hessian information | |
| CN113269224A (en) | Scene image classification method, system and storage medium | |
| CN116680988A (en) | Porous medium permeability prediction method based on Transformer network | |
| CN110930378A (en) | Emphysema image processing method and system based on low data demand | |
| CN112950570A (en) | Crack detection method combining deep learning and dense continuous central point | |
| CN116957422B (en) | Ecological environment evaluation method based on convolution self-coding and sharp point mutation model | |
| Gao et al. | Tetgan: A convolutional neural network for tetrahedral mesh generation | |
| CN117726183A (en) | Gas operation data prediction method based on space high-order convolution | |
| CN115953330B (en) | Texture optimization method, device, equipment and storage medium for virtual scene image | |
| CN117408167A (en) | Debris flow disaster vulnerability prediction method based on deep neural network | |
| CN115759291B (en) | Spatial nonlinear regression method and system based on ensemble learning | |
| CN117372617A (en) | Point cloud data reconstruction method based on GCN-converter model and electronic equipment | |
| CN117237663A (en) | Point cloud restoration method for large receptive field | |
| CN116721071A (en) | Industrial product surface defect detection method and device based on weak supervision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |