CN117476106B - Multi-class unbalanced protein secondary structure prediction method and system - Google Patents

Abstract

The present invention discloses a method and system for predicting the secondary structure of multiple types of unbalanced proteins, and relates to the technical field of computer-aided drug development. The method comprises obtaining a target protein sequence to be predicted, inputting the target protein sequence to be predicted into a pre-constructed Transformer-based multiple types of unbalanced protein secondary structure prediction model, obtaining a secondary structure prediction value of the target protein sequence to be predicted, and outputting a secondary structure prediction value of the target protein sequence. When predicting the secondary structure of a protein sequence, the method has low global dependence on amino acids and improves the prediction accuracy of the secondary structure of rare proteins.

Description

A multi-category unbalanced protein secondary structure prediction method and system

Technical field

The present invention relates to the technical field of computer-aided drug development, and in particular to a method and system for predicting secondary structures of multiple types of unbalanced proteins.

Background technique

Proteins are important organic compounds that constitute and repair human tissues. A large number of computational methods have emerged around the problem of protein secondary structure prediction. Commonly used methods include DeepACLSTM and MUFold-SS. The DeepACLSTM model uses protein sequence features and Profile features. The Profile feature is a sequence feature spectrum constructed from multiple sequence alignments. Here, it specifically refers to the Position Special Scoring Matrix (PSSM). Specifically, the convolution kernel is 1× The asymmetric convolutional network (ACNN) composed of 42 one-dimensional convolution and 3×1 two-dimensional convolution is combined with the bidirectional long short term memory (BiLSTM) network to obtain the relationship between amino acid residues. Local and global correlation; the feature matrix of the MUFold-SS model is composed of the physical and chemical properties of amino acids, the PSI-BLAST map and the HHBlits map. It contains rich evolutionary information and uses parallel convolution with convolution kernel scales of 1 and 3. A deep convolutional network is constructed to extract local and global correlations between amino acids. However, when predicting protein secondary structure, these two methods have the problem of strong global dependence on amino acids and low accuracy in predicting the secondary structure of rare proteins.

Contents of the invention

In order to solve the deficiencies in the above-mentioned background technology, the current technology mainly focuses on the problem that when predicting the secondary structure of proteins, the global dependence on amino acids is strong, and the prediction accuracy of the secondary structure of rare proteins is low. The present invention provides a method and system for predicting the secondary structure of multiple types of unbalanced proteins, which has low global dependence on amino acids and improves the accuracy of the prediction of the secondary structure of rare proteins.

In order to achieve the above objectives, the first aspect of the present invention provides a multi-category unbalanced protein secondary structure prediction method:

Obtain the target protein sequence to be predicted;

Input the target protein sequence to be predicted into the pre-constructed Transformer-based multi-class unbalanced protein secondary structure prediction model to obtain the secondary structure prediction value of the target protein sequence to be predicted; multi-class unbalanced protein structures are divided into 8 types: Represented by L, B, E, G, I, H, S and T respectively, but the proportion of B, G and S among these 8 structures is low, especially structure G, and rare classes are easily overwhelmed during the training process. , which reduces the prediction accuracy, so there is a multi-category imbalance problem in the protein secondary structure. Among them, the pre-built Transformer-based multi-category unbalanced protein secondary structure prediction models include:

A data preprocessing layer for weighting and batch normalization of input sample data;

A pytorch function layer used to process multiple preprocessed input sample data to obtain the first output matrix and the second output matrix;

The MobileNet v2 layer used to iteratively process the second output matrix to obtain the local feature matrix between protein sequences;

A Transformer layer used to process the third output matrix obtained by adding the first output matrix and the local feature matrix to obtain the correlation matrix between protein sequences;

A two-layer bidirectional gated cyclic unit layer used to process the correlation matrix between protein sequences and obtain the global feature matrix;

Convolutional layers and fully connected layers used to sequentially process the global feature matrix and obtain protein secondary structure prediction values;

Output the predicted secondary structure value of the target protein sequence to be predicted.

Optional steps for weighting and batch normalizing the input sample data include:

According to the feature matrix obtained by horizontally splicing the position-specific scoring matrix, the hidden Markov model feature matrix and the amino acid physicochemical property feature matrix, process the input sample data to obtain weighted sample data;

Get the mask matrix with all initial elements being false and the input feature matrix with all initial elements being 0 respectively;

According to the protein chain index, protein chain length and maximum protein chain length obtained by traversing the protein sequences of each batch in the weighted sample data, obtain the updated mask matrix and the updated input feature matrix;

Using the preset first formula and the second formula, calculate the mean and variance of the updated mask matrix and the updated input feature matrix respectively;

The batch normalization layer pre-constructed according to the preset third formula is used to process the mean and variance to obtain the batch normalization output matrix.

Optionally, the preset third formula includes:

Among them, μ represents the mean value obtained by using the preset first formula to process the updated mask matrix and the updated input feature matrix, and var represents using the preset second formula to process the updated mask matrix and the updated input. The variance obtained by the feature matrix, X _{B, C, max_L} represents the input weighted sample data, X ^* represents the output of the batch normalization layer, β and γ are the parameters to be learned in the batch normalization layer, where ε = 0.01;

Among them, the preset first formula and the preset second formula are respectively:

Among them, μ represents the mean of all sample data, var ₁ represents the variance of all sample data, M _bl represents the mask matrix, X _bcl is the element value of each amino acid vector in each protein sequence, B refers to the batch size set during training, that is, Batchsize, which is set to 16 here; b refers to the 16 proteins traversed in this batch, so b = [1, 16]; max_L refers to the maximum sequence length in the batch of proteins; L represents the sequence length of the batch of proteins;

Optional, the pytorch function layer includes the nn.Conv1d function, F.relu function, MaskedBatchNorm1d function and nn.Dropout function, which are connected in sequence. The parameters of the nn.Conv1d function include the size of the convolution, which is the number of output channels 57, and the number of input channels. 448, convolution kernel length 3, padding=1, bias bias=False, the input of the F.relu function is the output of the nn.Conv1d function, the dimension of the MaskedBatchNorm1d function is 448, and the neurons of the nn.Dropout function are not activated. The probability is 0.2.

Optionally, iteratively processing the second output matrix includes:

Using the second output matrix output by the nn.Dropout function as the initial input matrix of the MobileNet v2 layer to obtain an initial output matrix;

The initial output matrix is used as the input of the MobileNet v2 layer, and the intermediate output matrix processed by the MobileNet v2 layer is used as the input of the MobileNet v2 layer for multiple iterations to obtain the local feature matrix between protein sequences.

Optionally, the Transformer has two encoding layers, and each encoder is equipped with a set of eight attention head matrices.

Optionally, the output channels of the convolution layer and the fully connected layer are both 57, the convolution kernel of the convolution layer is 1, and the output size of the fully connected layer is 8.

Optionally, after the convolutional layer and fully connected layer used to sequentially process the global feature matrix to obtain the protein secondary structure prediction value, it also includes:

The prediction error is calculated using the label distribution aware marginal loss function, and the obtained prediction error is back-propagated to update the parameters β and γ of the batch normalization layer.

On the other hand, the present invention provides a multi-category unbalanced protein secondary structure prediction system, including:

Input module, used to obtain the target protein sequence to be predicted;

The processing module is used to input the target protein sequence to be predicted into the pre-built Transformer-based multi-type unbalanced protein secondary structure prediction model to obtain the secondary structure prediction value of the target protein sequence to be predicted; wherein, the pre-built based on Transformer's multi-category unbalanced protein secondary structure prediction models include:

A data preprocessing layer for weighting and batch normalizing the input sample data;

A pytorch function layer used to process multiple preprocessed input sample data to obtain the first output matrix and the second output matrix;

The MobileNet v2 layer used to iteratively process the second output matrix to obtain the local feature matrix between protein sequences;

A Transformer layer used to process the third output matrix obtained by adding the first output matrix and the local feature matrix to obtain the correlation matrix between protein sequences;

A two-layer bidirectional gated cyclic unit layer used to process the correlation matrix between protein sequences and obtain the global feature matrix;

Convolutional layers and fully connected layers are used to process the global feature matrix in sequence to obtain the predicted value of protein secondary structure;

The output module is used to output the secondary structure prediction value of the target protein sequence to be predicted.

Compared with the prior art, the beneficial effects of the present invention are:

The invention provides a multi-category unbalanced protein secondary structure prediction method and system. By inputting the target protein sequence to be predicted into a pre-constructed Transformer-based multi-category unbalanced protein secondary structure prediction model, the target to be predicted is obtained. Secondary structure prediction value of protein sequence; among them, the pre-built Transformer-based multi-category unbalanced protein secondary structure prediction model includes: a data preprocessing layer for weighting and batch normalization of input sample data; The pytorch function layer used to process multiple input sample data after preprocessing to obtain the first output matrix and the second output matrix; the MobileNet v2 used to iteratively process the second output matrix to obtain the local feature matrix between protein sequences The Transformer layer is used to process the third output matrix obtained by adding the first output matrix and the local feature matrix to obtain the correlation matrix between protein sequences; the Transformer layer is used to process the correlation matrix between protein sequences to obtain the global feature matrix. layer bidirectional gated cyclic unit layer; convolutional layer and fully connected layer used to sequentially process the global feature matrix to obtain protein secondary structure prediction values. When predicting protein secondary structure, this method has low global dependence on amino acids and improves the prediction accuracy of rare protein secondary structures.

Description of drawings

Figure 1 is a flow chart of a multi-category unbalanced protein secondary structure prediction method provided by the present invention;

Figure 2 is a histogram comparing the performance of a multi-category unbalanced protein secondary structure prediction method under different characteristics;

Figure 3 is a histogram comparing the performance of a multi-category unbalanced protein secondary structure prediction method under different BN models;

Figure 4 is a histogram comparing the performance of a multi-class unbalanced protein secondary structure prediction method with and without Transformer;

FIG. 5 is a schematic diagram of the structure of a multi-class imbalanced protein secondary structure prediction system.

Detailed ways

The present invention will be further described below with reference to specific embodiments and drawings, but the embodiments are not intended to limit the present invention.

Figure 1 is a flow chart of a multi-category unbalanced protein secondary structure prediction method provided by the present invention. As shown in Figure 1, the present invention provides a multi-category unbalanced protein secondary structure prediction method, including:

101. Obtain the target protein sequence to be predicted.

102. Input the target protein sequence to be predicted into the pre-constructed Transformer-based multi-type unbalanced protein secondary structure prediction model to obtain the secondary structure prediction value of the target protein sequence to be predicted. Among them, the pre-built Transformer-based multi-category unbalanced protein secondary structure prediction models include:

Data preprocessing layer for weighting and batch normalizing input sample data.

Among them, the steps of weighting and batch normalization processing of input sample data include:

Based on the feature matrix obtained from the horizontally spliced position-specific scoring matrix, hidden Markov model feature matrix, and amino acid physicochemical property feature matrix, the input sample data is processed to obtain weighted sample data.

Specifically, set the PSI-BLAST map convergence parameter e to 0.001, run the PSI-BLAST map for two iterations on the Uniref database, and generate a position-specific scoring matrix (PSSM) with a size of L×20, where L is the length of the protein sequence. .

Run the HHblits map for four iterations on the Uniprot20 database to generate a hidden Markov model (HMM) feature matrix of size L×30. Uniprot20 is a protein sequence database that contains sequence information of all species and all proteins in UniProtKB (Protein Knowledge Base), with a total of 70,432,686 protein sequences. It includes two parts: Swiss-Prot and TrEMBL. The Swiss-Prot part contains manually annotated high-quality protein sequences, while TrEMBL contains automatically annotated protein sequences and unverified predicted protein sequences, which can be obtained from

Download from http://wwwuser.gwdg.de/~compbiol/data/hhsuite/databases/hhsuite_dbs/old-releases/.

The obtained position-specific scoring matrix, hidden Markov model characteristic matrix and seven amino acid physical and chemical property characteristic matrices are spliced and fused. The amino acid physical and chemical properties include flake probability, helix probability, isoelectric point, hydrophobicity, van der Waals volume, and polarizability. and the graphic shape index to obtain a feature matrix with a size of L×57. The fused matrix of the three provides rich information for the model, effectively improving the prediction accuracy of the model.

Get the mask matrix with all initial elements as false and the input feature matrix with all initial elements as 0 respectively.

Obtain the protein chain index by traversing the protein sequences of each batch in the weighted sample data, and use the protein chain length and the maximum length of the protein chain to obtain an updated mask matrix and an updated input feature matrix.

Specifically, a mask matrix is introduced in the batch normalization layer of the network. Specifically, each batch of B protein chains is traversed to obtain the index batch_idx and length L of each protein chain in the batch and the maximum length max_L of the protein chain in the batch, and all elements in the mask matrix MB _{,max_L} are initialized to false, and all elements in the input feature matrix XB _{,C,max_L} are initialized to 0. For each row of _{MB,max_L} corresponding to each batch_idx of the mask matrix, elements 0 to ProteinLen-1 of each row of _{MB,max_L} are updated to True according to batch_idx and ProteinLen, where ProteinLen represents the true length of the protein sequence, and the final mask matrix MB _{,max_L} of each batch is obtained. For X _{B,C,max_L} , each batch_idx corresponds to a protein chain. According to batch_idx, X is filled into X _{B,C,max_L} according to X[batch_idx,ProteinLen]=X to obtain the final X _{B,C,max_L} ; specifically, in implementation, the dimension of input X is B×L×F, the dimension of Masks is B×L, the filled part in X is masked, the non-filled part is taken out and put into a new tensor, and the dimension of the new tensor is adjusted according to the length of the feature vector describing each amino acid, that is, num_features, and the mean variance in the batch normalization layer is calculated based on the new tensor.

In order to avoid the influence of 0-filled positions on the accuracy of feature extraction results at other positions in the subsequent feature extraction process, a mask matrix is introduced.

The preset first formula and second formula are used to calculate the mean and variance of the updated mask matrix and the updated input feature matrix respectively.

The batch normalization layer pre-constructed according to the preset third formula is used to process the mean and variance to obtain a batch normalized output matrix.

Specifically, the preset third formula includes:

Among them, μ represents the mean value obtained by using the preset first formula to process the updated mask matrix and the updated input feature matrix, and var represents using the preset second formula to process the updated mask matrix and the updated input. The variance obtained by the feature matrix, X _{B, C, max_L} represents the input weighted sample data, X ^* represents the output of the batch normalization layer, β and γ are the parameters to be learned in the batch normalization layer, ε = 0.01 .

Among them, the preset first formula and the preset second formula are respectively:

Among them, μ represents the mean of all sample data, var ₁ represents the variance of all sample data, M _bl represents the mask matrix, X _bcl is the element value of each amino acid vector in each protein sequence, B refers to the batch size set during training, that is, Batchsize, which is set to 16 here; b refers to the 16 proteins traversed in this batch, so b = [1,16]; max_L refers to the maximum sequence length in the batch of proteins; L represents the sequence length of the batch of proteins.

A pytorch function layer used to process multiple preprocessed input sample data to obtain the first output matrix and the second output matrix.

Among them, the pytorch function layer includes the nn.Conv1d function, F.relu function, MaskedBatchNorm1d function and nn.Dropout function connected in sequence, where the parameters of the nn.Conv1d function include the convolution size of 57 output channels, 448 input channels, 3 convolution kernel length, padding=1, bias=False, the input of the F.relu function is the output of the nn.Conv1d function, the dimension of the MaskedBatchNorm1d function is 448, and the probability of the neuron of the nn.Dropout function not being activated is 0.2.

Specifically, B samples that have been weighted and normalized are randomly selected from the training samples to form the input matrix X _{B, C, L.} X _{B, C, L} are used as inputs, and out1=nn.Conv1d ( 57, 448, 3, padding=1, bias=False), out=F.relu(out1), out=MaskedBatchNorm1d(448) and out=nn.Dropout(0.2), and the matrix X _{B, C, L} is obtained.

MobileNet v2 layer used to iteratively process the second output matrix to obtain the local feature matrix between protein sequences.

Among them, the second output matrix output by the nn.Dropout function is used as the initial input matrix of the MobileNet v2 layer to obtain the initial output matrix of the MobileNet v2 layer.

The initial output matrix processed by the MobileNet v2 layer is used as the input of the MobileNet v2 layer, and the intermediate output matrix processed by the MobileNet v2 layer is used as the input of the MobileNet v2 layer for multiple iterations to obtain the local feature matrix between protein sequences.

Specifically, the local feature extraction code of the MobileNet v2 network is run iteratively n times, and the matrix X _{B, C, L} obtained in the first iteration is used as input, and the i-th output is used as the i+1-th input, where i = 1, 2, 3, ..., n-1, to obtain the local feature matrix between amino acids in the sequence

The Transformer layer is used to process the third output matrix obtained by adding the first output matrix and the local feature matrix to obtain the correlation matrix between protein sequences.

Among them, the Transformer has two coding layers, and each encoder is equipped with a set of eight attention head matrices.

_Specifically , add _the local _{feature matrix}

A two-layer bidirectional gated cyclic unit layer used to process the correlation matrix between protein sequences and obtain the global feature matrix.

Specifically, a two-layer bidirectional gated recurrent unit (BiGRU) is run, and the correlation matrix X _E between amino acid sequences is used as input to obtain the global feature matrix X _G of the protein sequence.

Convolutional layers and fully connected layers are used to sequentially process the global feature matrix and obtain protein secondary structure prediction values.

Among them, the output channels of the convolution layer and the fully connected layer are both 57, the convolution kernel of the convolution layer is 1, and the output size of the fully connected layer is 8.

Specifically, run a one-dimensional convolution program with an output channel of 57 and a convolution kernel size of 1, take the global feature matrix X _G as input, _and obtain the final feature The connection layer program takes the final feature X _F as input to obtain the protein secondary structure prediction value P.

After the convolutional layer and fully connected layer used to sequentially process the global feature matrix and obtain the protein secondary structure prediction value, it also includes:

The label distribution-aware marginal loss function is used to calculate the prediction error, and the obtained prediction error is back-propagated to update the parameters β and γ of the batch normalization layer.

Specifically, the label distribution aware marginal loss function code is run to calculate the prediction error, and the obtained prediction error is backpropagated to update the parameters of the batch normalization layer, including β, λ and the weight of the network, and the maximum number of iteration steps is set to 200. The learning rate is set to 0.0001, the batch size is set to 16, and the model is iterated in a loop. When the prediction accuracy in the validation set no longer improves or reaches the maximum number of iteration steps, the training ends, and a multi-class imbalanced protein secondary structure prediction model based on Transformer is obtained. .

Table 1

Table 1 shows the comparison with state-of-the-art methods on the dataset CB513, with bold indicating the best performance. As can be seen from Table 1, the performance of the model in this paper is significantly better than other benchmark methods on the CB513 data set, which not only ensures the prediction accuracy of common classes among 8 types of secondary structures, but also the prediction effect of rare classes B, G, and S There has also been some improvement.

Table 2

Table 2 shows the comparison with state-of-the-art methods on the data set CASP12, bold indicates the best performance. On the data set CASP12, the overall prediction accuracy is 0.02% lower than MUFold_SS, but for rarer categories, such as categories B, S and G, the prediction accuracy is greatly improved compared with other methods, but due to the small number, Therefore, it has little impact on the overall prediction accuracy.

table 3

Table 3 shows the comparison with the state-of-the-art methods on the data set CASP13, bold indicates the best performance. On the data set CASP13, the overall prediction accuracy reaches the maximum, and the prediction accuracy for rare classes is also greatly improved. However, the prediction accuracy on categories L and T still lags behind other methods.

Table 4

Table 4 shows the comparison with the most advanced methods on the CASP14 dataset, and the best performance is indicated in bold. On the CASP14 dataset, the overall prediction accuracy is improved by 0.05% compared with MUFold_SS. The overall accuracy improvement is limited, but the prediction accuracy of rare classes is improved significantly. Observing the above results, it can be found that the lower the proportion (the smaller the sample size), the lower the Q ₈ accuracy of the structural class, especially the probability of the "I" class structure in the eight states is less than one thousandth, and almost all methods are difficult to predict correctly. There is reason to believe that if the sample size of low-frequency subclasses can be expanded, the prediction effect will be further improved. Therefore, data enhancement for protein secondary structure prediction is a direction worthy of research.

Figure 2 is a histogram comparing the performance of a multi-category unbalanced protein secondary structure prediction method under different characteristics;

Figure 3 is a performance comparison histogram of a multi-class unbalanced protein secondary structure prediction method under different BN models, where BN refers to the batch normalization layer, i.e., Batch Normalization;

Figure 4 is a histogram comparing the performance of a multi-category unbalanced protein secondary structure prediction method with and without a Transformer model.

As the information source and basis for structure prediction, different amino acid codes contain different evolutionary information. The exploration of feature representation will be carried out under the model parameters with the best prediction results in the above experiments. Here we first analyze the impact of different encoding methods on prediction accuracy one by one, and then conduct combined experiments on them. Figure 2 shows the impact of 9 different input features on the model's prediction performance based on Q ₈ accuracy on the validation set. Specifically, we first consider PSSM coding and HMM coding separately. Since one-hot coding and physical and chemical property coding have nothing to do with position and do not contain evolutionary information, they are not considered separately. From the experimental results, we can see that the performance of PSSM is better than HMM. Maybe PSSM contains richer evolutionary information. Then PSSM and HMM were combined with one-hot coding and physical and chemical property coding respectively, and PSSM coding and HMM coding were combined. We found that when PSSM coding was combined with one-hot coding and physical and chemical property coding respectively, the model prediction performance was close, while HMM When combined with these two separately, the performance is lower than that of PSSM combined with them. This also confirms the conclusion that PSSM contains richer information or is more suitable for secondary structure prediction. However, when combined with physical and chemical property encoding, the prediction performance is lower than One-hot encoding is not in line with expectations, but in subsequent experiments it can be seen that in the evolutionary process of proteins forming stable structures, due to the different properties of different amino acids, these properties affect the way they interact with surrounding amino acids in the sequence, thus Affects protein structure. When PSSM coding and HMM coding are combined, the prediction performance is significantly improved compared to the previous four. PSSM contains the same amino acid in different sequences, and the probability of mutation into other amino acids is reduced in the process of forming a stable protein structure, while HMM coding contains With the matching state probability, translation frequency and local diversity of different amino acids, it can be seen that the two contain certain complementary information useful for protein secondary structure prediction. Based on this, the combination of PSSM coding and HMM coding is combined with one-hot coding and physical and chemical property coding respectively. From the results, it can be seen that combining with physical and chemical properties is the best coding method, and the prediction accuracy reaches the highest, which also reaches our level. expected.

For the improved batch normalization (Batch Normalization) layer, as shown in Figure 3, after introducing the mask matrix on the test set CB513, the Q ₈ accuracy increased by 0.23%, and the F ₁ increased by about 0.05%. The characteristics of the filling position can be seen The vector may become a non-zero vector during the feature extraction process, which will affect the prediction results of the secondary structure corresponding to the amino acid at the non-filled position. The introduction of the mask matrix can alleviate this problem to a certain extent and improve the accuracy of feature extraction. This further improves the accuracy of secondary structure prediction.

Figure 4 shows the comparison results of network performance with and without feature enhancement modules. As shown in the figure, after adding the feature enhancement module to the network, the prediction accuracy increased by 0.16% and the F ₁ value increased by 0.79%. This shows that adding 2 layers of 8-head Transformer Encoder before long-distance action enhances the residues in the sequence. The expression of dependencies between them can make subsequent BiGRU more flexibly capture long-distance dependencies. In summary, the present invention obtains the secondary structure prediction value of the target protein sequence to be predicted by inputting the target protein sequence to be predicted into the pre-constructed Transformer-based multi-type unbalanced protein secondary structure prediction model. Among them, the pre-built Transformer-based multi-class unbalanced protein secondary structure prediction model includes: a data preprocessing layer for weighting and batch normalization of input sample data. A pytorch function layer used to process multiple preprocessed input sample data to obtain the first output matrix and the second output matrix. The MobileNet v2 layer is used to iteratively process the second output matrix and obtain the local feature matrix between protein sequences. Transformer layer used to process the third output matrix obtained by adding the first output matrix and the local feature matrix to obtain the correlation matrix between protein sequences. A two-layer bidirectional gated cyclic unit layer used to process the correlation matrix between protein sequences and obtain the global feature matrix. Convolutional layers and fully connected layers are used to sequentially process the global feature matrix and obtain protein secondary structure prediction values. When predicting protein secondary structure, this method has low global dependence on amino acids and improves the prediction accuracy of rare protein secondary structures.

103. Output the secondary structure prediction value P of the target protein sequence to be predicted.

Figure 5 is a schematic diagram of the structure 200 of a multi-type unbalanced protein secondary structure prediction system. As shown in Figure 5, the device includes:

Input module 201 is used to obtain the target protein sequence to be predicted.

The processing module 202 is used to input the target protein sequence to be predicted into the pre-constructed Transformer-based multi-type unbalanced protein secondary structure prediction model, and obtain the secondary structure prediction value of the target protein sequence to be predicted. Among them, the pre-built Transformer-based multi-category unbalanced protein secondary structure prediction models include:

Data preprocessing layer for weighting and batch normalizing input sample data.

A pytorch function layer used to process multiple preprocessed input sample data to obtain a first output matrix and a second output matrix.

MobileNet v2 layer used to iteratively process the second output matrix to obtain the local feature matrix between protein sequences.

Transformer layer used to process the third output matrix obtained by adding the first output matrix and the local feature matrix to obtain the correlation matrix between protein sequences.

A two-layer bidirectional gated cyclic unit layer used to process the correlation matrix between protein sequences and obtain the global feature matrix.

The convolutional layer and fully connected layer are used to process the global feature matrix in sequence to obtain the protein secondary structure prediction value.

The output module 203 is used to output the secondary structure prediction value of the target protein sequence to be predicted.

It should be understood by those skilled in the art that the present invention may take the form of a computer program product implemented on one or more computers containing available program codes and available storage media. Although the embodiments of the present invention have been shown and described, it is understood by those skilled in the art that various changes, modifications, substitutions and variations may be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims (6)

1. A method for predicting a secondary structure of a plurality of unbalanced proteins, comprising:

obtaining a target protein sequence to be predicted;

inputting a target protein sequence to be predicted into a pre-constructed multi-class unbalanced protein secondary structure prediction model based on a transducer to obtain a secondary structure prediction value of the target protein sequence to be predicted; the pre-constructed transducer-based multi-class unbalanced protein secondary structure prediction model comprises the following components:

a data preprocessing layer for performing weighting processing and batch normalization processing on the input sample data;

the pytorch function layer is used for processing the preprocessed multiple input sample data to obtain a first output matrix and a second output matrix;

the MobileNet v2 layer is used for iteratively processing the second output matrix to obtain a local feature matrix among protein sequences;

the transducer layer is used for processing a third output matrix obtained by adding the first output matrix and the local feature matrix to obtain an association matrix between protein sequences;

the two-layer bidirectional gating cyclic unit layer is used for processing the incidence matrix among protein sequences to obtain a global feature matrix;

the convolution layer and the full connection layer are used for sequentially processing the global feature matrix to obtain a protein secondary structure predicted value;

outputting a secondary structure predicted value of the target protein sequence to be predicted;

the pyrach function layer comprises an nn.Conv1d function, an F.relu function, a MaskedBatchNorm1d function and an nn.Dropout function which are connected in sequence, wherein parameters of the nn.Conv1d function comprise a convolution size of 57 output channels, 448 input channels, 3 convolution kernels, padding=1, bias bias=false, the input of the F.relu function is the output of the nn.Conv1d function, the dimension of the MaskedBatchNorm1d function is 448, and the probability that neurons of the nn.Dropout function are not activated is 0.2;

the coding layers of the transducer are two layers, and each coder is provided with a set of eight attention head matrixes;

the output channels of the convolution layer and the full connection layer are 57, the convolution kernel of the convolution layer is 1, and the output size of the full connection layer is 8.

2. The method of claim 1, wherein the step of weighting and batch normalizing the input sample data comprises:

processing input sample data according to a position specificity scoring matrix, a hidden Markov model feature matrix and a feature matrix obtained by an amino acid physicochemical property feature matrix of horizontal splicing to obtain weighted sample data;

respectively obtaining mask matrixes with all initial elements being false and input feature matrixes with all initial elements being 0;

according to protein chain indexes, protein chain lengths and protein chain maximum lengths obtained by traversing protein sequences of each batch in the weighted sample data, an updated mask matrix and an updated input feature matrix are obtained;

respectively calculating the mean value and the variance of the updated mask matrix and the updated input feature matrix by using a preset first formula and a preset second formula;

and processing the mean and variance by using a batch normalization layer pre-constructed according to a preset third formula to obtain a batch normalization output matrix.

3. The method for predicting a secondary structure of a plurality of unbalanced proteins according to claim 2, wherein the predetermined third formula comprises:

wherein μ represents the average value obtained by processing the updated mask matrix and the updated input feature matrix using a preset first formula, var represents the variance obtained by processing the updated mask matrix and the updated input feature matrix using a preset second formula, X _{B，C，max_L} Representing input weighted sample data, X ^* Representing the output of the batch normalization layer, β and γ being parameters to be learned in the batch normalization layer, where ε=0.01;

the preset first formula and the preset second formula are respectively as follows:

wherein μ represents the mean value of all sample data, var ₁ Representing the variance of all sample data, M _bl Represents a mask matrix, X _bcl Is the element value of each amino acid vector in each protein sequence, B is the batch size set during training, namely, batchsize, here set to 16; b refers to traversing the batch of 16 proteins, so b= [1,16]The method comprises the steps of carrying out a first treatment on the surface of the max_l refers to the batch of proteinsMaximum sequence length in the stroma; l represents the sequence length of the batch of proteins.

4. The method of claim 1, wherein iteratively processing the second output matrix comprises:

taking the second output matrix output by the nn. Dropout function as an initial input matrix of a MobileNet v2 layer to obtain an initial output matrix;

and (3) taking the initial output matrix as the input of the MobileNet v2 layer, and carrying out multiple iterations on the obtained intermediate output matrix processed by the MobileNet v2 layer as the input of the MobileNet v2 layer to obtain the local feature matrix among protein sequences.

5. A method for predicting a secondary structure of a multi-class unbalanced protein according to claim 3, wherein after the convolution layer and the full-connection layer for sequentially processing the global feature matrix to obtain the predicted value of the secondary structure of the protein, the method further comprises:

and calculating a prediction error by using a label distribution perception marginal loss function, and reversely propagating and updating parameters beta and gamma of the batch normalization layer by the obtained prediction error.

6. A multi-class unbalanced protein secondary structure prediction system, comprising:

the input module is used for acquiring a target protein sequence to be predicted;

the processing module is used for inputting the target protein sequence to be predicted into a pre-constructed multi-class unbalanced protein secondary structure prediction model based on a transducer to obtain a secondary structure predicted value of the target protein sequence to be predicted; the pre-constructed transducer-based multi-class unbalanced protein secondary structure prediction model comprises the following components:

a data preprocessing layer for performing weighting processing and batch normalization processing on the input sample data;

the pytorch function layer is used for processing the preprocessed multiple input sample data to obtain a first output matrix and a second output matrix;

the MobileNet v2 layer is used for iteratively processing the second output matrix to obtain a local feature matrix among protein sequences;

the transducer layer is used for processing a third output matrix obtained by adding the first output matrix and the local feature matrix to obtain an association matrix between protein sequences;

the two-layer bidirectional gating cyclic unit layer is used for processing the incidence matrix among protein sequences to obtain a global feature matrix;

the convolution layer and the full connection layer are used for sequentially processing the global feature matrix to obtain a protein secondary structure predicted value;

the output module is used for outputting the predicted value of the secondary structure of the target protein sequence to be predicted;

the pyrach function layer comprises an nn.Conv1d function, an F.relu function, a MaskedBatchNorm1d function and an nn.Dropout function which are connected in sequence, wherein parameters of the nn.Conv1d function comprise a convolution size of 57 output channels, 448 input channels, 3 convolution kernels, padding=1, bias bias=false, the input of the F.relu function is the output of the nn.Conv1d function, the dimension of the MaskedBatchNorm1d function is 448, and the probability that neurons of the nn.Dropout function are not activated is 0.2;

the coding layers of the transducer are two layers, and each coder is provided with a set of eight attention head matrixes;

the output channels of the convolution layer and the full connection layer are 57, the convolution kernel of the convolution layer is 1, and the output size of the full connection layer is 8.