KR20200044731A - Deep learning-based technology for pre-training deep convolutional neural networks - Google Patents

Abstract

The disclosed technology discloses systems and methods for reducing overfitting of neural network implementation models that process amino acid sequences and concomitant positional frequency matrices. The system produces a positively labeled supplemental training example sequence pair that includes from the starting position to the target amino acid position through the end position. The complementary sequence pair complements the pathogenic or positive missense training example sequence pair. It has the same amino acids in the reference and replacement amino acid sequences. The system includes logic circuitry to input the complementary training position frequency matrix (PFM) equal to the PFM of the positive or pathogenic missense at the matching start and end positions, with each complementary sequence pair. The system includes logic circuitry to attenuate the training impact of the training PFM while training the neural network implementation model by including a supplemental training example PFM in the training data.

Description

Deep learning-based technology for pre-training deep convolutional neural networks

Priority application

The present application was filed on October 15, 2018 with the following three PCT applications and three U.S. regular applications: (1) PCT patent application number PCT / US2018 / 055840 (name of invention: DEEP LEARNING-BASED TECHNIQUES FOR TRAINING DEEP CONVOLUTIONAL NEURAL NETWORKS ", filing date: October 15, 2018 (Agent number ILLM 1000-8 / IP-1611-PCT); (2) PCT patent application number PCT / US2018 / 055878 (Invention name: "DEEP CONVOLUTIONAL NEURAL NETWORKS FOR VARIANT CLASSIFICATION", filing date: October 15, 2018 (Agent management number ILLM 1000-9 / IP-1612-PCT) ); (3) PCT patent application number PCT / US2018 / 055881 (Invention name: "SEMI-SUPERVISED LEARNING FOR TRAINING AN ENSEMBLE OF DEEP CONVOLUTIONAL NEURAL NETWORKS", filing date: October 15, 2018 (Agent management number ILLM 1000-10 / IP-1613-PCT)); (4) US regular patent application number 16 / 160,903 (Invention name: "DEEP LEARNING-BASED TECHNIQUES FOR TRAINING DEEP CONVOLUTIONAL NEURAL NETWORKS", filing date: October 15, 2018 (Agent management number ILLM 1000-5 / IP-1611 -US)); (5) U.S. regular patent application number 16 / 160,986 (invention name: "DEEP CONVOLUTIONAL NEURAL NETWORKS FOR VARIANT CLASSIFICATION", filing date: October 15, 2018 (Agent management number ILLM 1000-6 / IP-1612-US)) ; And (6) US regular patent application number 16 / 160,968 (invention name: "SEMI-SUPERVISED LEARNING FOR TRAINING AN ENSEMBLE OF DEEP CONVOLUTIONAL NEURAL NETWORKS", filing date: October 15, 2018 (Agent management number ILLM 1000-7 / IP-1613-US)), and continues to claim priority in the U.S., part number 16 / 407,149 (name of invention: "DEEP LEARNING-BASED TECHNIQUES FOR PRE-TRAINING DEEP CONVOLUTIONAL NEURAL NETWORKS", filing date: 2019 Claim priority on May 8, 2016 (Agent Management Number ILLM 1010-1 / IP-1734-US). All three PCT applications and three U.S. regular applications claim the priority and / or benefit of the following four U.S. provisional applications listed below.

U.S. Provisional Patent Application No. 62 / 573,144 (Invention name: "TRAINING A DEEP PATHOGENICITY CLASSIFIER USING LARGE-SCALE BENIGN TRAINING DATA", filing date: October 16, 2017 (Agent management number ILLM 1000-1 / IP-1611-PRV )).

U.S. Provisional Patent Application No. 62 / 573,149 (Name of invention: "PATHOGENICITY CLASSIFIER BASED ON DEEP CONVOLUTIONAL NEURAL NETWORKS (CNNS)", filing date: October 16, 2017 (Agent management number ILLM 1000-2 / IP-1612-PRV) ).

United States Provisional Patent Application No. 62 / 573,153 (Name of invention: "DEEP SEMI-SUPERVISED LEARNING THAT GENERATES LARGE-SCALE PATHOGENIC TRAINING DATA", filing date: October 16, 2017 (Agent management number ILLM 1000-3 / IP-1613- PRV)).

U.S. Provisional Patent Application No. 62 / 582,898 (Name of invention: "PATHOGENICITY CLASSIFICATION OF GENOMIC DATA USING DEEP CONVOLUTIONAL NEURAL NETWORKS (CNNs)", filing date: November 7, 2017 (Agent number ILLM 1000-4 / IP-1618- PRV)).

absorption

The following is incorporated herein for all purposes as if the entire contents were fully described herein:

U.S. Provisional Patent Application No. 62 / 573,144 (Invention name: "TRAINING A DEEP PATHOGENICITY CLASSIFIER USING LARGE-SCALE BENIGN TRAINING DATA", Inventor: Hong Gao, Kai-How Farh, Laksshman Sundaram and Jeremy Francis McRae, Filed: 10, 2017 Month 16 (Agent number ILLM 1000-1 / IP-1611-PRV).

U.S. Provisional Patent Application No. 62 / 573,149 (Invention name: "PATHOGENICITY CLASSIFIER BASED ON DEEP CONVOLUTIONAL NEURAL NETWORKS (CNNS)", Inventor: Laksshman Sundaram, Kai-How Farh, Hong Gao, Samskruthi Reddy Padigepati and Jeremy Francis McRae, filing date: October 16, 2017 (Agent number ILLM 1000-2 / IP-1612-PRV).

U.S. Provisional Patent Application No. 62 / 573,153 (Invention Name: "DEEP SEMI-SUPERVISED LEARNING THAT GENERATES LARGE-SCALE PATHOGENIC TRAINING DATA", Inventor: Hong Gao, Kai-How Farh, Laksshman Sundaram and Jeremy Francis McRae, Filed: 2017 October 16 (Agent number ILLM 1000-3 / IP-1613-PRV).

U.S. Provisional Patent Application No. 62 / 582,898 (Invention Name: "PATHOGENICITY CLASSIFICATION OF GENOMIC DATA USING DEEP CONVOLUTIONAL NEURAL NETWORKS (CNNs)", Inventor: Hong Gao, Kai-How Farh, Laksshman Sundaram, Filed: November 7, 2017 (Agent number ILLM 1000-4 / IP-1618-PRV).

PCT patent application number PCT / US18 / 55840 (invention name: "DEEP LEARNING-BASED TECHNIQUES FOR TRAINING DEEP CONVOLUTIONAL NEURAL NETWORKS", inventor: Hong Gao, Kai-How Farh, Laksshman Sundaram and Jeremy Francis McRae, filing date: 2018 10 Month 15 (Agent number ILLM 1000-8 / IP-1611-PCT).

PCT patent application number PCT / US2018 / 55878 (invention name: "DEEP CONVOLUTIONAL NEURAL NETWORKS FOR VARIANT CLASSIFICATION", inventor: Laksshman Sundaram, Kai-How Farh, Hong Gao, Samskruthi Reddy Padigepati and Jeremy Francis McRae, filing date: 2018 10 May 15 (Agent number ILLM 1000-9 / IP-1612-PCT).

PCT patent application number PCT / US2018 / 55881 (invention name: "SEMI-SUPERVISED LEARNING FOR TRAINING AN ENSEMBLE OF DEEP CONVOLUTIONAL NEURAL NETWORKS", inventor: Laksshman Sundaram, Kai-How Farh, Hong Gao and Jeremy Francis McRae, filing date: 2018 15 October (Agent #ILLM 1000-10 / IP-1613-PCT).

U.S. regular patent application number 16 / 160,903 (Invention name: "DEEP LEARNING-BASED TECHNIQUES FOR TRAINING DEEP CONVOLUTIONAL NEURAL NETWORKS '', Inventor: Hong Gao, Kai-How Farh, Laksshman Sundaram and Jeremy Francis McRae, filed: October 2018 15th (Agent number ILLM 1000-5 / IP-1611-US).

U.S. regular patent application number 16 / 160,986 (invention name: "DEEP CONVOLUTIONAL NEURAL NETWORKS FOR VARIANT CLASSIFICATION", inventor: Laksshman Sundaram, Kai-How Farh, Hong Gao and Jeremy Francis McRae, filing date: October 15, 2018 (Agent) Management number ILLM 1000-6 / IP-1612-US)).

U.S. regular patent application number 16 / 160,968 (Invention name: "SEMI-SUPERVISED LEARNING FOR TRAINING AN ENSEMBLE OF DEEP CONVOLUTIONAL NEURAL NETWORKS", inventor: Laksshman Sundaram, Kai-How Farh, Hong Gao and Jeremy Francis McRae, filing date: 2018 October 15 (Agent number ILLM 1000-7 / IP-1613-US).

Literature 1-AVD Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu, "WAVENET: A GENERATIVE MODEL FOR RAW AUDIO," arXiv: 1609.03499, 2016;

. Arik, M. Chrzanowski, A. Coates, G. Diamos, A. Gibiansky, Y. Kang, X. Li, J. Miller, A. Ng, J. Raiman, S. Sengupta and M. Shoeybi, "DEEP VOICE: REAL-TIME NEURAL TEXT-TO-SPEECH," arXiv:1702.07825, 2017;Document 2-S.

. Arik, M. Chrzanowski, A. Coates, G. Diamos, A. Gibiansky, Y. Kang, X. Li, J. Miller, A. Ng, J. Raiman, S. Sengupta and M. Shoeybi, "DEEP VOICE: REAL-TIME NEURAL TEXT-TO-SPEECH, "arXiv: 1702.07825, 2017;

Document 3-F. Yu and V. Koltun, "MULTI-SCALE CONTEXT AGGREGATION BY DILATED CONVOLUTIONS," arXiv: 1511.07122, 2016;

Document 4-K. He, X. Zhang, S. Ren, and J. Sun, "DEEP RESIDUAL LEARNING FOR IMAGE RECOGNITION," arXiv: 1512.03385, 2015;

Document 5-R.K. Srivastava, K. Greff, and J. Schmidhuber, "HIGHWAY NETWORKS," arXiv: 1505.00387, 2015;

Document 6-G. Huang, Z. Liu, L. van der Maaten and K. Q. Weinberger, "DENSELY CONNECTED CONVOLUTIONAL NETWORKS," arXiv: 1608.06993, 2017;

Literature 7-C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, "GOING DEEPER WITH CONVOLUTIONS," arXiv: 1409.4842 , 2014;

Document 8-S. Ioffe and C. Szegedy, "BATCH NORMALIZATION: ACCELERATING DEEP NETWORK TRAINING BY REDUCING INTERNAL COVARIATE SHIFT," arXiv: 1502.03167, 2015;

, "DILATED CONVOLUTIONAL NEURAL NETWORKS FOR CARDIOVASCULAR MR SEGMENTATION IN CONGENITAL HEART DISEASE," arXiv:1704.03669, 2017;Document 9-JM Wolterink, T. Leiner, MA Viergever, and I.

, "DILATED CONVOLUTIONAL NEURAL NETWORKS FOR CARDIOVASCULAR MR SEGMENTATION IN CONGENITAL HEART DISEASE," arXiv: 1704.03669, 2017;

Document 10-L. C. Piqueras, "AUTOREGRESSIVE MODEL BASED ON A DEEP CONVOLUTIONAL NEURAL NETWORK FOR AUDIO GENERATION," Tampere University of Technology, 2016;

Document 11-J. Wu, "Introduction to Convolutional Neural Networks," Nanjing University, 2017;

Document 12-I. J. Goodfellow, D. Warde-Farley, M. Mirza, A. Courville, and Y. Bengio, "CONVOLUTIONAL NETWORKS", Deep Learning, MIT Press, 2016; And

Literature 13-J. Gu, Z. Wang, J. Kuen, L. Ma, A. Shahroudy, B. Shuai, T. Liu, X. Wang, and G. Wang, "RECENT ADVANCES IN CONVOLUTIONAL NEURAL NETWORKS," arXiv : 1512.07108, 2017.

Document 1 describes a group of residual blocks, a batch normalization layer, a rectified linear unit (ReLU) layer, a dimensional change layer (with a convolution filter having the same convolution window size) dimensionality altering layer, an atrus convolutional layer with an exponentially growing atrus convolution rate, a skip connection, and an output sequence that accepts the input sequence and scores the entry of the input sequence. Describes a deep convolutional neural network architecture using a softmax classification layer. The disclosed technology uses the neural network components and parameters described in Document 1. In one embodiment, the disclosed technology modifies the parameters of neural network components described in Document 1. For example, unlike document 1, the atrus convolution rate in the disclosed technique proceeds non-exponentially from a low residual block group to a high residual block group. As another example, unlike Document 1, the convolution window size in the disclosed technique varies between groups of residual blocks.

Literature 2 describes the details of the deep convolutional neural network architecture described in Literature 1.

Document 3 describes an atrus convolution used by the disclosed technology. As used herein, atrus convolution is also referred to as “dilated convolution”. The atrus / expansion convolution allows for large receptacles with few trainable parameters. The atrus / expansion convolution is a convolution in which the kernel is applied over an area greater than its length by skipping input values with a predetermined step, also called atrus convolution rate or expansion factor. The atrus / expansion convolution adds a gap between elements of the convolution filter / kernel so that adjacent input entries (e.g., nucleotides, amino acids) are considered at a wider interval when the convolution operation is performed. This makes it possible to incorporate long-distance context dependencies into the input. Atrus convolution preserves partial convolution calculations for reuse when adjacent nucleotides are processed.

Document 4 describes residual blocks and residual connections used by the disclosed techniques.

Document 5 describes a skip connection used by the disclosed technique. As used herein, a skip connection is also referred to as a "highway network."

Document 6 describes a tightly coupled convolutional network architecture used by the disclosed technology.

Document 7 describes a dimensional change convolutional layer and module based processing pipeline used by the disclosed technology. An example of a dimension change convolution is 1 × 1 convolution.

Document 8 describes a batch normalization layer used by the disclosed technique.

Document 9 also describes the atrus / expansion convolution used by the disclosed technology.

Document 10 describes various architectures of deep neural networks that can be used by the disclosed techniques, including convolutional neural networks, deep convolutional neural networks, and deep convolutional neural networks with atlas / expansion convolution.

Document 11 describes the details of a convolutional neural network that can be used by the disclosed technology, including algorithms for training a convolutional neural network with subsampling layers (e.g., pooling) and fully connected layers. do.

Document 12 describes details of various convolutional operations that can be used by the disclosed technology.

Document 13 describes various architectures of convolutional neural networks that can be used by the disclosed technology.

Technical field

The disclosed technology includes artificial intelligent computers and digital data processing systems and corresponding data processing methods, including systems for uncertain reasoning (eg, fuzzy logic systems), adaptive systems, machine learning systems and artificial neural networks. It relates to products for intelligent emulation (ie knowledge-based systems, reasoning systems and knowledge acquisition systems). In particular, the disclosed technology relates to using deep learning based techniques for training deep convolutional neural networks. In particular, the disclosed technique relates to pretraining deep convolutional neural networks to avoid overfitting.

The subject matter disclosed in this section should not be assumed to be prior art just for the reasons mentioned in this section. Similarly, problems related to the subject mentioned in this section or provided as background art should not be assumed to have been previously recognized in the prior art. The subject matter in this section also represents only other approaches that may correspond to implementations of the claimed technology.

Machine learning

In machine learning, input variables are used to predict output variables. Input variables, often referred to as features, are represented by X = (X ₁ , X ₂ , ..., X _k ), where each X _i (i ∈ 1, ..., k) is a feature . The output variable, often referred to as the response or dependent variable, is represented by Yi. The relationship between Y and the corresponding X can be expressed by the following general formula:

Y = f (X) + ∈

In the above equation, f is a function of the features (X ₁ , X ₂ , ..., X _k ), and ∈ is a random error term. The error term is independent of X and has an average value of zero.

In fact, feature X can be used without Y or without knowing the exact relationship between X and Y. Since the error term has an average value of 0, the goal is to estimate f.

는 ∈의 추정이고, 이는, 종종 블랙 박스라고 간주되며,

의 입력과 출력 간의 관계만이 알려져 있지만, 그것이 그렇게 기능하는 이유에 대한 질문은 답변되지 않은 것을 의미한다.In the above formula,

Is an estimate of ∈, which is often considered a black box,

Only the relationship between the input and output of is known, but the question of why it functions so means that it has not been answered.

는 학습을 사용하여 발견된다. 감독 학습과 비감독 학습은 이 작업을 위한 기계 학습에 사용되는 두 가지 방법이다. 감독 학습에서는, 표지된 데이터(labeled data)가 훈련에 사용된다. 입력과 대응 출력(=표지)을 표시함으로써, 함수

는 출력에 근접하도록 최적화된다. 비감독 학습에서는, 목표는 표지 없는 데이터로부터 숨겨진 구조를 찾는 것이다. 이 알고리즘은 입력 데이터의 정확도의 척도를 갖지 않아서, 감독 학습과 구별된다.function

Is discovered using learning. Supervised learning and non-supervised learning are the two methods used for machine learning for this task. In supervised learning, labeled data is used for training. Function by displaying input and corresponding output (= cover)

Is optimized to approximate the output. In non-supervised learning, the goal is to find hidden structures from unlabeled data. This algorithm does not have a measure of the accuracy of the input data, so it is distinguished from supervised learning.

Neural network

를 갖고, 출력층은 활성화 함수

를 갖는다. 연결에는 훈련 프로세스 동안 조정되는 숫자 가중치(예를 들어, w11, w21, w12, w31, w22, w32, v11, v22)가 있으므로, 인식할 이미지를 공급할 때 올바르게 훈련된 네트워크가 올바르게 응답한다. 입력층은 원시 입력을 처리하고, 숨겨진 층은, 입력층과 숨겨진 층 간의 연결의 가중치에 기초하여 입력층으로부터의 출력을 처리한다. 출력층은, 숨겨진 층으로부터 출력을 가져 와서 숨겨진 층과 출력층 간의 연결의 가중치에 기초하여 출력을 처리한다. 망은 피처 검출 뉴런의 다수의 층을 포함한다. 각 층은, 이전 층으로부터의 입력의 상이한 조합에 응답하는 많은 뉴런을 갖는다. 이들 층은, 제1 층이 입력 화상 데이터에서 프리미티브 패턴들의 세트를 검출하고 제2 층이 패턴들 중 패턴을 검출하고 제3 층이 이들 패턴 중 패턴을 검출하도록 구성된다.Neural networks are systems of interconnected artificial neurons (eg, a1, a2, a3) that exchange messages with each other. The illustrated neural network has 3 inputs, 2 neurons in the hidden layer, and 2 neurons in the output layer. Hidden layer activation function

And the output layer is the activation function

Have The connection has numeric weights that are adjusted during the training process (e.g., w11, w21, w12, w31, w22, w32, v11, v22), so a properly trained network responds correctly when supplying images to be recognized. The input layer processes the raw input, and the hidden layer processes the output from the input layer based on the weight of the connection between the input layer and the hidden layer. The output layer takes output from the hidden layer and processes the output based on the weight of the connection between the hidden layer and the output layer. The network contains multiple layers of feature detection neurons. Each layer has many neurons that respond to different combinations of inputs from the previous layer. These layers are configured such that the first layer detects a set of primitive patterns in the input image data, the second layer detects a pattern among the patterns, and the third layer detects a pattern among these patterns.

The neural network model is trained using training samples before being used to predict output for production samples. The predicted quality of the trained model is evaluated by using a test set of training samples that were not provided as input during training. If the model correctly predicts the output of the test sample, the model can be used to infer with high confidence. However, if the model does not correctly predict the output of the test sample, it can be said that the model is overfit to the training data and not generalized to the invisible test data.

Investigations applying deep learning in genomics can be found in the following publications:

T. Ching et al., Opportunities And Obstacles For Deep Learning In Biology And Medicine, www.biorxiv.org:142760, 2017;

T. Ching et al., Opportunities And Obstacles For Deep Learning In Biology And Medicine, www.biorxiv.org:142760, 2017;

Angermueller C, P

rnamaa T, Parts L, Stegle O. Deep Learning For Computational Biology. Mol Syst Biol. 2016;12:878;

Angermueller C, P

rnamaa T, Parts L, Stegle O. Deep Learning For Computational Biology. Mol Syst Biol. 2016; 12: 878;

Park Y, Kellis M. 2015 Deep Learning For Regulatory Genomics. Nat. Biotechnol. 33, 825826. (doi:10.1038/nbt.3313);

Park Y, Kellis M. 2015 Deep Learning For Regulatory Genomics. Nat. Biotechnol. 33, 825826. (doi: 10.1038 / nbt.3313);

Min, S., Lee, B. & Yoon, S. Deep Learning In Bioinformatics. Brief. Bioinform. bbw068 (2016);

Min, S., Lee, B. & Yoon, S. Deep Learning In Bioinformatics. Brief. Bioinform. bbw068 (2016);

Leung MK, Delong A, Alipanahi B et al. Machine Learning In Genomic Medicine: A Review of Computational Problems and Data Sets 2016; and

Leung MK, Delong A, Alipanahi B et al. Machine Learning In Genomic Medicine: A Review of Computational Problems and Data Sets 2016; and

Libbrecht MW, Noble WS. Machine Learning Applications In Genetics and Genomics. Nature Reviews Genetics 2015;16(6):321-32.

Libbrecht MW, Noble WS. Machine Learning Applications In Genetics and Genomics. Nature Reviews Genetics 2015; 16 (6): 321-32.

In the drawings, similar reference numerals generally refer to similar parts throughout the drawings. In addition, the drawings are not necessarily drawn to scale, but instead are generally emphasized to illustrate the principles of the disclosed technology. In the following description, various implementations of the disclosed technology will be described with reference to the following drawings.
1 is an architectural level schematic diagram of a system for reducing overfitting during training of a variant pathogenic predictive model using supplemental training examples;
2 shows an exemplary architecture of a deep residual network for pathogenic prediction referred to herein as “PrimateAI”;
3 is a schematic diagram of PrimateAI, a deep learning network architecture for pathogenic classification;
4 is a diagram showing an embodiment of the operation of a convolutional neural network;
5 is a block diagram for training a convolutional neural network according to one embodiment of the disclosed technology;
6 shows an exemplary missense variant and corresponding supplemental positive training example;
7 shows the disclosed prior training of the pathogenicity prediction model using a supplemental data set;
8 shows training of a pre-trained pathogenicity prediction model after a pre-training epoch;
9 shows the application of a trained pathogenic predictive model to evaluate non-labeled variants;
10 shows position frequency matrix starting points for exemplary amino acid sequences with pathogenic missense variants and corresponding supplemental positive training examples;
FIG. 11 depicts the position frequency matrix starting point for an exemplary amino acid sequence with positive missense variants and corresponding supplemental positive training examples;
12 is a diagram illustrating the configuration of a position frequency matrix for amino acid sequences of primates, mammals, and vertebrates;
13 shows an exemplary hot encoding of human reference amino acid sequences and human replacement amino acid sequences;
14 is a diagram showing an input example for a variant pathogenicity prediction model;
15 is a simplified block diagram of a computer system that can be used to implement the disclosed technology.

The following description is presented to enable those skilled in the art to manufacture and use the disclosed technology, and is provided in the context of specific applications and their requirements. Various modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the disclosed technology. will be. Accordingly, the disclosed technology is not intended to be limited to the illustrated embodiments, but is intended to follow the broadest scope consistent with the principles and features disclosed herein.

Introduction

The sections of this application are repeated in that incorporated by reference application literature to provide a background for the disclosed improvements. Previous application literature disclosed a deep learning system trained using non-human primate missense variant data as described below. Introduce the improvements disclosed before providing the background.

The inventors have observed empirically that some patterns of training sometimes over-emphasize the location frequency matrix input to the deep learning system. Overfitting the positional frequency matrix can reduce the ability of the system to distinguish typically positive amino acid missenses such as R-> K from harmful amino acid missenses such as R-> W. Supplementing the training set with specially selected training examples can improve training results by reducing or canceling overfitting.

A positively labeled supplemental training example includes the same location frequency matrix ("PFM") as a missense training example that may be unlabeled (and presumed to be pathogenic), labeled pathogenic, or positive. The intuitive effect of this supplementary positive training example is that reverse propagation training allows distinction between positive and pathogenic based on location frequency matrices.

Supplemental positive training examples are configured to contrast with pathogenic or unlabeled examples in the training set. Supplemental positive training examples may reinforce positive missense examples. In contrast, a pathogenic missense can be a selected pathogenic missense or an example produced in combination in a training set. The selected positive variant can be a synonymous variant that expresses the same amino acid from two different codons, two different trinucleotide sequences encoded for the same amino acid. When synonymous positive variants are used, they are not randomly constructed; Instead, it is selected from synonymous variants observed in sequenced, ie, sequenced populations. A synonym variant is more likely a human variant because there is more sequence data available to humans than other primates, mammals or vertebrates. The complementary positive training example has the same amino acid sequence in the reference and replacement amino acid sequences. Alternatively, the selected positive variant can simply be in the same position as the control training example. This can be as effective in offsetting overfitting as using a synonymous positive variant.

Using the complementary training example, it can be stopped after the initial training epoch or continued throughout the training because the examples accurately reflect nature.

Convolution  Neural network

As a background, convolutional neural networks are a special type of neural network. This is the fundamental difference between a densely connected layer and a convolutional layer. The dense layer learns the global pattern in the input feature space, while the convolutional layer learns the local pattern, and in the case of images, the pattern found in the small 2D window of the input. These key characteristics provide two interesting properties to the convolutional neural network: (1) the pattern learned by the convolutional layer is transform-invariant, and (2) the spatial layer of the pattern can be learned.

In connection with the first, the convolutional layer can recognize a pattern at an arbitrary position, for example, in the upper left corner, after learning a predetermined pattern in the lower right corner of the image. In a densely connected network, when a new pattern appears in a new location, it must be learned. Thus, this makes the convolutional neural network data efficient because it requires fewer training samples to learn expressions with generalization capabilities.

In relation to the second, the first convolutional layer can learn a small local pattern such as an edge, and the second convolutional layer learns a large pattern made up of features of the first convolutional layer. This allows convolutional neural networks to become increasingly complex and efficiently learn abstract visual concepts.

Convolutional neural networks learn highly nonlinear mapping by interconnecting artificial neuron layers arranged in many other layers with activation functions that depend on those layers. It includes one or more subsampling layers and one or more convolutional layers interspersed with nonlinear layers, which are usually followed by one or more fully connected layers. Each element of the convolutional neural network receives input from a set of features in the previous layer. Convolutional neural networks learn simultaneously because neurons of the same feature map have the same weight. This local shared weight reduces the complexity of the neural network so that when the multidimensional input data is input to the convolutional neural network, the convolutional neural network avoids the complexity of data reconstruction in the feature extraction and regression or classification process.

Convolution works in a 3D tensor called feature map with two spatial axes (height and width) and a depth axis (also called a channel axis). In the case of an RGB image, the dimension of the depth axis is 3, because the image has three color channels: red, green, and blue channels. For black and white photos, the depth is 1 (gray level). The convolution operation extracts a patch from its input feature map and applies the same transformation to all of these patches to produce an output feature map. This output feature map is still a 3D tensor and has a width and height. The depth can be arbitrary, because the output depth is a parameter of the layer, and different channels of that depth axis no longer exhibit a specific color as in the RGB input, but rather a filter. The filter encodes certain aspects of the input data, for example, at the height level, a single filter can encode the concept of "there is a face in the input".

For example, the first convolution layer takes a feature map of size 28, 28, 1 and outputs a feature map of size 26, 26, 32, and computes 32 filters on its input. Each of these 32 output channels includes a value of 26 × 26 grid, which is a filter's response map to the input, indicating the response of the corresponding filter pattern at different locations of the input. This is the meaning of the term feature map, all dimensions of the depth axis are features (or filters), and the 2D tensor output ([:,:, n]) is a 2D spatial map of the response of these filters to the input.

Convolution is the following two main parameters: (1) the size of the patch extracted from the input (they are typically 1 × 1, 3 × 3 or 5 × 5) and (2) the depth of the output feature map (number of filters) Is calculated by convolution). Often, these convolutions start at depth 32, continue to depth 64, and end at depth 128 or 256.

Convolution slides these 3x3 or 5x5 sized windows over the 3D input feature map, stops at any location, and 3D patches of surrounding features (shape (window_height, window_width, input_depth) ). Each of these 3D patches is then converted to a 1D vector of shape (output_depth) (via a tensor product with the same learning weight matrix called the convolution kernel). Subsequently, these vectors are spatially reassembled into a 3D output map of all shapes (height, width, output_depth). All spatial locations of the output feature map correspond to the same location of the input feature map (eg, the lower right corner of the output contains information about the lower right corner of the input). For example, for a 3x3 window, the vector output ([i, j ,:]) is from the 3D patch input ([i-1: i + 1, j-1: J + 1 ,:]). . The entire process is detailed in FIG. 4 (indicated by 400).

x + b)을 계산함으로써 입력 X와 W의 컨볼루션을 수행하며, 여기서, x는 X의 인스턴스이고, b는 편향이다. 컨볼루션 필터가 입력을 가로질러 슬라이딩하는 단차 크기를 보폭이라고 하며, 필터 면적(m × n)을 수용장(receptive field)이라고 한다. 동일한 컨볼루션 필터가 입력의 상이한 위치에 걸쳐 적용되며, 이는 학습되는 가중치의 수를 감소시킨다. 이것은, 또한, 위치 불변 학습을 가능하게 하며, 즉, 중요한 패턴이 입력에 존재하는 경우, 컨볼루션 필터는 시퀀스의 위치에 관계없이 그 패턴을 학습한다.The convolutional neural network includes a convolutional filter (weighted matrix) that is learned over many gradient update iterations during training and a convolutional layer that performs convolutional operations between input values. If (m, n) is the filter size and W is the weight matrix, the convolutional layer is the dot product (W

Convolution of inputs X and W by calculating x + b), where x is an instance of X and b is biased. The step size at which the convolution filter slides across the input is called the stride, and the filter area (m × n) is called the receptive field. The same convolution filter is applied across different locations of the input, which reduces the number of weights learned. This also enables position-invariant learning, that is, when an important pattern is present at the input, the convolution filter learns the pattern regardless of the position of the sequence.

Convolution  Neural Network Training

As another background, FIG. 5 shows a block diagram 500 for training a convolutional neural network according to one implementation of the disclosed technology. The convolutional neural network is adjusted or trained such that the input data leads to a specific output estimate. The convolutional neural network is adjusted using back propagation based on the comparison between the output estimate and the measured data until the output estimate gradually matches or approaches the ground truth.

Convolutional neural networks are trained by adjusting weights between neurons based on differences between actual data and actual output. This is mathematically explained by the following formula:

Where δ = (actual data)-(actual output)

In one embodiment, the training rule is defined by the following formula:

은 뉴런(m)의 연산된 현재 출력이고,

은 입력(n)이고,

는 학습률이다.In the above formula, the arrow indicates the update of the value, t _m is the target value of the neuron (m),

Is the calculated current output of the neuron (m),

Is the input (n),

Is the learning rate.

The intermediate step of training includes generating a feature vector from the input data using a convolutional layer. Starting from the output, we compute the gradient for each layer's weight. This is called reverse pass or reverse. The network weight is updated using a combination of the negative gradient and the previous weight.

In one implementation, the convolutional neural network uses a probabilistic gradient update algorithm (such as ADAM) that performs backpropagation of errors by gradient descent. An example of a back propagation algorithm based on a sigmoid function is described below:

In the above sigmoid function, h is the weighted sum calculated on the neuron. The sigmoid function has the following derivatives:

The algorithm includes calculating the activation of all neurons in the network, generating an output for the forward pass. The activation of neurons (m) in the hidden layer is described by the following formula:

This is done for all hidden layers to get the activation described by the formula below:

Then, errors and corrected weights are calculated for each layer. The error in the output is calculated using the following formula:

The weight of the hidden layer is calculated with the following formula:

The weight of the output layer is updated with the following formula:

The weight of the hidden layer is updated using the learning rate (α) with the following formula:

)에 대하여, 손실 함수는, 표적이 y인 경우,

를 예측하는 비용에 대하여 ℓ로서, 즉,

로서 정의된다. 예측 출력(

)은, 함수(f)를 사용하여 입력 피처 벡터(x)로부터 변환된다. 함수(f)는 컨볼루션 신경망의 가중치에 의해 파라미터화되며, 즉,

이다. 손실 함수는,

또는

로서 기술되며, 여기서, z는 입력 및 출력 데이터 쌍(x,y)이다. 그래디언트 하강 최적화는 다음 수식에 따라 가중치를 업데이트함으로써 수행된다:In one implementation, the convolutional neural network computes the error across all layers using gradient descent optimization. In this optimization, the input feature vector (x) and the predicted output (

For), the loss function, if the target is y,

ℓ for the cost of predicting, that is,

Is defined as Predictive output (

) Is converted from the input feature vector (x) using the function (f). The function f is parameterized by the weight of the convolutional neural network, that is,

to be. The loss function,

or

, Where z is the input and output data pair (x, y). The gradient descent optimization is performed by updating the weights according to the following formula:

In the above formula, α is the learning rate. In addition, the loss is calculated as the average over a set of n data pairs. The calculation ends when the learning rate α in linear convergence is sufficiently small. In another implementation, the gradient is calculated using only selected data pairs supplied to Nestorov's accelerated and adaptive gradients to inject computational efficiency.

In one implementation, the convolutional neural network calculates the cost function using stochastic gradient descent (SGD). SGD approximates the gradient with respect to the weight of the loss function by computing the gradient only from one randomized data pair (z _t ) described by the following equation:

이다. 다른 구현예에서, 컨볼루션 신경망은 유클리드 손실 및 소프트맥스 손실 등의 상이한 손실 함수를 사용한다. 추가 구현예에서는, 컨볼루션 신경망은 아담(Adam) 확률적 최적화기를 사용한다.In the above equation, α is the learning rate, μ is the momentum, and t is the current weighted state before the update. The rate of convergence of SGD is approximately when the learning rate α is sufficiently reduced for both fast and slow cases.

to be. In other implementations, the convolutional neural network uses different loss functions such as Euclidean loss and Softmax loss. In a further embodiment, the convolutional neural network uses an Adam stochastic optimizer.

Additional disclosures and descriptions of the convolutional layer, subsampling layer, and nonlinear layer are found in what is incorporated into the reference application literature along with convolutional examples and training descriptions by back propagation. In addition, architectural variants for the basic CNN technology are included in the reference material.

One variant for the repeated balanced sampling described above is to select the entire elite training set in one or two cycles, not 20 cycles. Only one or two training cycles or 3 to 5 training cycles were trained by semi-supervised training between well-known positive training examples and reliable classified predicted pathogenic variants that would be sufficient to constitute an elite training set. There can be enough distinction. Variations of the disclosed methods and apparatus to describe only one cycle or two cycles or a range of 3 to 5 cycles are disclosed herein, and the previously disclosed iterations are 1 or 2 or 3 to 5 cycles. It can be easily achieved by converting to cycles.

Deep learning in genomics

Some important contributions incorporated into the reference application literature are repeated here. Genetic variants can help explain many diseases. Every human has a unique genetic code and there are many genetic variants within a group of individuals. Most of the harmful genetic variants were depleted from the genome by natural selection. It is important to identify genetic variants that are pathogenic or potentially harmful. This will help researchers focus on genetic variants that may be pathogenic and accelerate the diagnosis and treatment of many diseases.

Modeling the properties and functional effects (eg, pathogenicity) of variants is an important but challenging task in the field of genomics. Despite the rapid development of functional genomic sequencing technology, interpreting the functional results of variants remains a big challenge due to the complexity of cell type transcription control systems.

Advances in biochemical technology over the past decades have emerged as a next generation sequencing (NGS) platform that rapidly generates genomic data at a much lower cost than ever. This overwhelmingly large amount of sequenced DNA is difficult to annotate. Supervised machine learning algorithms generally perform well when large amounts of labeled data are available. In the field of bioinformatics and a lot of other data-rich training, the process of creating an instance marker is expensive; Unlabeled instances are inexpensive and readily available. For scenarios where the amount of labeled data is relatively small and the amount of labeled data is quite large, semi-supervised learning represents a cost-effective alternative to manual labeling.

Opportunities to use semi-supervisory algorithms arise to construct a deep learning-based pathogenic classifier that accurately predicts the pathogenicity of variants. A database of pathogenic variants without human identification bias can be generated.

In the context of pathogenic classifiers, deep neural networks are a type of artificial neural network that continuously models high-level features using a number of nonlinear and complex transform layers. Deep neural networks provide feedback through back propagation that conveys the difference between the observed and predicted outputs to adjust the parameters. Deep neural networks have evolved with the availability of large sets of training data, the power of parallel and distributed computing, and sophisticated training algorithms. Deep neural networks have facilitated major advances in various areas such as computer vision, speech recognition, and natural language processing.

Convolutional neural networks (CNNs) and cyclic neural networks (RNNs) are components of deep neural networks. Convolutional neural networks have been particularly successful in recognizing images with architectures that include convolutional layers, nonlinear layers, and pooling layers. Cyclic neural networks are designed to use sequence information from input data through periodic connections between building blocks such as perceptrons, long-term short-term memory units, and gate cyclic units. In addition, many other emerging deep neural networks have been proposed in limited context, such as deep space-time neural networks, multidimensional cyclic neural networks, and convolutional automatic encoders.

The goal of training a deep neural network is to optimize the weighting parameters of each layer that gradually combines simple features into complex features to learn the best hierarchical representation from the data. The single cycle of the optimization process consists of: First, given a set of training data, the forward pass sequentially computes the output of each layer and propagates the functional signals through the network. The objective loss function in the final output layer measures the error between the deduced output and the specified marker. To minimize training errors, the reverse pass uses chain rules to propagate the error signal and compute gradients for all weights across the neural network. Finally, the weight parameter is updated using an optimization algorithm based on stochastic gradient descent. Batch gradient descent provides parameter updates for each entire data set, while stochastic gradient descent provides a probabilistic approximation by performing an update for each small set of data examples. Several optimization algorithms come from stochastic gradient descent. For example, the Adagrad and Adam training algorithms adaptively modify the learning rate based on the update frequency and moment of the gradient for each parameter while performing stochastic gradient descent.

Another key factor in training deep neural networks is normalization, which refers to strategies intended to avoid overfitting and thus achieve good generalization performance. For example, weight reduction adds a penalty term to the objective loss function so that the weight parameter converges to a smaller absolute value. Dropout randomly removes hidden units from the neural network during training and can be considered as an ensemble of possible subnetworks. To improve the dropout function, a modification of dropout and a new activation function maxout have been proposed for a cyclic neural network called rnnDrop. Batch normalization also provides a new method of normalization by learning each mean and variance as a parameter and normalizing the scalar features for each activation in the mini-batch.

Given that the sequenced data is multidimensional and high dimensional, deep neural networks have great potential for bioinformatics research due to their wide applicability and improved predictive capabilities. Convolutional neural networks have been adapted to solve sequence-based problems in genomics such as motif discovery, pathogenic variant identification and gene expression inference. Convolutional neural networks use a weight-sharing strategy that is particularly useful for studying DNA because it can capture sequence motifs, short and circulating local patterns in DNA that are believed to have important biological functions. A characteristic of convolutional neural networks is the use of convolution filters. Unlike traditional classification approaches based on sophisticatedly designed and manually crafted functions, convolutional filters perform adaptive learning of functions similar to the process of mapping raw input data into informational representations of knowledge. In this sense, the convolution filter acts as a series of motif scanners, because these sets of filters can recognize relevant patterns in the input and update themselves during the training process. Circulating neural networks can capture long-range dependence from sequence data of various lengths, such as protein or DNA sequences.

Thus, a powerful computational model for predicting the pathogenicity of variants can have great advantages for both basic science and transformation studies.

General polymorphism represents a natural experiment whose suitability has been tested by natural selection generation. Comparing the allele frequency distributions for human missense and synonymous substitutions, it has been found to reliably predict that if a missense variant is present with a high allele frequency in a non-human primate species, the variant is also under neutral selection in the human population. In contrast, common variants of distant species experience negative selection with increasing evolutionary distance.

We use common variants from six non-human primate species to train a semi-supervised deep learning network that accurately classifies clinical de novo missense mutations using only sequences. Using more than 500 known species, the primate lineage includes common variants sufficient to systematically model the effects of most human variants of unknown importance.

The human reference genome has more than 70 million potential protein-modifying missense substitutions, many of which are rare mutations whose impact on human health has not been characterized. These variants, of unknown importance, present challenges to genomic interpretation in clinical applications and are an obstacle to long-term sequencing of screened and individualized medications across populations.

Classifying common variants across various human populations is an effective strategy for identifying clinically positive variants, but the common variants available in modern humans are limited by bottlenecks in the distant past of human species. Humans and chimpanzees share 99% sequence identity, suggesting that natural selection acting on chimpanzee variants has the potential to model the effects of variants of the same state in humans. Since the average adhesion time for neutral polymorphism in the human population is a fraction of the species' divergence time, naturally occurring chimpanzee variants overlap with human variants, except for the rare haplotype types maintained by balancing selection. The non-mutable space is usually searched.

The recent availability of exome data aggregated from 60,706 humans allows testing this hypothesis by comparing allele frequency spectra for missense and synonymous mutations. The singleton variant of ExAC is almost identical to the expected 2.2: 1 missense: agreement ratio predicted by the de novo mutation after adjusting the mutation rate using a trinucleotide context, but at high allele frequencies The number of sense variants decreases due to filtering harmful variants by natural selection. A pattern of missense: consent rates across the allele frequency spectrum indicates that a significant proportion of missense variants with a population frequency of less than 0.1% are slightly detrimental, i.e. not pathogenic enough to ensure immediate removal from the population, and a more limited population It indicates that it is not neutral enough to be present at high allele frequencies, consistent with prior observations for the data. The results of these studies filter variants with allele frequencies greater than 0.1% to 1% likely to be positive for permeable genetic disease, with a few well-documented exceptions caused by balancing selection and founder effects. Supports extensive empirical practice by the diagnostic laboratory to be removed.

Repeating this analysis with a subset of human variants that are in the same state as the common chimpanzee variant (observed more than once in chimpanzee population sequencing), found that the missense: copper ratio was generally constant across the allele frequency spectrum. The strong allele frequency of these variants in the chimpanzee population has already passed through a natural selection sieve in the chimpanzee, and strong evidence that the neutral effect on the suitability of the human population is highly consistent in the selective pressure on the missense variant in both species Indicates that. The low missense: copper ratio observed in chimpanzees is consistent with the larger effective population size in the ancestral chimpanzee population, allowing more efficient filtering of slightly detrimental variants.

In contrast, the rare chimpanzee variant (observed only once in chimpanzee population sequencing) indicates a moderate decrease in missense: copper ratio at high allele frequencies. When simulating cohorts of the same size from human variation data, only 64% of variants observed once in this size cohort, allele frequencies greater than 0.1% in the general population compared to 99.8% for variants observed multiple times in the cohort It is presumed to have, indicating that not all of the rare chimpanzee variants have passed through the screening sieve. Overall, it is estimated that 16% of the identified chimpanzee missense variants have an allele frequency of less than 0.1% in the general population and are subject to negative selection at high allele frequencies.

Next, we characterize human variants that have the same status as those observed in other non-human primate species (Bonobo, Gorilla, Orangutan, Rhesus and Marmoset). Similar to chimpanzees, the missense variability is not slightly depleted at high allele frequencies, but the missense: copper ratio is observed to be approximately the same across the allele frequency spectrum, which is a minority rare variant (about 5% to 15%) It is expected because it contains. These results suggest that the selective force on the missense variant is almost consistent with the new world monkey, presumed to have deviated from the human ancestor lineage at least about 35 million years ago within the primate lineage.

Human missense variants in the same state as variants of other primates are strongly enriched for positive results in ClinVar. After excluding unknown or conflicting annotated variants, human variants with primate parallel orthologs are approximately 95% more likely to annotate as positive or similar positives in ClinVar compared to 45% of missense mutations in general. It is observed to be. Small fractions of ClinVar variants classified as pathogenic from non-human primates can be compared to fractions of pathogenic ClinVar variants observed by identifying rare variants from similar sized healthy human cohorts. A significant portion of these variants, annotated as pathogenic or pseudopathogenic, indicate that a large allele frequency database was classified before it appeared and can be classified differently today.

The field of human genomics has long relied on model organisms to infer the clinical effects of human mutations, but the long evolutionary distances to most genetically manageable animal models raise concerns about the extent to which these findings can be re-generalized to humans. . To investigate the consensus of natural selection for missense variants in species farther from humans, we extended the analysis beyond the primate lineage to distant the four additional mammalian species (rat, pig, goat, cow). Includes common variations from two species of vertebrates (chicken, zebrafish). In contrast to previous primate analysis, we observed a significant depletion of the missense variant at a common allele frequency, particularly at larger evolutionary distances, compared to the rare allele frequency, which indicates that a significant fraction of common missense variants of more distant species Indicates that human populations are experiencing negative choices. Nevertheless, the observation of missense variants in more distant vertebrates is still, as the fraction of common missense variants depleted by natural selection is much smaller than about 50% depletion for human missense variants at baseline. Increase the likelihood of positive results. Consistent with these results, the likelihood that human missense variants observed in rats, dogs, pigs, and cows are annotated as positive or similar positives in ClinVar, compared to 95% for primate mutations overall and 45% for ClinVar databases overall. It was found to be about 85%.

The presence of pairs of closely related species at various evolutionary distances also provides an opportunity to assess the functional consequences of fixed missense substitutions in the human population. Within a pair of closely related species (branch length <0.1) in the mammalian pedigree, it was observed that the fixed missense variant was depleted to a common allele frequency compared to the rare allele frequency, which is equivalent to a fixed substitution between species. It indicates that the part is non-neutral in humans, even within the primate lineage. Comparing the size of missense depletion indicates that the interspecies fixed substitution is significantly less neutral than the intraspecies polymorphism. Interestingly, interspecies mutations between closely related mammals are substantially less pathogenic in ClinVar compared to common polymorphisms within species (83% are likely to be annotated as positive or pseudopositive), indicating that these changes disrupt protein function. Rather, it suggests that it reflects the adjustment of protein function, which confers adaptive advantages by species.

The decisive importance of classifying a large number of possible and correct variants for unknown clinical applications has inspired many attempts to solve the problem of machine learning, but this effort has led to insufficient amounts of common human variants and screened databases. It has been largely limited by the uncertain tin quality in Esau. The variation of the six non-human primates contributes to more than 300,000 unique missense variants with largely positive results that do not overlap with the common human variation, greatly expanding the size of training datasets that can be used in machine learning approaches. .

Unlike the initial model using multiple ergonomic features and meta-classifiers, we apply a simple deep learning residual network that takes as input only the amino acid sequence next to the variant of interest and parallel homologous sequence alignment in other species. To provide information about the protein structure to the network, we train two separate networks to learn secondary structure and solvent accessibility from sequence only, and to subnetwork them further into the deep learning network to predict the effect on the protein structure. As integrated. By using the sequence as a starting point, we avoid potential bias in protein structure and functional domain annotation, which may be incompletely identified or applied inconsistently.

The inventor trained to include only variants with positive markers by isolating random unknown variants consistent with mutation rates and sequencing coverage versus primate variants that may be positive by initially training the ensemble of networks using semi-directed learning. Overcome set problems. The ensemble of these networks is seeded to the next iteration of the classifier by biasing towards an unknown variant with more predictable pathogenicity and taking a gradual step at each iteration to prevent the model from converging in advance with the suboptimal results. To score a complete set of unknown variants and to influence selection.

Common primate mutations also provide a clean validation dataset for evaluating existing methods that are completely independent of previously used training data that were difficult to objectively evaluate due to meta-classifier proliferation. The performance of our model was evaluated using four different classification algorithms (Sift, Polyphen-2, CADD, M-CAP) using 10,000 pending primate common variants. Since approximately 50% of all human missense variants are eliminated by natural selection with a common allele frequency, we have each in a set of randomly selected missense variants consistent with 10,000 reserved primate common variants by mutation rate. The 50th-percentile score for the classifier was calculated and the reserved primate common variant was evaluated using the threshold. The accuracy of the deep learning model of the present invention was far superior to other classifiers in this independent validation dataset, using a deep learning network trained only on human common variants, or using both human common variants and primate variants.

In a recent trio sequencing study, thousands of de novo mutations in neurodevelopmental disorder patients and healthy siblings were listed, and the strength of various classification algorithms in isolating de novo missense mutations in cases and controls could be evaluated. Have it. For each of the four classification algorithms, each de novo missense variant was scored in a case-to-control group, the p-value of the Wilcoxon rank sum test for the difference between the two distributions was reported, and the primate variant (p ~ 10 ^-33 It has been shown that the deep learning method trained on) performed much better than the other classifiers (p ~ 10 ^-13 ~ 10 ^-19 ) in these clinical scenarios. From previous estimates that about 1.3-fold enrichment of de novo missense variants and about 20% of missense variants produce a loss-of-function effect, than we would expect for this cohort, we found that the complete classifiers ranged from p to ^10-40 . Expect to separate into two classes with p values.

The accuracy of the deep learning classifier is scaled according to the size of the training dataset, and the variation data from each of the six primate species independently contributes to improving the accuracy of the classifier. The number and diversity of non-human primate species in existence, along with evidence that selective pressure on protein-modifying variants mostly coincide within the primate lineage, has led to millions of unknown importance of currently limiting systematic primate population sequencing, clinical genomic interpretation. It suggests as an effective strategy for classifying human variants. Of the 504 known non-human primate species, about 60% are endangered due to hunting and habitat loss, and this is an imperative for this unique and irreplaceable species and worldwide conservation efforts that benefit ourselves.

Although the entire genomic data cannot be used as exome data, it limits the ability to detect the effects of natural selection in the deep intron region, and also cryptic splice mutations far from the exon region It was possible to calculate the observed count versus the expected count. Overall, we observe 60% depletion in the cryptic splice mutation at> 50nt distance from the exon-intron boundary. The reduced signal may be a combination of a smaller sample size and total genomic data compared to the exome, and it may be more difficult to predict the impact of in-depth intron variants.

Terms

For the full text of all documents and similar materials cited herein, including but not limited to patents, patent applications, articles, books, papers, and web pages, refer to this specification, regardless of the format of these documents and similar materials Is used as. In the event that one or more of the incorporated literature and similar material differs or contradicts this application, including but not limited to definition terms, term usage, described techniques, and the like, this application takes precedence.

As used herein, the following terms have the indicated meaning.

Base refers to a nucleotide base or nucleotide, A (adenine), C (cytosine), T (thymine) or G (guanine).

The present application uses the terms "protein" and "converted sequence" interchangeably.

The present application uses the terms "codon" and "base triplet" interchangeably.

The present application uses the terms "amino acid" and "converted unit" interchangeably.

The present application uses the phrases "variant pathogenic classifier", "convolutional neural network based classifier for variant classification" and "deep convolutional neural network based classifier for variant classification" interchangeably.

The term "chromosome" refers to the heredity-bearing gene carrier of living cells derived from chromatin strands comprising DNA and protein components (especially histones). Conventional internationally recognized individual human genome chromosome numbering systems are used herein.

The term “site” refers to a unique position on the reference genome (eg, chromosome ID, chromosome position, and orientation). In some embodiments, a site can be a residue, a sequence tag, or the position of a segment on a sequence. The term "locus" can be used to refer to a nucleic acid sequence on a reference chromosome or a specific location of a polymorphism.

The term “sample” herein refers to a biological fluid, cell, tissue, organ, or organism, typically containing a nucleic acid, or a nucleic acid containing at least one nucleic acid sequence to be sequenced and / or phased. Refers to a sample derived from a mixture of them. These samples include sputum / oral fluid, amniotic fluid, blood, blood fractions, microneedle biopsy samples (eg, surgical biopsy, microneedle biopsy, etc.), urine, peritoneal fluid, pleural fluid, tissue explants, organ cultures, And any other tissue or cell preparation, or fractions or derivatives thereof or fractions or derivatives isolated therefrom. Samples are often taken from human subjects (eg, patients), but samples are taken from any organism having chromosomes, including, but not limited to, dogs, cats, horses, goats, sheep, cows, pigs, etc. You can. The sample can be used as it is when it is obtained from a biological source or subsequent to pretreatment to alter the properties of the sample. For example, such pretreatment may include preparing plasma from blood and diluting viscous fluids and the like. Pretreatment methods may also include filtration, precipitation, dilution, distillation, mixing, centrifugation, freezing, lyophilization, concentration, amplification, nucleic acid fragmentation, inactivation of interfering components, addition of reagents, dissolution, and the like. It is not limited.

The term "sequence" includes or refers to strands of nucleotides linked together. The nucleotides can be based on DNA or RNA. It should be understood that one sequence may contain multiple sub-sequences. For example, a single sequence (eg, of a PCR amplicon) can have 350 nucleotides. Sample reads, ie reads, may contain multiple subsequences within these 350 nucleotides. For example, a sample read can include, for example, first and second flanking subsequences having 20 to 50 nucleotides. The first and second flanking subsequences may be located on either side of a repeat segment having corresponding subsequences (eg, 40 to 100 nucleotides). Each of the flanking subsequences may include primer subsequences (eg, 10 to 30 nucleotides) or part of a primer subsequence). For ease of reading, the term “sequence” will be referred to as “sequence,” but it is understood that the two sequences are not necessarily separated from each other on the common strand. To distinguish the various sequences described herein, the sequences can be provided with different labels (eg, target sequence, primer sequence, flanking sequence, reference sequence, etc.). Other terms such as “alleles” may be given different labels to distinguish similar objects.

The term "paired-end sequencing" refers to a sequencing method that sequences both ends of a target fraction. Paired-terminal sequencing can facilitate gene fusion and novel transcription as well as genome rearrangement and detection of repeat segments. Paired-terminal sequencing methods are described in PCT Publication WO07010252, PCT Application Serial No. PCTGB2007 / 003798, and US Patent Application Publication No. US 2009/0088327, each of which is incorporated herein by reference. In one example, a series of operations can be performed as follows: (a) creating a cluster of nucleic acids; (b) linearizing the nucleic acids; (c) "reverse" the target nucleic acid on the flow cell surface by hybridizing the first sequencing primer and performing repeated cycles of expansion, scanning and deblocking as described above, and (d) synthesizing a complementary copy. And (e) linearize the resynthesized strand, (f) hybridize the second sequencing primer and perform repeat cycles of expansion, scanning and deblocking as described above. Reversal can be performed by delivering the reagents as described above for a single cycle of bridge amplification.

The term “reference genome” or “reference sequence” refers to any particular known genomic sequence of any organism, whether partial or complete, that can be used to refer to a sequence identified from a subject. For example, reference genomes used in human subjects and many other organisms can be found at the National Center for Biotechnology Information at ncbi.nlm.nih.gov. "Genome" refers to complete genetic information of an organism or virus expressed in a nucleic acid sequence. The genome includes both non-coding sequences of genes and DNA. The reference sequence can be larger than the read aligned to this sequence. For example, the reference sequence can be at least about 100 times or more, or at least about 1000 times or more, or at least about 10,000 times or more, or at least about 105 times or more, or at least about 106 times or more, or at least about 107 times or more. . In one example, the reference genomic sequence is that of the full-length human genome. In another example, the reference genomic sequence is limited to a particular human chromosome, such as chromosome 13. In some embodiments, the reference chromosome is a chromosomal sequence from human genome version hg19. The term reference genome is intended to cover such sequences, but such sequences may be referred to as chromosomal reference sequences. Other examples of reference sequences include chromosomes of any species, subchromosomal regions (such as strands), and the like, as well as genomes of other species. In various embodiments, the reference genome is a consensus sequence or other combination derived from multiple individuals. However, in certain applications, the reference sequence can be taken from a particular individual.

The term “lead” refers to the collection of sequence data describing a nucleotide sample or fraction of a reference. The term "lead" may refer to a sample lead and / or a reference lead. Typically, but not necessarily, reads represent short sequences of contiguous base pairs in a sample or reference. Reads can be symbolically represented by base pair sequences (in ATCG) of a sample or reference fraction. The reads can be stored in a memory device and processed appropriately to determine whether the reads match the reference sequence or meet other criteria. Reads can be obtained directly from a sequencing device or indirectly from stored sequence information about a sample. In some cases, reads are of sufficient length (eg, at least about 25 bp) that can be used to identify larger sequences or regions, for example, aligned and specifically assigned to chromosomal or genomic regions or genes. It is a DNA sequence.

Next-generation sequencing methods include, for example, sequencing by synthetic techniques (Illumina), pyrosequencing 454, ion semiconductor technology (Ion Torrent sequencing), single-molecule real-time sequencing (Pacific Biosciences), and sequencing by ligation (SOLiD sequencing). Depending on the sequencing method, the length of each read can be varied to exceed about 30 bp to 10,000 bp. For example, an Illumina sequencing method using an SOLiD sequencer produces a nucleic acid read of about 50 bp. In another example, ion torrent sequencing produces a nucleic acid read of up to 400 bp, and 454 pyrosequencing produces a nucleic acid read of about 700 bp. In another example, a single-molecule real-time sequencing method can generate reads from 10,000 bp to 15,000 bp. Thus, in certain embodiments, the length of the nucleic acid sequence read is 30 bp to 100 bp, 50 bp to 200 bp, or 50 np to 400 bp in length.

The terms "sample read", "sample sequence" or "sample fraction" refer to sequence data for a genomic sequence of interest from a sample. For example, sample reads contain sequence data from PCR amplicons with forward and reverse primer sequences. Sequence data can be obtained from any selected sequence method. Sample reads may be, for example, from sequencing by synthesis (SBS) reactions, sequencing reactions by ligation, or from any other suitable sequencing method, for which It is necessary to determine the length and / or identity. The sample read can be a consensus (eg, average or weighted) sequence derived from multiple sample leads. In certain embodiments, providing a reference sequence includes identifying a locus of interest based on the primer sequence of the PCR amplicon.

The term "raw fraction" refers to sequence data for a portion of a genomic sequence of interest that at least partially overlaps a designated or secondary position of interest in a sample lead or sample fraction. Non-limiting examples of raw fractions include double stitch fractions, single stitch fractions, double unstitch fractions, and single unstitch fractions. The term “raw” refers to sequence data in which the raw fraction is partially related to the sequence data in the sample read, regardless of whether the raw fraction corresponds to a potential variant of the sample read and indicates a variant that certifies or identifies such potential variant. It is used to indicate that it contains. The term "raw fraction" does not indicate that the fraction necessarily includes a supporting variant that validates the variant call in the sample read. For example, when a sample read is determined by a variant call application to indicate a first variant, the variant call application can be expected to occur if one or more primitive fractions are different, which may occur if a variant of the sample read is given. It can be determined that it does not have a "support" variant of.

The terms “mapping”, “aligned”, “aligned”, or “aligning” refer to the process of comparing a read or tag to a reference sequence to determine whether the reference sequence comprises a lead sequence. If the reference sequence comprises a read, the read can be mapped to a reference sequence, or in certain embodiments to a specific position in the reference sequence. In some cases, alignment simply indicates whether the read is a member of a particular reference sequence (ie, whether the read is present or absent from the reference sequence). For example, alignment of a read with respect to a reference sequence for human chromosome 13 will indicate whether a read is present in the reference sequence for chromosome 13. The tool that provides this information is called the set membership tester. In some cases, alignment further indicates the position of the reference sequence to which the read tag is mapped. For example, if the reference sequence is an entire human genome sequence, alignment may further indicate that the read is on chromosome 13 and further indicate that the read is at a site of a particular strand and / or chromosome 13.

The term "indel" refers to the insertion and / or deletion of a base in an organism's DNA. Micro-indels represent indels resulting in a net change of 1 to 50 nucleotides. In the coding region of the genome, unless the length of the indel is a multiple of 3, it will generate a frameshift mutation. Indels can be contrasted with point mutations. Indels insert and delete nucleotides from sequences, while point mutations are substitution forms that replace one of the nucleotides without changing the total number of DNA. Indels can also be contrasted with a Tandem Base Mutation (TBM), which can be defined as a substitution in adjacent nucleotides (mainly corresponding to a substitution in two adjacent nucleotides, but a substitution in three adjacent nucleotides). Was observed).

The term "variant" refers to a nucleic acid sequence that is different from a nucleic acid reference. Typical nucleic acid sequence variants include single nucleotide polymorphism (SNP), short deletion and insertion polymorphism (Indel), copy number variation (CNV), microsatellite marker, or short tandem repeat and structural variation. Include without limitation. Somatic variant calls are an effort to identify variants present at low frequency in a DNA sample. Somatic variant calls are important in the context of cancer treatment. Cancer is caused by accumulation of mutations in DNA. DNA samples from tumors are generally heterogeneous, including some normal cells, some cells in the early stages of cancer progression (less mutations), and some late stage cells (high mutations). Because of this heterogeneity, when sequencing tumors (eg, from FFPE samples), somatic mutations often appear at low frequency. For example, SNV can only be seen in 10% of the leads that cover a given base. Variants classified as somatic or germ cells by the variant classifier are also referred to herein as “variants under test”.

The term “noise” refers to a false variant call due to one or more errors in the sequencing process and / or variant call application.

The term "variant frequency" refers to the expression of the relative frequency of alleles (variants of genes) at specific loci in a population expressed as fractions or percentages. For example, the fraction or percentage may be the fraction of all chromosomes in a population carrying the allele of interest. For example, the frequency of sample variants is the relative of the allele / variant at a particular locus / position along the genomic sequence of interest to the “population” corresponding to the number of samples and / or reads obtained for the genomic sequence of interest from the individual. Frequency. In another example, the baseline variant frequency refers to the relative frequency of alleles / variants at a particular locus / position along one or more baseline genomic sequences, where “population” is one or more baseline genomes from a population of normal individuals. It corresponds to the number of samples and / or reads obtained for the sequence.

The term “variant allele frequency” (VAF) refers to the percentage of sequenced reads observed to match the value of the variant divided by the total coverage at the target location. VAF is a measure of the percentage of sequenced reads that deliver variants.

The terms "position", "designated position" and "left" refer to the position or coordinates of one or more nucleotides within the sequence of nucleotides. The terms "position", "designated position" and "left" also refer to the position or coordinates of one or more base pairs in the sequence of nucleotides.

The term "haplotype" refers to a combination of alleles at adjacent sites on chromosomes that are inherited together. The haplotype can be a single locus, multiple loci, or the entire chromosome, depending on the number of recombination events that occurred between these sets, if a given set of loci occurred.

The term "threshold" herein refers to a numeric or non-numeric value used as a cutoff to characterize a sample, nucleic acid, or portion thereof (eg, read). The threshold can be varied based on empirical analysis. The threshold can be compared to a measured or calculated value to determine whether the source generating this value should be classified in a particular way. The threshold can be identified empirically or analytically. The choice of threshold depends on the level of confidence that the user wants to classify. Thresholds can be selected for a specific purpose (eg, to balance sensitivity and selectivity). As used herein, the term “threshold” refers to a point where the analysis process can be changed and / or a point where an action can be triggered. The threshold need not be a predetermined number. Instead, the threshold may be, for example, a function based on multiple factors. The threshold can be adaptive to the situation. Also, the threshold value may indicate an upper limit, a lower limit, or a range between limit values.

In some embodiments, a metric or score based on sequencing data can be compared to a threshold. As used herein, the terms "metric" or "score" may include values or results determined from sequencing data. Like the threshold, the metric or score can be adaptive depending on the situation. For example, the metric or score can be a normalized value. As an example of a score or metric, one or more implementations may use a count score when analyzing data. The count score can be based on the number of sample leads. A sample lead can undergo one or more filtering steps so that the sample lead has one or more common characteristics or qualities. For example, each sample read used to determine the count score can be aligned with a reference sequence or assigned as a potential allele. The number of sample leads with common characteristics can be counted to determine the lead count value. The count score can be based on a read count. In some implementations, the count score can be the same value as the read count. In other implementations, the count score can be based on a read count and other information. For example, the count score can be based on the number of leads for a particular allele of a genetic locus and the total number of leads for a genetic locus. In some implementations, the count score can be based on a read count for a genetic locus and previously acquired data. In some implementations, the count score can be a score normalized between predetermined values. The count score can also be a function of read count values from different loci of samples or a function of read count values from other samples executed concurrently with the sample of interest. For example, the count score can be a function of the read count of a particular allele and the read count of another locus in a sample and / or the read count from another sample. In one example, read counts from different loci and / or read counts from other samples can be used to normalize count scores for a particular allele.

The term "coverage" or "fragment coverage" refers to the count or other measure of multiple sample reads for the same fragment of a sequence. The lead count value may represent the count value of the number of leads covering the corresponding fragment. Alternatively, coverage can be determined by multiplying a read coefficient by a designated coefficient based on historical knowledge, sample knowledge, loci knowledge, and the like.

The term “lead depth” (typically the number followed by “x”) refers to the number of sequenced reads with overlapping alignments at the target position. This is often expressed as the average or percentage over cutoff over a set of intervals (eg exon, gene or panel). For example, according to clinical reports, it can be said that the panel average coverage is 1,105 × and 98% of the target base covers> 100 ×.

The term “base call quality score” or “Q score” refers to a PHRED-scale probability in the range of 0-20, where a single sequenced base is inversely proportional to the correct probability. For example, a T base call with Q of 20 can be considered correct if the confidence P-value is 0.01. All base calls with Q <20 should be considered low quality, and any variant with a significant portion of the sequenced reads supporting the variant identified as low quality should be considered a potential false positive.

The term "variant read" or "variant read number" refers to the number of sequenced reads that support the presence of the variant.

Sequencing process

This section provides a background for sequencing by synthesis (SBS) and identification of variants. The embodiments described herein can be applied to analyze nucleic acid sequences to identify sequence variations. Embodiments can be used to analyze potential variants / alleles of a gene location / location and determine the genotype of the genetic locus or, in other words, to provide a genotype call for the locus. For example, nucleic acid sequences can be analyzed according to the methods and systems described in U.S. Patent Application Publication No. 2016/0085910 and U.S. Patent Application Publication No. 2013/0296175, the full subject of which is incorporated herein by reference.

In one embodiment, the sequencing process comprises receiving a sample comprising or suspected of containing a nucleic acid, such as DNA. Samples can be derived from known or unknown sources, such as animals (eg, humans), plants, bacteria or fungi. Samples can be taken directly from the source. For example, blood or saliva can be taken directly from an individual. Alternatively, the sample may not be obtained directly from the source. The one or more processors then instruct the system to prepare the sample for sequencing. Preparation may include removing foreign substances and / or sequestering certain substances (eg, DNA). Biological samples can be prepared to include features for a particular assay. For example, biological samples can be prepared for synthetic sequencing (SBS). In certain embodiments, preparation may include amplification of certain regions of the genome. For example, preparation may include amplifying a predetermined genetic locus known to contain STR and / or SNP. The genetic locus can be amplified using a predetermined primer sequence.

Next, one or more processors can direct the system to sequence the sample. Sequencing can be performed through a variety of known sequencing protocols. In certain embodiments, sequencing comprises SBS. In SBS, a plurality of fluorescence-labeled nucleotides can be multiple clusters (millions of clusters) of amplified DNA present on the surface of an optical substrate (eg, a surface that at least partially defines a channel of a flow cell). ). The flow cell can include a nucleic acid sample for sequencing where the flow cell is placed in an appropriate flow cell holder.

Nucleic acids can be prepared such that the nucleic acid comprises a known primer sequence adjacent to an unknown target sequence. To initiate the first SBS sequencing cycle, one or more differently labeled nucleotides, and DNA polymerase, etc., can be flowed into / through flow cells by the fluid flow subsystem. A single type of nucleotide can be added at one time, or the nucleotides used in the sequencing procedure can be specifically designed to have reversible termination properties, so that each cycle of the sequencing reaction can be labeled with several types of labeled nucleotides (e.g. , A, C, T, G) in the presence. The nucleotides can include detectable label moieties such as fluorophores. When four nucleotides are mixed together, the polymerase can select the exact base to be incorporated, and each sequence is extended by a single base. Unmixed nucleotides can be washed by flowing a wash solution through flow cells. One or more lasers can stimulate nucleic acids and cause fluorescence. Fluorescence emitted from the nucleic acid is based on the fluorophore of the incorporated base, and different fluorophores can emit emitted light of different wavelengths. Deblocking reagents can be added to flow cells to remove reversible terminator groups from expanded and detected DNA strands. The deblocking reagent can then be washed by flowing the washing solution through the flow cells. The flow cells are then prepared for further cycles of sequencing, starting with the introduction of labeled nucleotides as described above. Fluid and detection operations can be repeated multiple times to complete the sequencing run. Examples of sequencing methods include, for example, Bentley et al., Nature 456: 53-59 (2008); International Application Publication No. WO04 / 018497; U.S. Patent No. 7,057,026; International Application Publication No. WO 91/06678; International Application Publication No. WO 07/123744; U.S. Patent No. 7,329,492; U.S. Patent No. 7,211,414; U.S. Patent No. 7,315,019; U.S. Patent No. 7,405,281; And US Patent Application Publication No. 2008/0108082, each of which is incorporated herein by reference.

In some embodiments, nucleic acids can be attached to a surface and amplified before or during sequencing. For example, amplification can be performed using bridge amplification to form nucleic acid clusters on the surface. Useful bridge amplification methods include, for example, US Pat. No. 5,641,658; United States Patent Application Publication No. 2002/0055100; U.S. Patent No. 7,115,400; United States Patent Application Publication No. 2004/0096853; United States Patent Application Publication No. 004/0002090; United States Patent Application Publication No. 2007/0128624; And US Patent Application Publication No. 2008/0009420, the full text of each of which is incorporated herein by reference. Another useful method for amplifying nucleic acids on the surface is, for example, Lizardi et al. Nat. Genet. 19: 225-232 (1998) and US Patent Application Publication No. 2007/0099208 A1, which is a Rolling Circle Amplification (RCA), each of which is incorporated herein by reference.

Examples of SBS protocols include modified nucleotides with removable 3 'blocks, as described, for example, in International Publication No. WO 04/018497, US Patent Application Publication No. 2007 / 0166705A1, and US Patent No. 7,057,026. And each of these documents is incorporated herein by reference. For example, repeated cycles of the SBS reagent can be delivered to flow cells to which the target nucleic acid is attached, for example as a result of a bridge amplification protocol. Nucleic acid clusters can be converted to single-stranded form using a linearization solution. The linearization solution can contain, for example, a restriction endonuclease capable of cleaving one strand of each cluster. Other cleavage methods include, in particular, chemical cleavage (eg cleavage of diol linkages with peroxates), cleavage of basic sites by cleavage with endonucleases (eg, based in Ipswich, Massachusetts, USA). 'USER' as supplied by NEB, Part No. M5505S), exposure to heat or alkali, cleavage of ribonucleotides incorporated with amplification products consisting of deoxyribonucleotides, photochemical cleavage, or cleavage of peptide linkers It can be used as an alternative method for limiting enzymes or nicking enzymes, including. After the linearization operation, the sequencing primers can be delivered to flow cells under conditions to hybridize the sequencing primers to the target nucleic acid to be sequenced.

The flow cells can then be contacted with SBS extension reagents with modified nucleotides with removable 3 'blocks and fluorescent labels under conditions that extend primers hybridized to each target nucleic acid by the addition of a single nucleotide. Once the modified nucleotides have been incorporated into a growing polynucleotide chain complementary to the region of the template being sequenced, only a single nucleotide is added to each primer because there are no free 3'-OH groups available to direct further sequence expansion. , Therefore, the polymerase cannot add additional nucleotides. The SBS extension reagent can be removed and replaced with a scanning reagent that contains a component that protects the sample in an excited state with radiation. Exemplary ingredients for scan reagents are described in US Patent Application Publication No. 2008/0280773 A1 and US Patent Application No. 13 / 018,255, each of which is incorporated herein by reference. The expanded nucleic acid can then be fluorescently detected in the presence of a scan reagent. Once fluorescence has been detected, the 3 'block can be removed using a deblocking reagent suitable for the blocking group used. Exemplary deblocking reagents useful for each blocking group are described in WO004018497, US 2007/0166705 A1, and US Pat. No. 7,057,026, each of which is incorporated herein by reference. The deblocking reagent is washed to allow the target nucleic acid to hybridize to an extended primer with a 3'-0H group, now a component for the addition of additional nucleotides. Thus, the cycle of adding expansion reagents, scan reagents, and deblocking reagents by selective washing between one or more operations can be repeated until the desired sequence is obtained. The cycle can be performed using a single extended reagent delivery per cycle when each modified nucleotide is attached with a different label known to correspond to a particular base. Different labels facilitate differentiation of nucleotides added during each incorporation operation. Alternatively, each cycle can include separate actions of extended reagent delivery and subsequent actions of reagent delivery and detection, in which case two or more nucleotides can have the same label and based on a known sequence of delivery. Can be distinguished.

Although sequencing operations have been described above with respect to a particular SBS protocol, it will be understood that other protocols for sequencing any of any of a variety of different molecular analyzes can be performed as needed.

The one or more processors of the system then receive sequencing data for subsequent analysis. Sequencing data can be formatted in a variety of ways, such as .BAM files. The sequencing data can include, for example, multiple sample leads. The sequencing data can include a plurality of sample reads having corresponding sample sequences of nucleotides. While only one sample read is described, it should be understood that sequencing data can include, for example, hundreds, thousands, hundreds of thousands, or millions of sample leads. Different sample leads can have different numbers of nucleotides. For example, sample reads can range from 10 nucleotides to about 500 nucleotides or more. Sample leads can span the entire genome of the source (s). In one example, sample lead values relate to predetermined genetic loci, such as those loci in which STR is suspected or SNP is suspected.

Each sample read may contain a sequence of nucleotides that may be referred to as a sample sequence, sample fraction or target sequence. Sample sequences can include, for example, primer sequences, flanking sequences, and target sequences. The number of nucleotides in the sample sequence may include 30, 40, 50, 60, 70, 80, 90, 100 or more. In some embodiments, one or more sample reads (or sample sequences) comprises at least 150 nucleotides, 200 nucleotides, 300 nucleotides, 400 nucleotides, 500 nucleotides or more. In some embodiments, a sample read can include more than 1000 nucleotides, more than 2000 nucleotides. The sample read (or sample sequence) may include primer sequences at one or both ends.

mberg, Michael. (2017). Pisces: An Accurate and Versatile Single Sample Somatic and Germline Variant Caller. 595-595. 10.1145/3107411.3108203 기사에 개시된 일루미나사(Illumina Inc.)(캘리포니아주 샌디에고 소재)에 의한 Pisees™이 있으며, 이 문헌의 완전한 주제 전문은 명백하게 본 명세서에 참고로 원용된다Next, the one or more processors analyze the sequencing data to obtain potential variant call (s) and sample variant frequency of sample variant call (s). This operation can also be referred to as a variant call application or variant caller. Thus, variant callers identify or detect variants, and variant classifiers classify the detected variants as somatic or germ cells, alternative variant callers can be used according to embodiments of the present invention, where different variant callers are: It can be used based on the type of sequencing operation performed based on features of the sample of interest, and the like. Non-limiting examples of variant call applications are hosted at https://github.com/Illumina/Pisces and are Dunn, Tamsen & Berry, Gwenn & Emig-Agius, Dorothea & Jiang, Yu & Iyer, Anita & Udar, Nitin & Str

mberg, Michael. (2017). Pisces: An Accurate and Versatile Single Sample Somatic and Germline Variant Caller. 595-595. There is Pisees ™ by Illumina Inc. (San Diego, CA) disclosed in the 10.1145 / 3107411.3108203 article, the full subject matter of which is expressly incorporated herein by reference.

Creating a training set

The creation of an extended training set is disclosed in what has been incorporated into the reference application literature. Millions of human genomes and exomes have been sequenced, but their clinical application is limited because it is difficult to distinguish positive genetic variants from disease-causing mutations. Here, by demonstrating that the common missense variant of both primate species is largely clinically benign in humans, pathogenic mutations can be systematically identified by the elimination process. Deep neural network capable of identifying pathogenic mutations in rare patients with 88% accuracy and discovering 14 new candidate genes with significant intellectual disabilities across the genome, using hundreds of thousands of common variants from population sequencing of six non-human primates Train. Classification of common variants from additional primate species will improve the interpretation of millions of uncertain and significant variants, further improving the clinical utility of human genome sequencing.

The clinical viability of diagnostic sequencing is limited by the difficulty in interpreting rare genetic variants in the human population and inferring their corresponding impact on disease risk. Due to the detrimental effects on health status, clinically significant genetic variants tend to appear extremely rarely in the population, and in most cases no impact on human health has been identified. The large number and rarity of these variants for their uncertain clinical significance poses a tremendous obstacle to the adoption of sequencing for personalized medicines and population-wide health screening.

Since most prevalent Mendelian diseases have a very low prevalence in the population, high frequency observations of the variant in the population provide strong evidence for positive results. Testing common mutations across various human populations is an effective strategy for classifying positive variants, but today the total amount of common mutations in humans is limited by bottlenecks in the recent history of our species, where much of the ancestral diversity has been lost. . Current population studies on humans represent notable inflation from the effective population size (Ne) of less than 10,000 individuals in the past 15,000 to 65,000 years, and a small pool of common polymorphisms is used to account for changes in populations of this size. It comes from a limited dose. Of the more than 70 million potential protein change missense substitutions in the reference genome, only about 1 in 1000 are present with a total population allele frequency of greater than 0.1%.

Outside the modern human population, chimpanzees contain the next closest existing species and share an amino acid sequence identity of 99.4%. The almost identical identity of the human and chimpanzee protein coding sequences suggests that selection selections that operate against chimpanzee protein-encoding variants can also model the results for the suitability of human mutations as IBS.

Since the average time for the neutral polymorphism to sustain the ancestral human lineage (~ 4N _e generations) is part of the species diversity time (about 6 million years ago), naturally occurring chimpanzee mutations are maintained by a sparse balanced choice. Except for haplotypes, most of the non-overlapping mutant spaces are explored. If polymorphism, an IBS, similarly affects the suitability of the two species, the presence of a variant with a high allele frequency in the chimpanzee population indicates a positive result in humans, and a variant known to have established a positive result by screening selection To enlarge its catalog. Significant additional details are provided in what is incorporated by reference.

Deep Learning network  architecture

In one embodiment disclosed in what is incorporated by reference application literature, the pathogenic prediction network takes as input the 51-length amino acid sequence centered on the variant of interest and the missense variant substituted at the central position is the secondary structure and the solvent access network (FIG. 2 and 3). Three 51-length positional frequency matrices are generated from multiple sequence alignments of 99 vertebrates, three of which are one for eleven primates, one for 50 mammals excluding primates, and excluding primates and mammals. One for 38 vertebrates.

The secondary structure deep learning network predicts the three-state secondary structure of the alpha-helix (H), beta sheet (B), and coil (C) at each amino acid position. The solvent access network predicts the three-state solvent accessibility, embedded (B), intermediate (I) and exposed (E) at each amino acid position. Both networks can take only flanking amino acid sequences as their input and can be trained using labels from non-overlapping crystal structures known in the protein databank. For input to pre-trained three-state secondary structures and three-state solvent access networks, a single length position frequency matrix generated from multiple sequence alignments for all 99 vertebrates of length 51 and depth 20 can be used. have. After pre-training a network of known crystal structures from a protein databank, the final two layers of the secondary structure and solvent model can be removed, and the output of the network directly connected to the inputs of the pathogenic model. The exemplary test accuracy achieved for the three-state quadratic structure prediction model was 79.86%. There was no substantial difference when comparing the prediction of neural networks when using DSSP-annotated structural markers for approximately 4,000 human proteins with crystal structures versus only using predicted structural markers.

Both the deep learning network (PrimateAI) of the present invention for predicting pathogenicity and the deep learning network for predicting secondary structure and solvent accessibility adopted the structure of the residual block. The detailed architecture of PrimateAI is described in FIG. 3.

2 shows an exemplary architecture 200 of a deep residual network for pathogenic prediction referred to herein as “PrimateAI”. In FIG. 2, 1D refers to the one-dimensional convolution layer. The predicted pathogenicity ranges from 0 (positive) to 1 (pathogenic). The network takes the human amino acid (AA) reference sequence and alternating sequence (51 AA) as inputs around the variant and predicts the positional weight matrix (PWM) conservation profile, calculated from 99 vertebrate species, and secondary structure and solvent accessibility. It takes the output of a deep learning network, which provides 3-state protein secondary structure (helix-H, beta sheet-B and coil-C) and 3-state solvent accessibility (embedded-B, medium-I and exposed-E). Predict.

3 shows a schematic example 300 of PrimateAI, a deep learning network architecture for pathogenic classification. Input to the model is represented by three 51-AA-length position-weighted matrices from 51 amino acid (AA), primate, mammalian and vertebrate alignments of the flanking sequence for both the reference and variant substituted sequences. Conservation, and output of pretrained secondary fenders and solvent access networks (which are also 51 AA long).

Improvement through pre-training

The present invention introduces pre-training a pathogenic predictive model to reduce or offset overfitting and improve training results. This system is described with reference to FIG. 1, which shows an architecture level schematic 100 of the system according to an implementation. 1 is an architectural diagram, certain details are intentionally omitted to improve clarity of description. The discussion of Figure 1 is structured as follows. First, the elements of FIG. 1 will be described, and then the interconnection will be described. The use of the elements of the system will then be described in more detail.

This paragraph names the labeled portion of the system shown in FIG. 1. The system includes four training datasets: pathogenic missense training example 121, supplemental positive training example 131, positive missense training example 161, and supplemental positive training example 181. The system further includes a trainer 114, a tester 116, a position frequency matrix (PFM) calculator 184, an input encoder 186, a variant pathogenicity prediction model 157 and network (s) 155. The supplemental positive training example 131 corresponds to the pathogenic missense training example 121 and is thus placed together in a box of dashed lines. Similarly, supplemental positive training example 181 corresponds to positive missense training example 161, so the two datasets are shown in the same box.

This system is described using PrimateAI as an exemplary variant pathogenicity prediction model 157 taking input amino acid sequence next to the variant of interest and parallel homologous sequence alignment in other species as input. The detailed structure of the PrimateAI model for predicting pathogenicity is presented above with reference to FIG. 3. Input of the amino acid sequence includes the variant of interest. The term "variant" refers to an amino acid sequence that is different from the amino acid reference sequence. The tri-nucleotide base sequence (also called a codon) at a specific position in the protein coding region of the chromosome expresses amino acids. There are 20 types of amino acids that can be formed by a combination of 61 tri-nucleotide sequences. Combinations of more than one codon or tri-nucleotide sequence can result in the same amino acid. For example, codons “AAA” and “AAG” refer to lysine amino acids (also referred to as symbol “K”).

Amino acid sequence variants can be caused by a single nucleotide polymorphism (SNP). SNP is a variant of a single nucleotide that occurs at a particular locus of a gene and is observed to some extent (eg> 1%) within the population. The disclosed technique focuses on SNPs occurring in the protein-coding region of a gene called exon. There are two types of SNPs: synonymous SNP and missense SNP. A synonymous SNP is a type of protein-coding SNP that replaces the first codon for an amino acid with a second codon for the same amino acid. On the other hand, missense SNP includes changing the first codon for the first amino acid to the second codon for the second amino acid.

6 shows an example 600 of “protein sequence pairs” for missense variants and correspondingly constructed synonymous variants. The phrase “protein sequence pair” or simply “sequence pair” refers to a reference protein sequence and an alternative protein sequence. The reference protein sequence includes a reference amino acid expressed by a reference codon or tri-nucleotide base. Alternate protein sequences include alternative codons or alternative amino acids expressed by tri-nucleotide bases, such that the alternative protein sequence occurs due to variants occurring in the reference codon expressing the reference amino acid of the reference protein sequence.

FIG. 6 shows the configuration of a supplementary positive consent counterpart training example (referred to as a supplementary positive training example above) corresponding to a missense variant. The missense variant can be a pathogenic missense training example or a positive missense training example. Consider a protein sequence pair for a missense variant with a reference amino acid sequence with the codon “TTT” at chromosome 1 at positions 5, 6 and 7 (ie 5: 7). It is now considered that SNPs occur at the same chromosome at position 6, resulting in a replacement sequence with the codon “TCT” at the same position, ie 5: 7. The codon “TTT” in the reference sequence results in the phenylalanine (F) amino acid, while the codon “TCT” in the replacement amino acid sequence results in the serine (S) amino acid. For simplicity, FIG. 6 shows only the amino acids and corresponding codons of the sequence pair at the target position. The flanking amino acids and each codon in the sequence pair are not shown. In the training dataset, missense variants are labeled pathogenic (labeled "1"). In order to reduce the overfitting of the model during training, the disclosed technique constitutes a counterpart complement positive training example for the corresponding missense variant. The reference sequence of the sequence pair for the constructed complementary positive training example is identical to the reference sequence in the missense variant shown in the left part of FIG. 6. The right part of Figure 6 shows a complementary positive training example that is a synonymous counterpart with the same reference sequence codon “TTT” at chromosome 1 at position (5: 7) as in the reference sequence for the missense variant. The replacement sequence constructed for the synonym counterpart has the SNP at position number (7), resulting in the codon “TTC”. This codon results in the same amino acid phenylalanine (F) in the replacement sequence as in the reference sequence at the same position on the same chromosome. Two different codons on the same chromosome at the same position express the same amino acid, so the synonym counterpart is labeled positive (or labeled as “0”). Two different codons at the same position in the reference and replacement sequences express the same amino acid at the target position. Positive counterparts are not randomly constructed; Instead, it is selected from synonymous variants observed in sequenced populations. The disclosed technique constitutes a supplemental positive training example to contrast with the pathogenic missense training example to reduce overfitting of the variant pathogenic predictive model during training.

The supplemental training example does not have to be consent. The disclosed technology can also constitute a complementary positive training example with the same amino acid in the replacement sequence constructed by the same tri-nucleotide codon as in the reference sequence. The associated positional frequency matrix (PFM) is identical for the same amino acid sequence regardless of whether the amino acids are expressed by the same or the same codon. Accordingly, this supplementary training example has the same effect during training as the consent counterpart training example shown in Fig. 6, that is, the effect of reducing the overfitting of the variant pathogenicity prediction model.

Now another element of the system shown in FIG. 1 will be described. Trainer 114 trains a variant pathogenic predictive model using the four training datasets shown in FIG. 1. In one embodiment, the variant pathogenicity prediction model is implemented as a convolutional neural network (CNN). Training of CNN was described above with reference to FIG. 5. The CNN is adjusted or trained during training so that the input data leads to a specific output estimate. Training involves adjusting the CNN using back propagation based on comparing the output estimate with the measured data until the output estimate gradually matches or approaches the measured data. After training, tester 116 benchmarks the variant pathogenicity prediction model using a test dataset. The input encoder 186 transforms classification-specific input data, such as reference and replacement amino acid sequences, into a form that can be provided as input to a variant pathogenic predictive model. This is further illustrated using exemplary reference and replacement sequences in FIG. 13.

The PFM calculator 184 calculates a position frequency matrix (PFM), also referred to as a position specific scoring matrix (PSSM) or a position weighting matrix (PWM). PFM shows the frequency of all amino acids (along the vertical axis) at each amino acid position (along the horizontal axis) as shown in FIGS. 10 and 11. The disclosed technique calculates three PFMs, one each for primates, mammals and vertebrates. The length of the amino acid sequence for each of the three PFMs can be 51, where the target amino acid can be flanked by at least 25 amino acids upstream and downstream. PFM has 20 rows for amino acids in the amino acid sequence and 51 columns for amino acid positions. The PFM calculator calculates a first PFM with an amino acid sequence for 11 primates, a second PFM with an amino acid sequence for 48 mammals, and a third PFM with an amino acid sequence for 40 vertebrates. Cells in PFM are the number of occurrences of amino acids at specific positions in the sequence. The amino acid sequences for the three PFMs are aligned. This means that the results calculated by position for primate, mammalian and vertebrate PFM for each amino acid position in the reference amino acid sequence or the replacement amino acid sequence are stored by position in the same order that the amino acid position occurs in the reference amino acid sequence or the replacement amino acid sequence. Or, it is stored based on the ordinal position.

The disclosed techniques use supplemental positive training examples 131 and 181 during initial training epochs, for example 2 or 3 or 5 or 8 or 10 epochs or 3 to 5, 3 to 8 or 2 to 10 epochs. 7, 8 and 9 show a pre-training epoch, a training epoch and a pathogenic prediction model during inference. FIG. 7 shows an example 700 of pre-trained epochs 1-5 with about 400,000 positive supplementary training examples 131 combined with about 400,000 pathogenic variants 121 predicted from a deep learning model. Less positive supplemental training examples, such as about 100,000, 200,000 or 300,000, can be combined with pathogenic variants. In one embodiment, a pathogenic variant data set is generated in 20 cycles using random samples from about 68 million synthetic variants as described above. In other embodiments, a pathogenic variant data set can be generated in one cycle from approximately 68 million synthetic variants. Pathogenic variants 121 and supplemental positive training examples 131 are provided as inputs to the ensemble of the network in the first five epochs. Similarly, approximately 400,000 supplemental positive training examples 181 are combined with approximately 400,000 positive variants 161 for ensemble training during pre-training epochs. Less positive training examples, such as about 100,000, 200,000 or 300,000, can be combined with positive variants.

Supplemental positive datasets 131 and 181 are not provided as inputs to the remaining training epochs 6 to n as shown in example 800 in FIG. 8. The network's ensemble training continues with pathogenic variant datasets and positive variant datasets across multiple epochs. Training ends after a predetermined number of training epochs or ends when an end condition is reached. A trained network is used during inference to evaluate synthetic variants 810 as shown in example 900 in FIG. 9. The trained network predicts the variant as pathogenic or positive.

The PFM (denoted by the number 1000) for an exemplary supplemental positive training example 1012 constructed as a counterpart of the pathogenic missense variant training example 1002 shown in FIG. 10 is now described. PFM is generated or referenced for training examples. The PFM for the training example depends only on the position of the reference sequence, so both training examples 1002 and 1012 have the same PFM. For example, two training examples are shown in FIG. 10. The first training example 1002 is a pathogenic / unlabeled variant. The second training example 1012 is an example of training to supplement the counterpart corresponding to the training example 1002. Training example 1002 has a reference sequence 1002R and a replacement sequence 1002A. The first PFM is accessed or generated for training example 1002 based only on the position of the reference sequence 1002R. The training example 1012 has a reference sequence 1012R and a replacement sequence 1012A. The first PFM, eg 1002, can be reused, for example 1012. PFM is calculated using amino acid sequences from multiple species, such as 99 primates, mammals, and vertebrates, as an indication of conservation of sequence across species. Humans may or may not be among the species presented when calculating PFM. The cells of this PFM contain the number of occurrences of amino acids across species in sequence. PFM 1022 is the starting point of PFM showing one hot encoding of a single sequence in the training example. When PFM is complete for 99 examples, the positions that are fully conserved across the species have a value of "99" instead of "1". In this example, partial preservation creates two or more rows in the column with values summing to 99. Because PFM relies on the entire sequence position, not the amino acid at the center position of the sequence, both the reference and replacement sequences have the same PFM.

Described now is the determination of PFM 1012 using positions in the exemplary reference sequence of FIG. 10. Exemplary reference and replacement amino acid sequences for the pathogenic / non-labeled training example 1002 and supplemental positive training example 1012 as shown in FIG. 10 have 51 amino acids. The reference amino acid sequence 1002R has an arginine amino acid denoted "R" at position 26 (also referred to as the target position) in the sequence. It is noted that at the nucleotide level, one of the six tri-nucleotide bases or codons (CGT, CGC, CGA, CGG, AGA and AAG) express the amino acid "R". This example does not show these codons, but focuses on simplifying the explanation and calculating the PFM. Consider an amino acid sequence (not shown) from one of 99 species that is aligned with the reference sequence and has the amino acid "R" at position 26. This will result in a value of "1" in the PFM 1022 of the cell at the intersection of row "R" and column "26". Similar values are determined for all columns of the PFM. The two PFMs (i.e., the PFM for the reference sequence (1002R) for the pathogenic missense variant 1002 and the PFM for the reference sequence (1012R) for the supplemental positive training example (1012)) are the same, but for illustrative purposes Only one PFM 1022 is shown. These two PFMs represent opposite examples of pathogenicity to related amino acids. One is labeled pathogenic or "1" and the other is labeled "0" as positive. Thus, the disclosed technique reduces this by providing this example to the model during training.

Construct a second set of supplemental positive training examples 181 corresponding to positive missense variants 161 in the training data set. FIG. 11 shows an example 1100 of calculating two PFMs for an exemplary positive missense variant 1102 and a corresponding supplemental positive training example 1122. As can be seen in this example, the reference sequences 1102R and 1112R are identical for both the positive missense variant 1102 and the supplemental positive training example 1112. The respective replacement sequences 1102A and 1112A are also shown in FIG. 11. Two PFMs are generated or referenced for two reference sequences as described above for the example shown in FIG. 10. Both PFMs are identical, and only one PFM 1122 is shown in FIG. 11 for illustration. All of these PFMs show amino acid sequences labeled positive ("0").

The disclosed technique calculates three PFMs, one for each of the 11 primate sequences, 48 mammalian sequences, and 40 vertebrate sequences. 12 shows three PFMs 1218, 1228 and 1238 with 20 rows and 51 columns, respectively. In one embodiment, the primate sequence does not include a human reference sequence. In other embodiments, the primate sequence comprises a human reference sequence. The values of the cells in the three PFMs are calculated by counting the occurrence of amino acids (row markers) present in all sequences for the PFM at a given position (column markers). For example, if the three primate sequences have the amino acid "K" at position 26, then the value of the cell with row marker "K" and column marker "26" has a value of "3".

One hot encoding is the process of converting classification-specific variables into a form that can provide input to a deep learning model. The classification-specific values represent alphanumeric values for entries in the data set. For example, the reference and replacement amino acid sequences each have 51 amino acid characters arranged in the sequence. The amino acid letter "T" at position "1" in the sequence represents the amino acid threonine at the first position in the sequence. The amino acid sequence is encoded by assigning a value of "1" in cells with row marker "T" and column marker "1" in one hot encoded expression. One hot-encoded expression for the amino acid sequence has zero in cells except for cells that represent amino acids (row markers) that occur at specific positions (column markers). FIG. 13 shows an example 1300 in which reference and replacement sequences for the supplemental positive training example are represented by one hot encoding. The reference and replacement amino acid sequences are entered in one hot encoded form as input to the variant pathogenicity prediction model. 14 includes an example 1400 showing input to a variant pathogenicity prediction model. Inputs include human reference and replacement amino acid sequences in one hot encoded form, and PFM 1218 for primates, PFM 1228 for mammals and PFM 1238 for vertebrates. As described above, PFM for primates may include only non-human primates or human and non-human primates.

Variants in this approach to supplement the training set also apply to the architecture described in the application literature, the entire contents of which are incorporated herein, and in combination with other data types, in particular any other using PFM with sequences of amino acids or nucleotides. It also applies to architecture.

result

The performance of the neural network based model (eg PrimateAI model presented above) is improved by using the pre-trained epoch presented above. The following table is an example of the test results. The results of the table consist of six headings. Briefly describe the title before presenting the results. The "Replication" column shows the results of 20 replicate runs. Each run can be an ensemble of 8 models with different random seeds. “Accuracy” is the proportion of 10,000 reserved primate positive variants classified as positive. "P value_DDD" presents the results of the Wilcoxon ranking test to assess the degree of de novo mutation separation in affected children with developmental disabilities from unaffected siblings. The "p-value_605 gene" indicates a test result similar to the p-value_DDD except that in this case, a de novo mutation was used in the 605 disease-related gene. "Corr_RK_RW" represents the correlation of PrimateAI scores between amino acid changes from R to K and R to W. The smaller the Corr_RK_RW value, the better the performance. "P value_Corr" represents the p value of the correlation in the previous column, that is, Corr_RK_RW.

The results show that the median accuracy of prediction of positive variants using a median score of unknown variants as a cutoff is 91.44% over 20 replicate runs. The log p-value of the Wilcoxon rank sum test is 29.39 to separate the de novo miss sense variant of DDD patients from the de novo miss sense variant of the control. Similarly, the logarithmic p-value of the rank sum test is only 16.18 compared to the de novo missense variant in the 605 disease gene. The metric was improved over the previously reported results. The correlation between R-> K and R-> W is significantly reduced by p-value = 3.11e-70 as measured by the Wilcoxon rank sum test.

certain Implementation

We describe systems, methods, and articles of manufacture for pretraining neural network implementation models that process amino acid sequences and concomitant position frequency matrices (PFMs). One or more functions of one implementation can be combined with a basic implementation. It is understood that embodiments that are not mutually exclusive are combinable. One or more functions of one implementation can be combined with other implementations. This disclosure periodically reminds the user of these options. Omissions from some implementations of the repetition of these options should not be considered as limiting the combinations described in the previous section, which are incorporated in each implementation below.

System implementations of the disclosed technology include one or more processors coupled to memory. The memory is loaded with computer instructions to reduce overfitting of the neural network implementation model that processes the amino acid sequence and the accompanying position frequency matrix (PFM). The system includes a logic circuit to generate a positively labeled supplemental training example sequence pair that includes from the starting position to the target amino acid position to the final position. The complementary sequence pair matches the start and end positions of the missense training example sequence pair. It has the same amino acids in the reference and replacement amino acid sequences. The system includes logic circuitry to input the complement training PFM with each complement sequence pair identical to the PFM of the missense training example sequence pair at the matching start and end positions. The system includes logic circuits to train the neural network implementation model using PFMs of positive training examples sequence pairs and complementary training examples PFM, and missense training examples sequence pairs and missense training examples sequence pairs at matching start and end positions. do. Training The impact of PFM training is attenuated during training.

The disclosed system implementations and other systems optionally include one or more of the following features. The system can also include features described in connection with the disclosed method. For brevity, alternative combinations of system functions are not listed individually. Features applied to systems, methods and articles of manufacture are not repeated for each statutory category of basic features. Readers will understand how the functions identified in this sector can be easily combined with the basic functions of other statutory categories.

The system can include logic circuitry to construct the complementary sequence pair so that each complementary sequence pair matches the start and end positions of the positive missense training example sequence pair.

The system can include logic circuitry to construct the complementary sequence pair so that each complementary sequence pair matches the pathogenic missense training example start and end positions of the sequence pair.

The system includes logic circuitry to modify the training of the neural network implementation model to stop using a pre-determined number of training epochs and then supplemental training examples sequence pairs and supplemental training PFM.

The system includes logic circuitry to modify the training of the neural network implementation model to stop using three training epochs and supplemental training examples sequence pairs and supplemental training PFM.

The system includes logic circuitry to modify the training of the neural network implementation model to stop using supplemental training example sequence pairs and supplemental training PFM after 5 training epochs.

The ratio of complementary training example sequence pair to pathogenic training example sequence pair may be 1: 1 to 1: 8. The system can use different values, for example, in the range of 1: 1 to 1:12, 1: 1 to 1:16, and 1: 1 to 1:24.

The ratio of complementary training example sequence pairs to positive training example sequence pairs may be 1: 2 to 1: 8. The system can use different values, for example, in the range of 1: 1 to 1:12, 1: 1 to 1:16 and 1: 1 to 1:24.

The system includes logic circuitry to use the position of amino acids from data from non-human primate and non-primate mammals when generating supplemental PFM.

Other implementations may include a non-transitory computer-readable storage medium that stores instructions executable by the processor to perform the functions of the system described above. Another implementation may include a method of performing the functions of the above-described system.

Method embodiments of the disclosed technology include generating a positively labeled supplemental training example sequence pair comprising from the starting position to the target amino acid position to the final position. Each complementary sequence pair matches the start and end positions of the missense training example sequence pair. It has the same amino acids in the reference and replacement amino acid sequences. The method includes entering a complement training PFM with each complement sequence pair identical to the PFM of the missense training example sequence pair at the matching start and end positions. The method includes training a neural network implementation model using a positive training example sequence pair and complementary training example PFM, and missense training example sequence pair and missense PFM at matching start and end positions. Training The impact of PFM training is attenuated during training.

The disclosed method embodiments and other methods optionally include one or more of the following features. The method may also include features described in connection with the disclosed system. Readers will understand how the functions identified in this sector can be easily combined with the basic functions of other statutory categories.

Other embodiments are one or more non-transitory computers that collectively store computer program instructions executable by one or more processors to reduce overfitting of the neural network implementation model to process amino acid sequences and concomitant position frequency matrices (PFM). It may include a set of readable storage media. Computer program instructions, when executed on one or more processors, implement a method comprising generating a positively labeled supplemental training example sequence pair that includes from a starting position to a target amino acid position to an ending position. Each complementary sequence pair matches the start and end positions of the missense training example sequence pair. It has the same amino acids in the reference and replacement amino acid sequences. The method includes entering a complement training PFM with each complement sequence pair identical to the PFM of the missense training example sequence pair at the matching start and end positions. The method includes training a neural network implementation model using a positive training example sequence pair and complementary training example PFM, and a missense training example sequence pair and missense training PFM at matching start and end positions. Training The impact of PFM training is attenuated during training.

Computer-readable media (CRM) implementations of the disclosed technology include one or more non-transitory computer-readable storage media storing computer program instructions that, when executed on one or more processors, implement the methods described above. This CRM implementation includes one or more of the following features. CRM implementations may also include features described in connection with the systems and methods described above.

The foregoing description is provided to enable the manufacture and use of the disclosed technology. Various modifications to the disclosed embodiments will be apparent, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of the disclosed technology. Accordingly, the disclosed technology is not intended to be limited to the illustrated implementations, but should be in accordance with the broadest scope consistent with the principles and features disclosed herein. The scope of the disclosed technology is defined by the appended claims.

Computer system

15 is a simplified block diagram 1500 of a computer system that can be used to implement the disclosed technology. Computer systems typically include at least one processor that communicates with multiple peripherals through a bus subsystem. Such peripheral devices may include, for example, a storage subsystem including a memory device and a file storage subsystem, a user interface input device, a user interface output device, and a network interface subsystem. Input and output devices allow user interaction with a computer system. The network interface subsystem provides interfaces to external networks, including interfaces to corresponding interface devices in other computer systems.

In one embodiment, neural networks such as variant pathogenic classifier 157, PFM calculator 184 and input encoder 186 are communicatively coupled to the storage subsystem and user interface input device.

The user interface input device includes a keyboard; Pointing devices such as a mouse, trackball, touchpad, or graphics tablet; scanner; A touch screen integrated into the display; Audio input devices such as speech recognition systems and microphones; And other types of input devices. In general, the use of the term "input device" is intended to include all possible types of devices and methods for inputting information into a computer system.

The user interface output device may include a non-visual display, such as a display subsystem, printer, fax machine, or audio output device. The display subsystem may include a flat panel device such as a cathode ray tube (CRT), a liquid crystal display (LCD), a projection device, or some other mechanism for generating a visual image. The display subsystem can also provide a non-visual display, such as an audio output device. In general, the use of the term "output device" is intended to include all possible types of devices and methods for outputting information from a computer system to a user or other machine or computer system.

The storage subsystem stores programming and data configurations that provide the functionality of some or all of the modules and methods described herein. These software modules are typically executed either alone or in combination with other processors.

The memory used in the storage subsystem may include a number of memories including a main random access memory (RAM) for storing instructions and data during program execution and a read-only memory (ROM) in which fixed instructions are stored. The file storage subsystem may provide permanent storage for program and data files, and may include a hard disk drive, a floppy disk drive with associated removable media, a CD-ROM drive, an optical drive, or a removable media cartridge. You can. Modules that implement the functionality of certain implementations may be stored by the file storage subsystem in a storage subsystem or other machine accessible to the processor.

The bus subsystem provides a mechanism for allowing various components and subsystems of the computer system to communicate with each other as intended. Although the bus subsystem is schematically represented as a single bus, multiple buses may be used in alternative implementations of the bus subsystem.

The computer system itself includes a personal computer, portable computer, workstation, computer terminal, network computer, television, mainframe, server farm, a widely distributed set of loosely networked computers, or any other data processing system or user device. It can be of various types including. Due to the constantly changing nature of computers and networks, the description of the computer system shown in FIG. 15 is intended only as a specific example to illustrate the disclosed technology. Many other configurations of computer systems with more or fewer components than the computer system shown in FIG. 15 are possible.

The deep learning processor can be a GPU or an FPGA, and can be hosted by a deep learning cloud platform such as Google Cloud Platform, Xilinx, and Syracscale. Examples of in-depth learning processors include rackmount solutions such as Google's Tensor Processing Unit (TPU), GX4 Rackmount Series, GX8 Rackmount Series, NVIDIA DGX-1, Microsoft's Stratix V FPGA, Graphcore's Intelligent Processor Unit (IPU), and Qualcomm Zeroth platform with Snapdragon processors from NVIDIA, Volta from NVIDIA, DRIVE PX from NVIDIA, JETSON TX1 / TX2 MODULE from NVIDIA, Nirvana from Intel, Movidius VPU, Fujitsu DPI, DynamicIQ from ARM, IBM TrueNorth and more.

Claims (24)

A method of reducing overfitting of a neural network implementation model that processes amino acid sequences and concomitant position frequency matrices (PFM),
Generating a positively labeled supplemental training example sequence pair from the start position through the target amino acid position to the end position, each complementary sequence pair comprising:
Missense training example matches the start position and the end position of a sequence pair;
Generating said supplemental training example sequence pair, having identical amino acids in the reference and replacement amino acid sequences;
Inputting a complementary training PFM identical to the PFM of the missense training example with each complementary sequence pair at matching start and end positions; And
Training the neural network implementation model using a positive training example sequence pair and supplemental training example PFM, and the missense training example sequence pair, and the missense PFM at the matching start and end positions,
A method of reducing overfitting of a neural network implementation model, wherein the training impact of the training PFM is attenuated during the training. The method of claim 1, wherein the complementary sequence pair matches pathogenic missense training, eg, the start position and the end position of the sequence pair, to reduce overfitting of the neural network implementation model. The method of claim 1, wherein the complementary sequence pair coincides with the starting position and the ending position of a positive missense training example sequence pair. According to claim 1,
Further comprising modifying training of the neural network implementation model to stop using the supplemental training example sequence pair and the supplemental training PFM after a predetermined number of training epochs, to reduce overfitting of the neural network implementation model. How to order. According to claim 1,
And modifying the training of the neural network implementation model to stop using the supplemental training example sequence pair and the supplemental training PFM after 5 training epochs. According to claim 2,
A method for reducing overfitting of a neural network implementation model, further comprising a ratio of the supplementary training example sequence pair to the pathogenic missense training example sequence pair is 1: 1 to 1: 8. According to claim 3,
The ratio of the complement training example sequence pair to the positive missense training example sequence pair further comprising 1: 1 to 1: 8. According to claim 1,
A method of reducing overfitting of a neural network implementation model, further comprising using amino acid positions from data for non-human primate and non-primate mammals when generating the supplemental PFM. A system comprising one or more processors coupled to memory, wherein the memory is loaded with computer instructions to reduce overfitting of the neural network implementation model to process amino acid sequences and concomitant position frequency matrices (PFM), the instructions being: When running on the processor,
Generating a positively labeled supplemental training example sequence pair from the starting position to the target amino acid position and including the final position, each complementary sequence pair comprising:
Missense training example matches the start position and the end position of a sequence pair;
Generating said supplemental training example sequence pair with identical amino acids in the reference and replacement amino acid sequences;
Inputting a complementary training PFM identical to the PFM of the missense training example with each supplementary sequence pair at matching start and end positions; And
Implement an operation of training the neural network implementation model using the positive training example sequence pair and supplementary training example PFM, and the missense training example sequence pair, and the missense PFM at the matching start and end positions,
The training effect of the training PFM is attenuated or canceled during training. 10. The system of claim 9, wherein the complementary sequence pair matches the pathogenic missense training example the starting position and the ending position of the sequence pair. 10. The system of claim 9, wherein the complementary sequence pair coincides with a positive missense training example the starting position and the ending position of the sequence pair. The method of claim 9,
And further implement the operation of modifying the training of the neural network implementation model to stop using the supplemental training example sequence pair and the supplemental training PFM after a predetermined number of training epochs. The method of claim 9,
The system further implements an operation of modifying training of the neural network implementation model to stop using the supplemental training example sequence pair and the supplemental training PFM after 5 training epochs. The method of claim 10,
Wherein the ratio of the supplemental training example sequence pair to the pathogenic missense training example sequence pair is between 1: 1 and 1: 8. The method of claim 11,
Wherein the ratio of the complement training example sequence pair to the positive missense training example sequence pair is between 1: 1 and 1: 8. The method of claim 9,
A system further implementing the operation of using amino acid positions from data for non-human primate and non-primate mammals when generating the supplemental PFM. A non-transitory computer readable storage medium having computer program instructions stored therein to reduce overfitting of a neural network implementation model that processes amino acid sequences and concomitant position frequency matrices (PFMs), when the instructions are executed on a processor,
Generating a positively labeled supplemental training example sequence pair from the start position through the target amino acid position to the end position, each complementary sequence pair comprising:
Missense training example matches the start position and the end position of a sequence pair;
Generating said supplemental training example sequence pair, having identical amino acids in the reference and replacement amino acid sequences;
Entering a complementary training PFM identical to the PFM of the missense at each matching start and end position, with each supplemental sequence pair; And
Implementing a method comprising training the neural network implementation model using a positive training example sequence pair and supplementary training example PFM, and missense training example sequence pair, and the missense PFM at the matching start and end positions. and;
A non-transitory computer readable storage medium in which the training impact of the training PFM is attenuated during training. 18. The non-transitory computer readable storage medium of claim 17, wherein the complementary sequence pair matches the pathogenic missense training example the starting position and the ending position of the sequence pair. 18. The non-transitory computer readable storage medium of claim 17, wherein the complementary sequence pair coincides with the starting position and the ending position of a positive missense training example sequence pair. The method of claim 17,
Non-transitory computer readable storage medium embodying a method further comprising modifying training of the neural network implementation model to stop using the supplemental training example sequence pair and the supplemental training PFM after a predetermined number of training epochs. . The method of claim 17,
A non-transitory computer readable storage medium implementing the method further comprising modifying the training of the neural network implementation model to stop using the supplemental training example sequence pair and the supplemental training PFM after 5 training epochs. The method of claim 18,
A non-transitory computer readable storage medium embodying a method further comprising the ratio of the supplemental training example sequence pair to the pathogenic missense training example sequence pair is 1: 1 to 1: 8. The method of claim 19,
A non-transitory computer readable storage medium embodying a method further comprising the ratio of the supplemental training example sequence pair to the positive missense training example sequence pair is 1: 1 to 1: 8. The method of claim 17,
A non-transitory computer readable storage medium implementing the method further comprising using amino acid positions from data for non-human primate and non-primate mammals when generating the supplemental PFM.

Images