US20200168302A1 - Methods, systems and non-transitory computer readable media for automated design of molecules with desired properties using artificial intelligence - Google Patents
Methods, systems and non-transitory computer readable media for automated design of molecules with desired properties using artificial intelligence Download PDFInfo
- Publication number
- US20200168302A1 US20200168302A1 US16/632,328 US201816632328A US2020168302A1 US 20200168302 A1 US20200168302 A1 US 20200168302A1 US 201816632328 A US201816632328 A US 201816632328A US 2020168302 A1 US2020168302 A1 US 2020168302A1
- Authority
- US
- United States
- Prior art keywords
- molecules
- neural network
- property
- model
- trained
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000013461 design Methods 0.000 title claims description 13
- 238000013473 artificial intelligence Methods 0.000 title claims description 9
- 238000012549 training Methods 0.000 claims abstract description 31
- 238000013135 deep learning Methods 0.000 claims abstract description 7
- 238000013528 artificial neural network Methods 0.000 claims description 35
- 239000000126 substance Substances 0.000 claims description 28
- 230000006870 function Effects 0.000 claims description 21
- 230000015654 memory Effects 0.000 claims description 16
- 230000000306 recurrent effect Effects 0.000 claims description 12
- 230000004071 biological effect Effects 0.000 claims description 7
- 230000000704 physical effect Effects 0.000 claims description 5
- 150000005829 chemical entities Chemical class 0.000 claims 2
- 150000001875 compounds Chemical class 0.000 abstract description 25
- 230000002787 reinforcement Effects 0.000 abstract description 14
- 239000013598 vector Substances 0.000 abstract description 14
- 238000002474 experimental method Methods 0.000 abstract description 9
- 238000013459 approach Methods 0.000 abstract description 7
- 238000007876 drug discovery Methods 0.000 abstract description 7
- 230000008569 process Effects 0.000 abstract description 6
- 230000001537 neural effect Effects 0.000 abstract description 3
- 238000000205 computational method Methods 0.000 abstract 1
- 238000012800 visualization Methods 0.000 abstract 1
- 238000009826 distribution Methods 0.000 description 20
- 238000002844 melting Methods 0.000 description 11
- 238000001994 activation Methods 0.000 description 10
- 230000008018 melting Effects 0.000 description 10
- 101000997832 Homo sapiens Tyrosine-protein kinase JAK2 Proteins 0.000 description 9
- 102100033444 Tyrosine-protein kinase JAK2 Human genes 0.000 description 9
- 230000004913 activation Effects 0.000 description 9
- 210000004027 cell Anatomy 0.000 description 9
- 229940079593 drug Drugs 0.000 description 8
- 239000003814 drug Substances 0.000 description 8
- 238000005457 optimization Methods 0.000 description 8
- 230000009471 action Effects 0.000 description 7
- 206010048669 Terminal state Diseases 0.000 description 6
- 238000004458 analytical method Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- KBPLFHHGFOOTCA-UHFFFAOYSA-N caprylic alcohol Natural products CCCCCCCCO KBPLFHHGFOOTCA-UHFFFAOYSA-N 0.000 description 4
- 239000003795 chemical substances by application Substances 0.000 description 4
- 230000000875 corresponding effect Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- HCHKCACWOHOZIP-UHFFFAOYSA-N Zinc Chemical compound [Zn] HCHKCACWOHOZIP-UHFFFAOYSA-N 0.000 description 3
- 230000003190 augmentative effect Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 125000003636 chemical group Chemical group 0.000 description 3
- 230000007423 decrease Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 239000000203 mixture Substances 0.000 description 3
- 238000003032 molecular docking Methods 0.000 description 3
- 210000002569 neuron Anatomy 0.000 description 3
- 238000005192 partition Methods 0.000 description 3
- 125000001997 phenyl group Chemical group [H]C1=C([H])C([H])=C(*)C([H])=C1[H] 0.000 description 3
- 125000001424 substituent group Chemical group 0.000 description 3
- 239000011701 zinc Substances 0.000 description 3
- 230000035495 ADMET Effects 0.000 description 2
- 108091000080 Phosphotransferase Proteins 0.000 description 2
- 238000010535 acyclic diene metathesis reaction Methods 0.000 description 2
- 125000003118 aryl group Chemical group 0.000 description 2
- 125000004429 atom Chemical group 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000012938 design process Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 229940000406 drug candidate Drugs 0.000 description 2
- 238000009510 drug design Methods 0.000 description 2
- 238000009509 drug development Methods 0.000 description 2
- 238000009472 formulation Methods 0.000 description 2
- 230000005764 inhibitory process Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000012886 linear function Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000002547 new drug Substances 0.000 description 2
- 102000020233 phosphotransferase Human genes 0.000 description 2
- 229920000642 polymer Polymers 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 150000003384 small molecules Chemical class 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 229910052725 zinc Inorganic materials 0.000 description 2
- 241000531891 Alburnus alburnus Species 0.000 description 1
- WKBOTKDWSSQWDR-UHFFFAOYSA-N Bromine atom Chemical compound [Br] WKBOTKDWSSQWDR-UHFFFAOYSA-N 0.000 description 1
- 108010033040 Histones Proteins 0.000 description 1
- 101000801295 Homo sapiens Protein O-mannosyl-transferase TMTC3 Proteins 0.000 description 1
- 108010019437 Janus Kinase 2 Proteins 0.000 description 1
- 102000006503 Janus Kinase 2 Human genes 0.000 description 1
- 208000014767 Myeloproliferative disease Diseases 0.000 description 1
- 229910019142 PO4 Inorganic materials 0.000 description 1
- 102100033736 Protein O-mannosyl-transferase TMTC3 Human genes 0.000 description 1
- 238000004617 QSAR study Methods 0.000 description 1
- 208000005485 Thrombocytosis Diseases 0.000 description 1
- 230000004721 adaptive immunity Effects 0.000 description 1
- 230000003416 augmentation Effects 0.000 description 1
- 230000033228 biological regulation Effects 0.000 description 1
- GDTBXPJZTBHREO-UHFFFAOYSA-N bromine Substances BrBr GDTBXPJZTBHREO-UHFFFAOYSA-N 0.000 description 1
- 229910052794 bromium Inorganic materials 0.000 description 1
- -1 butane Chemical class 0.000 description 1
- 239000001273 butane Substances 0.000 description 1
- 125000002915 carbonyl group Chemical group [*:2]C([*:1])=O 0.000 description 1
- 125000003178 carboxy group Chemical group [H]OC(*)=O 0.000 description 1
- 230000011712 cell development Effects 0.000 description 1
- 230000024245 cell differentiation Effects 0.000 description 1
- 230000010261 cell growth Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 239000013078 crystal Substances 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 210000000805 cytoplasm Anatomy 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- PXBRQCKWGAHEHS-UHFFFAOYSA-N dichlorodifluoromethane Chemical compound FC(F)(Cl)Cl PXBRQCKWGAHEHS-UHFFFAOYSA-N 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 125000000623 heterocyclic group Chemical group 0.000 description 1
- 238000013537 high throughput screening Methods 0.000 description 1
- 229930195733 hydrocarbon Natural products 0.000 description 1
- 150000002430 hydrocarbons Chemical class 0.000 description 1
- 125000004435 hydrogen atom Chemical group [H]* 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000000338 in vitro Methods 0.000 description 1
- 230000015788 innate immune response Effects 0.000 description 1
- 150000002611 lead compounds Chemical class 0.000 description 1
- 239000003446 ligand Substances 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000003278 mimic effect Effects 0.000 description 1
- 230000035772 mutation Effects 0.000 description 1
- 206010028537 myelofibrosis Diseases 0.000 description 1
- TVMXDCGIABBOFY-UHFFFAOYSA-N n-Octanol Natural products CCCCCCCC TVMXDCGIABBOFY-UHFFFAOYSA-N 0.000 description 1
- IJDNQMDRQITEOD-UHFFFAOYSA-N n-butane Chemical compound CCCC IJDNQMDRQITEOD-UHFFFAOYSA-N 0.000 description 1
- OFBQJSOFQDEBGM-UHFFFAOYSA-N n-pentane Natural products CCCCC OFBQJSOFQDEBGM-UHFFFAOYSA-N 0.000 description 1
- 125000000449 nitro group Chemical group [O-][N+](*)=O 0.000 description 1
- 102000037979 non-receptor tyrosine kinases Human genes 0.000 description 1
- 108091008046 non-receptor tyrosine kinases Proteins 0.000 description 1
- 238000012803 optimization experiment Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 150000002902 organometallic compounds Chemical class 0.000 description 1
- 238000012856 packing Methods 0.000 description 1
- 230000007170 pathology Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 235000021317 phosphate Nutrition 0.000 description 1
- 150000003013 phosphoric acid derivatives Chemical class 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000006798 ring closing metathesis reaction Methods 0.000 description 1
- 238000007142 ring opening reaction Methods 0.000 description 1
- 150000003839 salts Chemical class 0.000 description 1
- 238000003034 scaffold hopping Methods 0.000 description 1
- 230000019491 signal transduction Effects 0.000 description 1
- 230000007781 signaling event Effects 0.000 description 1
- 229910052717 sulfur Inorganic materials 0.000 description 1
- 239000011593 sulfur Substances 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
- 229940121358 tyrosine kinase inhibitor Drugs 0.000 description 1
- 239000005483 tyrosine kinase inhibitor Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16C—COMPUTATIONAL CHEMISTRY; CHEMOINFORMATICS; COMPUTATIONAL MATERIALS SCIENCE
- G16C20/00—Chemoinformatics, i.e. ICT specially adapted for the handling of physicochemical or structural data of chemical particles, elements, compounds or mixtures
- G16C20/50—Molecular design, e.g. of drugs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/004—Artificial life, i.e. computing arrangements simulating life
- G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G06N3/0445—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G06N3/0454—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/088—Non-supervised learning, e.g. competitive learning
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16C—COMPUTATIONAL CHEMISTRY; CHEMOINFORMATICS; COMPUTATIONAL MATERIALS SCIENCE
- G16C20/00—Chemoinformatics, i.e. ICT specially adapted for the handling of physicochemical or structural data of chemical particles, elements, compounds or mixtures
- G16C20/30—Prediction of properties of chemical compounds, compositions or mixtures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16C—COMPUTATIONAL CHEMISTRY; CHEMOINFORMATICS; COMPUTATIONAL MATERIALS SCIENCE
- G16C20/00—Chemoinformatics, i.e. ICT specially adapted for the handling of physicochemical or structural data of chemical particles, elements, compounds or mixtures
- G16C20/60—In silico combinatorial chemistry
- G16C20/62—Design of libraries
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16C—COMPUTATIONAL CHEMISTRY; CHEMOINFORMATICS; COMPUTATIONAL MATERIALS SCIENCE
- G16C20/00—Chemoinformatics, i.e. ICT specially adapted for the handling of physicochemical or structural data of chemical particles, elements, compounds or mixtures
- G16C20/70—Machine learning, data mining or chemometrics
Definitions
- the subject matter described herein relates to molecule design. More particularly, the subject matter described herein relates to methods, systems, and non-transitory computer readable media for automated design of molecules with desired properties using artificial intelligence.
- the drug discovery pipeline is notoriously sequential. Hits from a high throughput screen (HTS) are slowly progressed toward promising lead compounds.
- the ADMET and selectivity profile is optimized with a challenge to maintain high potency and efficacy. High failure rates in late-stage clinical trials could be potentially avoided if the relevant information were available earlier or if the available data could provide clues as to whether a drug will actually perform as expected in clinical practice.
- the crucial step is the formulation of a well-motivated hypothesis for compound generation (de novo design) or compound picking from a library based on the available SAR data.
- an interdisciplinary team of scientists generates the new hypothesis by relying on their expertise and medicinal chemistry intuition. Therefore, any design hypothesis is easily biased towards preferred chemistry 9 or model interpretation. 10
- Reinforcement learning is a subset of artificial intelligence, which is used to solve dynamic decision problems.
- RL involves analyzing possible actions, estimating the statistical relationship between the actions and their possible outcomes and then determining a treatment regime that attempts to find the most desirable outcome based on the analysis
- the integration of reinforcement learning and neural networks dated back to 1990s 21 .
- new powerful algorithmic approaches are emerging.
- Most recently RL has led to superhuman performance in game Go, 23 considered practically intractable due to the theoretical complexity of over 10 140 . 24 Therefore, we see an example of attacking a problem of the difficulty comparable to a chemical space exploration without brute-force computing every possible solution.
- the subject matter described herein may be implemented in software in combination with hardware and/or firmware.
- the subject matter described herein may be implemented in software executed by a processor, which may be a general-purpose processor, a special purpose processor, such as a graphics processing unit (GPU) or a field programmable gate array (FPGA).
- a processor which may be a general-purpose processor, a special purpose processor, such as a graphics processing unit (GPU) or a field programmable gate array (FPGA).
- the subject matter described herein may be implemented using a computer readable medium having stored thereon computer executable instructions that when executed by the processor of a computer control the computer to perform steps.
- Exemplary computer readable media suitable for implementing the subject matter described herein include non-transitory devices, such as disk memory devices, chip memory devices, programmable logic devices, and application specific integrated circuits.
- a computer readable medium that implements the subject matter described herein may be located on a single device or computing platform or may be distributed across multiple devices or computing platforms
- FIG. 1 is a block diagram of a general workflow for a reinforcement learning system for novel molecule generation
- FIG. 2 is a diagram of a generative stack augmented RNN for use in the generative modeler illustrated in FIG. 1 ;
- the left hand diagram in FIG. 2 illustrates the training step of the generative stack-augmented RNN;
- the right hand diagram in FIG. 2 illustrates the generator step of the generative stack-augmented RNN.
- FIG. 3 is a diagram illustrating sample molecules generated by the generative modeler in FIG. 1 ;
- FIG. 4 illustrates graphs of distributions for properties of un-optimized and untrained data
- FIG. 5 illustrates the evolution of generated structures as a chemical substructure reward increases
- FIG. 6 illustrates example of Stack-RNN cells with interpretable gate activations
- FIG. 7 illustrates clustering of generated molecules by T distributed stochastic neighbor embedding (T-SNE), where molecules are colored based on predicted properties by model P;
- FIG. 8 illustrates distributions of residuals for predictive models
- FIG. 9 illustrates initial (left) and truncated (right) distributions of SMILES links
- FIG. 10 illustrates a learning curve for the generative model
- FIG. 11 illustrates graphs of reward functions
- FIG. 12 illustrates distribution of a Synthetic Accessibility Score (SAS) for the full ChEMBL21 database, random subsample of 1M molecules from ZINC15 and generated dataset of molecules with generator model G; and
- SAS Synthetic Accessibility Score
- FIG. 13 illustrates performance of the generative model G, with and without stack-augmented memory.
- the left hand graph in FIG. 13 illustrates Internal diversity of generated libraries.
- the right hand graph in FIG. 13 illustrates similarity of the generated libraries to the training dataset from the ChEMBL database.
- the subject matter described herein includes an RL based de novo design method for generating chemical compounds with desired physical, chemical or bioactivity properties.
- the general workflow ( FIG. 1 ) is represented by two deep neural networks (generative G and predictive P).
- the process of training consists of two stages. During the first stage, both models are trained separately with supervised learning algorithms, and during the second stage, models are trained jointly with reinforcement learning approach optimizing target properties.
- the generative model plays the role of agent.
- the predictive model plays the role of critic, which estimates the agent's behavior by assigning a numerical reward to every generated molecule. The reward is function of the numerical property predicted by the predictive model.
- the generative model is trained to maximize the expected reward.
- the components illustrated in FIG. 1 can be implemented on a computing platform including at least one processor.
- the set of actions A is defined as an alphabet of SMILES notation.
- the set of states S is defined as all possible strings in the alphabet with lengths from 0 to some T.
- the state s 0 with length 0 is unique and considered to be an initial state.
- the state s T of length T is called the terminal state and it causes episode to end.
- Reward r(s T ) is calculated in the end of an episode, when terminal state is reached. Intermediate rewards r(s T ), t ⁇ T are equal to 0.
- the generator network G can be treated as a policy approximation model.
- G takes previous state s t-1 as an input and estimates probability distribution p(a t
- Reward r(s T ) is a function of the predicted property of s T by the predictive model P.
- J( ⁇ ) estimation is following: sequentially sample a t from the model G for t from 0 to T.
- the unbiased estimation for J( ⁇ ) is the sum of all rewards in every time step which in our case equals to the reward for the terminal state as we assume that intermediate rewards are equal to 0.
- this quantity needed to me maximizes we need to compute its gradient. This can be done, for example with a REINFORCE algorithm 25 which uses approximation of mathematical expectation as a sum, which we provided above, and the following trick:
- the first model G is a generative recurrent neural network 26-28 , which outputs molecules in the simplified molecular-input line-entry system (SMILES) notation 29 .
- SMS simplified molecular-input line-entry system
- Stack-RNN stack-augmented recurrent neural network
- Regular recurrent neural networks like LSTM 31 and GRU 32 are failing to solve the sequence prediction problems due to inability to count.
- One of the challenging examples of sequences which cannot be properly modeled by regular recurrent network is words from Dyck language, a language of balanced strings of brackets 33,34 .
- Another weakness of regular recurrent neural networks is their inability to capture long term dependencies which leads to difficulties in generalizing to longer sequences 35 . All of these properties are required to learn language of SMILES notation. In a valid SMILES molecule, in addition to correct valence for all atoms, one must count, ring opening and closure, as well as bracket sequence with several bracket types. Therefore, Stack RNNs are a proper choice for modeling such sequence dependencies.
- the Stack-RNN defines (See FIG. 2 ) a new neuron or cell structure on top of standard GRU cell. It has two additional multiplicative gates, referred to as the memory stack, which allow the Stack-RNN to learn meaningful long-range interdependencies.
- the stack is a differentiable structure onto and from which continuous vectors are pushed and popped. These traditionally discrete operations are continuous here by letting push and pop operations be real values in the interval (0, 1). Intuitively, we can interpret these values as the degree of certainty with which some controller wishes to push a vector v onto the stack, or pop the top of the stack.
- FIG. 2 illustrates, on the left hand side, the scheme of a generative stack-augmented RNN time step.
- This model has two modes—training and generating.
- the input token is a character of the currently processed SMILES string from the training set.
- the model outputs probability vector p ⁇ (a t
- a vector of parameters ⁇ is optimized by a cross-entropy loss function minimization.
- the input token is a previously generated character.
- a next character a t is sampled randomly from the distribution p ⁇ (a t
- the right hand side of FIG. 2 illustrates the scheme of the predictive model.
- This model takes a SMILES string as an input and provides one real number, which is an estimated property value, as an output. Parameters of the model are trained by l 2 squared loss function minimization.
- the second model P is a predictive model (see FIG. 1 ) for estimating physical, chemical or bioactivity properties of molecules.
- This property prediction model is a deep neural network, which consists of an embedding layer 36 , LSTM layer and two dense layers. This network is designed to calculate user-specified property (activity) of the molecule taking the SMILES string as an input data vector.
- the advantage of such approach is that no numerical descriptors are needed, as it learns directly from SMILES notation.
- Murcko scaffolds 39 between the training and the virtual molecules provide a hierarchical molecular organization scheme by dividing small molecules into R-groups, linkers, and frameworks or scaffolds. They contain the ring systems of a molecule by removing side chain atoms. We find that less than 10% scaffolds are present in ChEMBL. Overall, this analysis suggests that the generative Stack RNN model did not simply memorized training SMILEs sequences but is capable to generate extremely diverse but realistic molecules.
- the reward function in this case was defined as a piecewise linear function of log P with a constant region from 1.0 to 4.0 (see FIG. 11 ).
- the goal was set the goal to uniformly synthesize molecules according to a typical Lipinski's constraints. After training 88% of generated molecules were within drug-like region between 0 and 5.
- JAK2 is non-receptor tyrosine kinase involved in various processes such as cell growth, development, differentiation or histone modifications. It mediates essential signaling events in both innate and adaptive immunity. In the cytoplasm it also plays an important role in signal transduction. 40 Mutations in JAK2 have been implicated in multiple conditions like thrombocythemia, myelofibrosis or myeloproliferative disorders. 41
- the reward function in both cases was defined as exponential functions of pIC50 (see FIG. 11 ).
- the results of optimization are demonstrated in FIG. 11 , graph (c).
- maximization strategy properties of generated molecules were more tightly distributed bet.
- models virtually synthesized known and novel compounds based on one scaffold as well as suggested new scaffolds.
- FIG. 5 illustrates evolution of generated structures as structural reward increases. We see the model progress to generate increasingly complex molecules while maintaining realistic chemistry without very small or large rings.
- a substructure bias could be a valuable tool to generate novel medicinal chemistry hypothesis.
- Table 1 shows a decrease in the proportion of the valid molecules after the optimization.
- Table 1 shows a decrease in the proportion of the valid molecules after the optimization.
- the generative model G tends to find some local optima of the reward function that correspond to invalid molecules, but predictive model P assigns high rewards to these molecules.
- This explanation is also supported by the results of structure bias optimization experiments, as we did not use any predictive models in these experiments and the decrease in the proportion of valid molecules was insignificant.
- SAS synthetic accessibility score
- the library obtained without stack memory showed a decrease of the internal diversity by 0.2 units of the Tanimoto coefficient and yet, a four-fold increase in the number of duplicates, from just about 1% to 5%.
- the right hand graph in FIG. 13 shows that the number of molecules similar to the training dataset (Ts>0.85) for library obtained without stack memory (28% of all molecules) is twice that obtained with stack memory (14%).
- Model interpretation is of utmost importance in any ML study.
- Stack-RNN learns and memorizes useful information from the SMILES string that it is currently processing.
- FIG. 6 lists several examples of cells with interpretable gate activations.
- each line corresponds to activations of a specific neuron at different time steps of processing SMILES string.
- Each letter is colored according to the value of activation in a cool-warm color map from dark blue to dark red—from ⁇ 1 to 1.
- our RNN has several interpretable cells, that can be divided into two groups—chemically sensible, that activates on specific chemical groups or moieties, and syntactic, that keep tracks on numbers, bracket opening and closure, or even keep track when the molecule ends. Unfortunately, large portion of cells that do something, that is not interpretable.
- FIG. 6 illustrates examples of Stack-RNN cells with interpretable gate activations.
- the color-coding in FIG. 6 corresponds to GRU cells with a tan h activation function, where ⁇ 1 is dark blue and 1 is red.
- t-SNE t-Distributed Stochastic Neighbor Embedding
- the partition coefficient ( FIG. 7 , block A) is smoothly changing across the chemical space, and we can observe well defined clustering of molecules with similar log P. In contrast, for melting temperature ( FIG. 7 , block B) there are no such clusters. High and low T m molecules are intermixed together. This observation can be explained by the fact that melting temperature depends not only on a chemical structure of the molecule, but also on intermolecular forces between them and packing into the crystal lattice. Therefore, neural could not find optimal representation for this property. In the case of JAK2 kinase model we could observe two large non-overlapping areas roughly corresponding to inactive (pIC50 ⁇ 6) and active(pIC50>6) compounds.
- Two deep neural networks represent the general RL workflow.
- the process of training consists of two stages. During the first stage, both models are trained separately in a supervised learning fashion, and during the second stage, models are trained jointly with a reinforcement learning method.
- Both neural networks are end-to-end deep learning that do not rely on pre-defined chemical descriptors and can be trained directly from chemical structure as represented by SMILES strings.
- Modeling-ready curated dataset contained 14,176 compounds for log P, 15,549 compounds for JAK2 kinase and 47,425 for melting temperature.
- Each model consists of embedding layer, which transforms sequence of discrete tokens into a vector of 100 continuous numbers, LSTM layer with 100 units and tan h nonlinearity, one dense layers with 100 units and rectify nonlinearity function and one dense layer with one unit and identity activation function. All three models were trained with learning rate decay technique until convergence. curated datasets were divided into training and validation sets in a ratio of 3: 1. The results and accuracy of the model are shown in FIG. 8 .
- the generative model has two modes of processing sequences—training and generating.
- training mode at each time step, the generative network takes a current prefix of training object and predicts the probability distribution of next character. Then, the next character is sampled from this predicted probability distribution and is compared to the ground truth. Afterwards, based on this comparison the cross-entropy loss function is calculated and parameters of the model are updated.
- generating mode at each time step the generative network takes a prefix of already generated sequence and then, similar to training mode, predicts probability distribution of next character and samples it from this predicted distribution. In generating model, we do not update model parameters.
- generative plays the role of agent, whose action space is represented by the SMILES notation alphabet and state space is represented by all possible strings in this alphabet.
- Predictive model plays the role of critic, which estimates the agent's behavior by assigning a numerical reward to every generated molecule. The reward is function of numerical property predicted by predictive model.
- the generative model is trained to maximize the expected reward.
- the whole pipeline is illustrated in FIG. 1 .
- h t is a vector of hidden states
- h t-1 vector of hidden states from the previous time step
- x t input vector at time step t
- W i parameters of the input layers
- W h parameters of the hidden layer
- ⁇ activation function
- the stack memory is used to keep the information and deliver it to the hidden layer at the next time step.
- a stack is a type of persistent memory which can be only accessed through its topmost element.
- the top element of the stack has value s t [0] and is stored at position 0:
- s t [0] a t [PUSH] ⁇ ( Dh t )+ a t [POP] s t-1 [1]+ a t [NO-OP] s t-1 [0].
- s t [ i ] a t [PUSH] s t-1 [ i ⁇ 1]+ a t [POP] s t-1 [ i+ 1]+ a t [NO-OP] s t-1 [ i ].
- P is a matrix of size m ⁇ k and s t-1 k are the first $k$ elements for the top of the stack at time step t ⁇ 1.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- General Engineering & Computer Science (AREA)
- Crystallography & Structural Chemistry (AREA)
- Bioinformatics & Computational Biology (AREA)
- Medicinal Chemistry (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Pharmacology & Pharmacy (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Library & Information Science (AREA)
- Probability & Statistics with Applications (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
Description
- This application claims the priority benefit of U.S. Provisional Patent Application Ser. No. 62/535,069 filed Jul. 20, 2017, the disclosure of which is incorporated herein by reference in its entirety.
- This invention was made with government support under grant number N0014-16-1-2311 awarded by the Office of Naval Research and grant number ACI-1053575 awarded by the National Science Foundation. The government has certain rights in the invention.
- The subject matter described herein relates to molecule design. More particularly, the subject matter described herein relates to methods, systems, and non-transitory computer readable media for automated design of molecules with desired properties using artificial intelligence.
- The analysis of recent trends in drug development and approval presents rather bleak picture.1 The approval of new drugs has been flat over the last two decades. Less than one out of every 10,000 drug candidates becomes an approved marketed drug. Only three out of every 20 approved drugs bring in enough revenue to cover developmental costs. Moreover, it takes approximately 10-15 years and an average cost of $1-3 billion to develop each new drug. Many promising drug candidates fail in phase II and phase III—later stages of the clinical development process. These high attrition rates at a time when projects have already incurred high costs make for very expensive failures. This so-called innovation gap can be attributed to several challenges ranging from drug safety concerns, lack of efficacy to great complexity of diseases and tightened regulations. Thus, pharmaceutical industry is currently challenged to increase the efficiency of drug development.
- Increasingly scientific advancements are more subject to error and harder to reproduce. Human activities are identified as a principal bottleneck in technological innovations, which leads to inefficiencies, potential errors, and incomplete explorations of the hypothesis and data analysis space. Artificial intelligence (AI) systems can radically transform the practice of scientific discovery. The combination of big data and artificial intelligence, referred to by some as the fourth industrial revolution.2 Today as machine learning also enables our computers to teach themselves drive cars or automatically understand speech. AI is revolutionizing radiology, pathology along with other medical specialties.3,4 Application of Deep Learning (DL) see significant improvement in docking scoring5, learning from small data6, reaction mechanism7 and energy prediction.8
- The drug discovery pipeline is notoriously sequential. Hits from a high throughput screen (HTS) are slowly progressed toward promising lead compounds. Next, the ADMET and selectivity profile is optimized with a challenge to maintain high potency and efficacy. High failure rates in late-stage clinical trials could be potentially avoided if the relevant information were available earlier or if the available data could provide clues as to whether a drug will actually perform as expected in clinical practice. The crucial step is the formulation of a well-motivated hypothesis for compound generation (de novo design) or compound picking from a library based on the available SAR data. Commonly, an interdisciplinary team of scientists generates the new hypothesis by relying on their expertise and medicinal chemistry intuition. Therefore, any design hypothesis is easily biased towards preferred chemistry9 or model interpretation.10
- Idea of automated drug design is not new.11,12 It has contributed to drug discovery projects since 2000s by suggesting novel molecular structures with desired properties from scratch and has become an active field of research. In an attempt to design new compounds, both a medicinal chemist and algorithm is confronted with a virtually infinite chemical space. Today the range of potential drug-like molecules is estimated to be between 1030 and 1060.13,14 Unfortunately, such a large space prohibits exhaustive searching, despite great advances in high-throughput screening (HTS) technology.15 Instead of the systematic construction and evaluation of each individual compound, navigation in the de novo design process relies on the principle of local optimization, which does not necessarily lead to the optimal solution: the design process converges on a local or ‘practical’ optimum by stochastic sampling or restricts the search to a defined section of chemical space which can be screened exhaustively.11,16-18 However, recently a method for exploring chemical space based on continuous encodings of molecules was proposed.19 It allows directed gradient-based search through chemical space. Other method generating focused molecule libraries with Recurrent Neural Networks is also proposed.20
- Here we propose a novel method based on deep reinforcement learning (RL) for generating chemical compounds with desired physical, chemical or bioactivity properties. Reinforcement learning (RL) is a subset of artificial intelligence, which is used to solve dynamic decision problems. RL involves analyzing possible actions, estimating the statistical relationship between the actions and their possible outcomes and then determining a treatment regime that attempts to find the most desirable outcome based on the analysis The integration of reinforcement learning and neural networks dated back to 1990s21. However, with recent achievements of DL, benefiting from big data, new powerful algorithmic approaches are emerging. We are currently witnessing the renaissance of reinforcement learning22, especially, the combination of reinforcement learning and deep neural networks, i.e., deep reinforcement learning (Deep RL). Most recently RL has led to superhuman performance in game Go,23 considered practically intractable due to the theoretical complexity of over 10140.24 Therefore, we see an example of attacking a problem of the difficulty comparable to a chemical space exploration without brute-force computing every possible solution.
- The subject matter described herein may be implemented in software in combination with hardware and/or firmware. For example, the subject matter described herein may be implemented in software executed by a processor, which may be a general-purpose processor, a special purpose processor, such as a graphics processing unit (GPU) or a field programmable gate array (FPGA). In one exemplary implementation, the subject matter described herein may be implemented using a computer readable medium having stored thereon computer executable instructions that when executed by the processor of a computer control the computer to perform steps. Exemplary computer readable media suitable for implementing the subject matter described herein include non-transitory devices, such as disk memory devices, chip memory devices, programmable logic devices, and application specific integrated circuits. In addition, a computer readable medium that implements the subject matter described herein may be located on a single device or computing platform or may be distributed across multiple devices or computing platforms.
-
FIG. 1 is a block diagram of a general workflow for a reinforcement learning system for novel molecule generation; -
FIG. 2 is a diagram of a generative stack augmented RNN for use in the generative modeler illustrated inFIG. 1 ; The left hand diagram inFIG. 2 illustrates the training step of the generative stack-augmented RNN; The right hand diagram inFIG. 2 illustrates the generator step of the generative stack-augmented RNN. -
FIG. 3 is a diagram illustrating sample molecules generated by the generative modeler inFIG. 1 ; -
FIG. 4 illustrates graphs of distributions for properties of un-optimized and untrained data; -
FIG. 5 illustrates the evolution of generated structures as a chemical substructure reward increases; -
FIG. 6 illustrates example of Stack-RNN cells with interpretable gate activations; -
FIG. 7 illustrates clustering of generated molecules by T distributed stochastic neighbor embedding (T-SNE), where molecules are colored based on predicted properties by model P; -
FIG. 8 illustrates distributions of residuals for predictive models; -
FIG. 9 illustrates initial (left) and truncated (right) distributions of SMILES links; -
FIG. 10 illustrates a learning curve for the generative model; -
FIG. 11 illustrates graphs of reward functions; -
FIG. 12 illustrates distribution of a Synthetic Accessibility Score (SAS) for the full ChEMBL21 database, random subsample of 1M molecules from ZINC15 and generated dataset of molecules with generator model G; and -
FIG. 13 illustrates performance of the generative model G, with and without stack-augmented memory. The left hand graph inFIG. 13 illustrates Internal diversity of generated libraries. The right hand graph inFIG. 13 illustrates similarity of the generated libraries to the training dataset from the ChEMBL database. - The subject matter described herein includes an RL based de novo design method for generating chemical compounds with desired physical, chemical or bioactivity properties. The general workflow (
FIG. 1 ) is represented by two deep neural networks (generative G and predictive P). The process of training consists of two stages. During the first stage, both models are trained separately with supervised learning algorithms, and during the second stage, models are trained jointly with reinforcement learning approach optimizing target properties. In this system, the generative model plays the role of agent. The predictive model plays the role of critic, which estimates the agent's behavior by assigning a numerical reward to every generated molecule. The reward is function of the numerical property predicted by the predictive model. The generative model is trained to maximize the expected reward. The components illustrated inFIG. 1 can be implemented on a computing platform including at least one processor. - Reinforcement Learning Formulation.
- Both generative G and predictive model P are combined into one RL system. The set of actions A is defined as an alphabet of SMILES notation. The set of states S is defined as all possible strings in the alphabet with lengths from 0 to some T. The state s0 with
length 0 is unique and considered to be an initial state. The state sT of length T is called the terminal state and it causes episode to end. The subset of terminal states S*={sT E S} of S which contains all the states sT with length T is called the terminal states set. Reward r(sT) is calculated in the end of an episode, when terminal state is reached. Intermediate rewards r(sT), t<T are equal to 0. In these terms, the generator network G can be treated as a policy approximation model. At each time step t, 0<t<T, G takes previous state st-1 as an input and estimates probability distribution p(at|st-1) of the next action. Afterwards, the next action at is sampled from this estimated probability. Reward r(sT) is a function of the predicted property of sT by the predictive model P. -
r(s T)=ƒT)), - where ƒ is chosen expertly depending on the task. Some examples of the functions ƒ are provided further in the computational experiment section. Given these notations and assumptions, the problem of generating chemical compounds with desired properties can be formulated as a task of finding a vector of parameters Θ of policy network G which maximizes the expected reward:
-
- This sum iterates over the set S* of terminal states. In our case, this set is exponential and the sum cannot be exactly computed. According to the law of large numbers, we can approximate this sum as a mathematical expectation by sampling terminal sequences from the model distribution: J(Θ)=[r(sT)|s0, Θ]= a
1 ˜pΘ (a1 |s0 ) a2 ˜pΘ (a2 |s1 ) . . . aT ·pΘ (aT |sT-1 )r(sT). - The procedure for J(Θ) estimation is following: sequentially sample at from the model G for t from 0 to T. The unbiased estimation for J(Θ) is the sum of all rewards in every time step which in our case equals to the reward for the terminal state as we assume that intermediate rewards are equal to 0. As this quantity needed to me maximizes, we need to compute its gradient. This can be done, for example with a REINFORCE algorithm25 which uses approximation of mathematical expectation as a sum, which we provided above, and the following trick:
-
- Therefore, the gradient of J(0) can be written as:
-
- which gives an algorithm for ∂ΘJ(Θ) estimation.
- Neural Networks Architectures.
- The first model G is a generative recurrent neural network26-28, which outputs molecules in the simplified molecular-input line-entry system (SMILES) notation29. We use a special type stack-augmented recurrent neural network (Stack-RNN)30.
- Regular recurrent neural networks like LSTM31 and GRU32 are failing to solve the sequence prediction problems due to inability to count. One of the challenging examples of sequences which cannot be properly modeled by regular recurrent network is words from Dyck language, a language of balanced strings of brackets33,34. Another weakness of regular recurrent neural networks is their inability to capture long term dependencies which leads to difficulties in generalizing to longer sequences35. All of these properties are required to learn language of SMILES notation. In a valid SMILES molecule, in addition to correct valence for all atoms, one must count, ring opening and closure, as well as bracket sequence with several bracket types. Therefore, Stack RNNs are a proper choice for modeling such sequence dependencies.
- The Stack-RNN defines (See
FIG. 2 ) a new neuron or cell structure on top of standard GRU cell. It has two additional multiplicative gates, referred to as the memory stack, which allow the Stack-RNN to learn meaningful long-range interdependencies. The stack is a differentiable structure onto and from which continuous vectors are pushed and popped. These traditionally discrete operations are continuous here by letting push and pop operations be real values in the interval (0, 1). Intuitively, we can interpret these values as the degree of certainty with which some controller wishes to push a vector v onto the stack, or pop the top of the stack. -
FIG. 2 illustrates, on the left hand side, the scheme of a generative stack-augmented RNN time step. This model has two modes—training and generating. During training, the input token is a character of the currently processed SMILES string from the training set. The model outputs probability vector pΘ(at|st-1) of the next character given the prefix. A vector of parameters Θ is optimized by a cross-entropy loss function minimization. During generating, the input token is a previously generated character. A next character at is sampled randomly from the distribution pΘ(at|st-1). The right hand side ofFIG. 2 illustrates the scheme of the predictive model. This model takes a SMILES string as an input and provides one real number, which is an estimated property value, as an output. Parameters of the model are trained by l2 squared loss function minimization. - The second model P is a predictive model (see
FIG. 1 ) for estimating physical, chemical or bioactivity properties of molecules. This property prediction model is a deep neural network, which consists of an embedding layer36, LSTM layer and two dense layers. This network is designed to calculate user-specified property (activity) of the molecule taking the SMILES string as an input data vector. The advantage of such approach is that no numerical descriptors are needed, as it learns directly from SMILES notation. - Unbiased molecule generation. To demonstrate the versatility of the baseline (unbiased) Stack RNN we generated a dataset of over one million virtually synthesized compounds. All structures are available for download from [https://www.dropbox.com/s/ghd668rulye6rxu/1M_dataset.zip?dl=0]. Random examples of the generated compounds are illustrated in
FIG. 3 . - Over 95% of generated structures were valid chemically sensible molecules. The validity check was performed by the structure checker from ChemAxon37. When compared with ChEMBL (See Methods), model produced less than 0.1% of structures from the training dataset. Additional comparison with ZINC15 database38 of 320M synthetically accessible drug-like molecules show match of about 2% structures.
- In order to understand the novelty of the de novo generated molecules we analyzed the Murcko scaffolds39 between the training and the virtual molecules Murcko scaffolds provide a hierarchical molecular organization scheme by dividing small molecules into R-groups, linkers, and frameworks or scaffolds. They contain the ring systems of a molecule by removing side chain atoms. We find that less than 10% scaffolds are present in ChEMBL. Overall, this analysis suggests that the generative Stack RNN model did not simply memorized training SMILEs sequences but is capable to generate extremely diverse but realistic molecules.
- RL System.
- To explore the utility of the RL algorithm in a drug design setting we have set up a multiple case studies that optimize three types of rewards: a) physical property, b) biological activity and c) chemical substructure. For physical properties, we selected melting temperature (Tm) and n-octanol/water partition coefficient (log P). Inhibition potency in form of IC50 to
Janus kinase 2 or JAK2 was used as biological activity. IC50 is the concentration of drug that is required to inhibit 50% of a specific biological target in vitro. Finally, number of benzene rings and number of substituents (like —OH, —NH2, —CH3—CN, etc.) was used as a structural reward.FIG. 4 shows distributions of predicted properties of interest before and after experiments. In both cases, we sampled 10,000 molecules by the untrained and optimized generative models and calculated their properties with a corresponding predictive model. Values of the substructure features were calculated directly from the 2D structure. Table 1 summarizes analysis of generated molecules and descriptive statistics. - Melting Temperature.
- In this experiment, we set two goals to minimize and maximize the target property. Upon minimization the mean of the distribution was shifted by 44° C. Optimized generator virtually synthesized simple hydrocarbons like butane, and poly-halogenated compounds CF2Cl2 and C6H4F2. CF4 molecule has the lowest Tm=−184° C. in the produced dataset. This property minimization strategy is extremely effective, it allowed discovery of molecules in the regions of chemical space far beyond available in the training examples. In the maximization regime mean of the melting temperature is increased by 20° C. to 200° C. The generator synthesized substantially more complex molecules with an abundance of sulfur heterocycles, phosphates as well as conjugated double bonds. The reward functions in both cases are defined as piecewise linear function from melting temperature (see
FIG. 11 ). - Log P.
- To better mimic requirements of drug-likeliness instead of property minimization, we imposed to the range. The reward function in this case was defined as a piecewise linear function of log P with a constant region from 1.0 to 4.0 (see
FIG. 11 ). In other words, we set the goal to uniformly synthesize molecules according to a typical Lipinski's constraints. After training 88% of generated molecules were within drug-like region between 0 and 5. - Inhibition of JAK2 Kinase.
- In the third experiment, perhaps most relevant to the practical drug discovery application we directly minimized and maximized pIC50 values for JAK2 kinase. JAK2 is non-receptor tyrosine kinase involved in various processes such as cell growth, development, differentiation or histone modifications. It mediates essential signaling events in both innate and adaptive immunity. In the cytoplasm it also plays an important role in signal transduction.40 Mutations in JAK2 have been implicated in multiple conditions like thrombocythemia, myelofibrosis or myeloproliferative disorders.41
- The reward function in both cases (min and max) was defined as exponential functions of pIC50 (see
FIG. 11 ). The results of optimization are demonstrated inFIG. 11 , graph (c). With minimization, the mean of predicted pIC050 distribution was shifted by about one unit. However, distribution is heavily tailed, and 24% of molecules are predicted to have practically no activity (pIC50<=4). In the maximization strategy, properties of generated molecules were more tightly distributed bet. In both cases models virtually synthesized known and novel compounds based on one scaffold as well as suggested new scaffolds. - The generation of known compounds (i.e. not included in the training set) can be regarded as model validation. Indeed, the system retrospectively discovered 793 commercially available compounds deposited in the ZINC database, which constituted about 5% of the total generated library. Importantly, as many as 15 of them (exemplified by ZINC263823677-http://zinc15.docking.org/substances/ZINC000263823677/ and ZINC271402431 http://zinc15.docking.org/substances/ZINC000271402431/) were actually annotated as possible tyrosine kinase inhibitors.
- Substructure Bias.
- Finally, we also performed two simple experiments mimicking biasing chemical libraries to a user-defined substructure without predicting any property. We defined the reward function as the exponent of a) number of benzene rings (-Ph) and b) total number of small groups substituents. Among all case studies described, structure bias was the easiest to optimize.
FIG. 5 illustrates evolution of generated structures as structural reward increases. We see the model progress to generate increasingly complex molecules while maintaining realistic chemistry without very small or large rings. - In realistic drug discovery applications, a substructure bias could be a valuable tool to generate novel medicinal chemistry hypothesis. One can bias de novo generated molecules to a specific chemotype or a scaffold. Conversely, it also allows avoidance of particular chemical groups or substructures (like bromine or carboxyl group). Finally, one could use a substructure or pharmacophore similarity42 reward to explore chemical space with finer granularity.
- Table 1 shows a decrease in the proportion of the valid molecules after the optimization. We may explain this phenomenon by the weaknesses of predictive models P (See
FIG. 1 ) and the integration of predictive and generative models into a single design system. We presume that the generative model G tends to find some local optima of the reward function that correspond to invalid molecules, but predictive model P assigns high rewards to these molecules. This explanation is also supported by the results of structure bias optimization experiments, as we did not use any predictive models in these experiments and the decrease in the proportion of valid molecules was insignificant. We also noticed, that among all experiments with predictive models, those with Log P optimization showed the highest proportion of valid molecules and, at the same time, the predictive model for Log P estimation had the highest accuracy R2=0.91 (see Methods). Probably it is harder for RL system to exploit high quality predictive model P and produce fictitious SMILES strings with predicted properties in the desired region. -
TABLE 1 Comparison of statistics for optimized, untrained and training molecules datasets Mean Match with Match with Valid Mean value of ZINC15 ChEMBL molecules, molar target database38, database43, Property % mass property % % Melting untrained 95 435.4 181.30 4.7 1.5 temperature minimized 31 279.6 137.17 4.6 1.6 maximized 53 413.2 200.715 2.4 0.9 pIC50 for untrained 95 435.4 5.70 4.7 1.5 jak2 kinase minimized 60 481.8 4.89 2.5 1.0 maximized 45 275.4 7.85 4.5 1.8 log P untrained 95 435.4 3.63 4.7 1.5 optimized 70 369.7 2.58 5.8 1.8 Number of untrained 95 435.4 0.59 4.7 1.5 benzene rings optimized 83 496.0 2.41 5.5 1.6 Number of untrained 95 435.4 3.8 4.7 1.5 substituents optimized 80 471.7 7.93 3.1 0.7 - In addition to passing the structure checker, an important requirement for de novo generated molecules is their synthetic feasibility. To this end, we employed the synthetic accessibility score (SAS) method (41), which relies on the knowledge extracted from known synthetic reactions and adds penalty for high molecular complexity. For ease of interpretation, SAS is scaled to be between 1 and 10. Molecules with the high SAS value, typically above 6 are considered difficult to synthesize, whereas, molecules with the low SAS values are easily synthetically accessible. The distribution of SAS values calculated for 1M molecules generated is shown in
FIG. 12 . To illustrate the robustness of the de novo generated chemical library, we compared its SAS distribution with that of the SAS values both for the full ChEMBL library (˜1.5M molecules) and for 1M random sample of molecules in ZINC. Similar to typical commercial vendor libraries, distribution of SAS for generated compounds is skewed towards more easily synthesizable molecules. Median SAS values were 2.9 for ChEMBL and 3.1 for both ZINC and ReLeaSE. Over 99.5% of de novo generated molecules had SAS values below 6. Therefore, despite their high novelty, vast majority of generated compounds can be considered as synthetically accessible. - To assess the importance of using stack memory augmented network as described in Methods, we compared several characteristics of chemical libraries generated by models developed either with or without stack memory. For this purpose, we trained another generative recurrent neural network with the same architecture but without using stack memory. Libraries were compared by the percentage of valid generated SMILES, internal diversity, and similarity of the generated molecules to those in the training dataset (ChEMBL). The model without stack memory showed that only 86% of molecules in the respective library were valid compared to 95% of valid molecules in the library generated with stack memory network. As expected (cf. the justification for using stack memory augmented network in Methods), in the former library, syntactic errors such as open brackets, unclosed cycles and incorrect bond types in SMILES strings were more frequent. Based on the analysis of respective sets of 10000 molecules generated by each method (See left hand graph in
FIG. 13 ), the library obtained without stack memory showed a decrease of the internal diversity by 0.2 units of the Tanimoto coefficient and yet, a four-fold increase in the number of duplicates, from just about 1% to 5%. In addition, the right hand graph inFIG. 13 shows that the number of molecules similar to the training dataset (Ts>0.85) for library obtained without stack memory (28% of all molecules) is twice that obtained with stack memory (14%). These results clearly highlight the advantages of using neural network with memory for generating the highest number of realistic and predominantly novel molecules, which is one of the chief objectives of de novo chemical design. - Model interpretation is of utmost importance in any ML study. In this section, we demonstrate how Stack-RNN learns and memorizes useful information from the SMILES string that it is currently processing. We manually analyzed neuron's gate activations of the neural network while it processes the input data.
-
FIG. 6 lists several examples of cells with interpretable gate activations. InFIG. 6 , each line corresponds to activations of a specific neuron at different time steps of processing SMILES string. Each letter is colored according to the value of activation in a cool-warm color map from dark blue to dark red—from −1 to 1. We discovered that that our RNN has several interpretable cells, that can be divided into two groups—chemically sensible, that activates on specific chemical groups or moieties, and syntactic, that keep tracks on numbers, bracket opening and closure, or even keep track when the molecule ends. Unfortunately, large portion of cells that do something, that is not interpretable. -
FIG. 6 illustrates examples of Stack-RNN cells with interpretable gate activations. The color-coding inFIG. 6 corresponds to GRU cells with a tan h activation function, where −1 is dark blue and 1 is red. - For instance, we see cells acting on presence of carbonyl group, aromatic groups or NH moiety in heterocycles. We also discovered that in two of these three examples there are counter-cells that deactivate in presence of the same chemical groups.
- In order to understand the how the generative model populates chemical space, we used t-Distributed Stochastic Neighbor Embedding (t-SNE) for dimensionality reduction44. We selected datasets for three endpoints (melting temperature, JAK2 kinase and log P) produced with corresponding optimized generative models G. For every molecule, we calculated a latent vector of representation from the feed-forward layer with ReLU activation function in the predictive model P for the corresponding property and constructed 2D projection using t-SNE. Obtained projections are illustrated in
FIG. 7 . Every point corresponds to a molecule and is colored according to its property value. - The partition coefficient (
FIG. 7 , block A) is smoothly changing across the chemical space, and we can observe well defined clustering of molecules with similar log P. In contrast, for melting temperature (FIG. 7 , block B) there are no such clusters. High and low Tm molecules are intermixed together. This observation can be explained by the fact that melting temperature depends not only on a chemical structure of the molecule, but also on intermolecular forces between them and packing into the crystal lattice. Therefore, neural could not find optimal representation for this property. In the case of JAK2 kinase model we could observe two large non-overlapping areas roughly corresponding to inactive (pIC50<6) and active(pIC50>6) compounds. - In this work, we propose a deep reinforcement learning approach for de novo molecular design. This strategy allows generating chemical compounds with desired properties. Two deep neural networks—generative and predictive, represent the general RL workflow. The process of training consists of two stages. During the first stage, both models are trained separately in a supervised learning fashion, and during the second stage, models are trained jointly with a reinforcement learning method. Both neural networks are end-to-end deep learning that do not rely on pre-defined chemical descriptors and can be trained directly from chemical structure as represented by SMILES strings.
- This distinction makes it clearly differentiated from traditional QSAR methods. Therefore, it is very simple to setup and to use. This approach and can be easily adapted to different chemical domains (industrial chemistry, polymers, etc.) or datasets (PubChem, in-house private data) with minimal modifications. It does not depend on domain knowledge or choice of a particular descriptor set.
- As proof of principle, the model was tested on optimization of three diverse types of endpoints: physical properties, biological activity and chemical substructure bias. Flexible reward function could minimize, maximize or impose a desired range to a property of interest. We also released a dataset of over 1M of novel compounds that model “virtually synthesized”.
- In this study, we conducted computational experiments, which demonstrate the efficiency of proposed strategy in a single-task regime. I.e. each endpoint as optimized separately. However, typical drug discovery pipeline includes multi-objective optimization like potency, selectivity, solubility, ADMET properties, and so on. Our future work will be focused in this direction too. Nevertheless, the multi-objective optimization could be expressed as a combination between multiple tasks of three types we described.
- Data.
- Melting point dataset was extracted from literature.45 The PHYSPROP database46 used to extract octanol/water partition coefficient, log P for diverse set of molecules. Experimental 1050 data tested against JAK2 kinase (CHEMBL ID 2971) was extracted from ChEMBL43, PubChem47 and in-house private databases. Compounds that had inconclusive IC50 values were considered unreliable and were not included in the modeling.
- Data Curation.
- Compiled datasets of compounds were carefully curated following the protocols proposed by Fourches et al.48 Briefly, explicit hydrogens were added, whereas specifics chemotypes such as aromatic and nitro groups were normalized using ChemAxon Standardizer. Polymers, inorganic salts, organometallic compounds, mixtures, and duplicates were removed. Modeling-ready curated dataset contained 14,176 compounds for log P, 15,549 compounds for JAK2 kinase and 47,425 for melting temperature.
- Predictive Models.
- We trained predictive models for three different properties—melting temperature, log P and pIC50 for jak2 kinase. Each model consists of embedding layer, which transforms sequence of discrete tokens into a vector of 100 continuous numbers, LSTM layer with 100 units and tan h nonlinearity, one dense layers with 100 units and rectify nonlinearity function and one dense layer with one unit and identity activation function. All three models were trained with learning rate decay technique until convergence. curated datasets were divided into training and validation sets in a ratio of 3: 1. The results and accuracy of the model are shown in
FIG. 8 . - In the first stage, we pretrain a generative model on a ChEMBL2143 dataset of approximately 1.5M drug-like compounds, so that the model is capable of producing chemically feasible molecules, but without property optimization. This network has 1500 units in recurrent GRU32 layer and 512 units in stack augmentation layer. The model was trained on GPU for 10000 epochs. The learning curve is illustrated in
FIG. 10 . - The generative model has two modes of processing sequences—training and generating. In training mode at each time step, the generative network takes a current prefix of training object and predicts the probability distribution of next character. Then, the next character is sampled from this predicted probability distribution and is compared to the ground truth. Afterwards, based on this comparison the cross-entropy loss function is calculated and parameters of the model are updated. In generating mode at each time step the generative network takes a prefix of already generated sequence and then, similar to training mode, predicts probability distribution of next character and samples it from this predicted distribution. In generating model, we do not update model parameters.
- At second stage, we combine both generative and predictive model into one reinforcement learning system. In this system, generative plays the role of agent, whose action space is represented by the SMILES notation alphabet and state space is represented by all possible strings in this alphabet. Predictive model plays the role of critic, which estimates the agent's behavior by assigning a numerical reward to every generated molecule. The reward is function of numerical property predicted by predictive model. At this stage, the generative model is trained to maximize the expected reward. The whole pipeline is illustrated in
FIG. 1 . - We trained a stack-augmented RNN as a generative model. As a training dataset, we took ChEMBL database of drug-like compounds, which includes approximately 1.5 million of SMILES strings. For training, we selected from initial training dataset just those molecules, which SMILES notation length was less than 100 characters. The length of 100 was chosen because more than 97% of SMILES in training dataset were 100 characters or less (see
FIG. 9 ). - Stack-Augmented Recurrent Neural Network.
- This section describes generative model G in more details. We assume, that the data is sequential, which means that it comes in the form of discrete tokens, for example, characters. The goal is to build a model, which is able to predict the next token taking all the previous ones. The regular recurrent neural network has an input layer and a hidden layer. At time step $t$ neural network takes the embedding vector of token number $t$ from the sequence as an input and models the probability distribution of the next token given all previous tokens, so that the next token can be sampled from this distribution. Information of all previously observed tokens is aggregated in the hidden layer. This can be written as follows:
-
h t=σ(W i x t +W h h t-1), - where ht is a vector of hidden states, ht-1—vector of hidden states from the previous time step, xt—input vector at time step t, Wi—parameters of the input layers, Wh—parameters of the hidden layer and σ—activation function.
- The stack memory is used to keep the information and deliver it to the hidden layer at the next time step. A stack is a type of persistent memory which can be only accessed through its topmost element. There are three basic operations supported by the stack: POP operation, which deletes an element from the top of the stack, PUSH operation, which puts a new element to the top of our stack and also a NO-OP operation, which performs no action. The top element of the stack has value st[0] and is stored at position 0:
-
s t[0]=a t[PUSH]σ(Dh t)+a t[POP]s t-1[1]+a t[NO-OP]s t-1[0]. - where D is 1×m matrix and at=[at[PUSH], at[POP], at[NO-OP]] is a vector of stack control variables, which define the next operation to be performed. If at[POP] is equal to 1, then the value below is used to replace the top element of the stack. If at[PUSH] is equal to 1, then a new value will be added to the top and all the rest values will be moved down. If at[NO-OP] equals 1 then stack keeps the same value on top.
- Similar rule is applied to the elements of the stack at a depth i>0:
-
s t[i]=a t[PUSH]s t-1[i−1]+a t[POP]s t-1[i+1]+a t[NO-OP]s t-1[i]. - Now the hidden layer ht is updated as:
-
h t=σ(Ux t +Rh t-1 +Ps t-1 k), - where P is a matrix of size m×k and st-1 k are the first $k$ elements for the top of the stack at time step t−1.
- The disclosure of each of the following references is incorporated herein by reference in its entirety.
-
- (1) Scannell, J. W.; Blanckley, A.; Boldon, H.; Warrington, B. Diagnosing the Decline in Pharmaceutical R&D Efficiency. Nat. Rev. Drug Discov. 2012, 11, 191-200.
- (2) Artificial intelligence: The return of the machinery question http://www.economist.com/news/special-report/21700761-after-many-false-starts-artificial-intelligence-has-taken-will-it-cause-mass (accessed Feb. 23, 2017).
- (3) Jha, S.; Topol, E. J. Adapting to Artificial Intelligence. JAMA 2016, 316, 2353.
- (4) Chockley, K.; Emanuel, E. The End of Radiology? Three Threats to the Future Practice of Radiology. J. Am. Coll. Radiol. 2016, 13, 1415-1420.
- (5) Ragoza, M.; Hochuli, J.; Idrobo, E.; Sunseri, J.; Koes, D. R. Protein-Ligand Scoring with Convolutional Neural Networks. J. Chem. Inf. Model. 2017, 57, 942-957.
- (6) Altae-Tran, H.; Ramsundar, B.; Pappu, A. S.; Pande, V. Low Data Drug Discovery with One-Shot Learning. ACS Cent. Sci. 2017, 3, 283-293.
- (7) Segler, M. H. S.; Waller, M. P. Modelling Chemical Reasoning to Predict and Invent Reactions. Chem.—A Eur. J. 2017, 23, 6118-6128.
- (8) Smith, J. S.; lsayev, O.; Roitberg, A. E. ANI-1: An Extensible Neural Network Potential with DFT Accuracy at Force Field Computational Cost. Chem. Sci. 2017, 8, 3192-3203.
- (9) Schnecke, V.; Boström, J. Computational Chemistry-Driven Decision Making in Lead Generation. Drug Discovery Today, 2006, 11, 43-50.
- (10) Macarron, R. Critical Review of the Role of HTS in Drug Discovery. Drug Discovery Today. 2006, pp. 277-279.
- (11) Schneider, G.; Fechner, U. Computer-Based de Novo Design of Drug-like Molecules. Nat. Rev. Drug Discov. 2005, 4, 649-663.
- (12) Mauser, H.; Guba, W. Recent Developments in de Novo Design and Scaffold Hopping. Curr. Opin. Drug Discov. Devel. 2008, 11, 365-374.
- (13) Ruddigkeit, L.; van Deursen, R.; Blum, L. C.; Reymond, J.-L. Enumeration of 166 Billion Organic Small Molecules in the Chemical Universe Database GDB-17. J. Chem. Inf. Model. 2012, 52, 2864-2875.
- (14) Polishchuk, P. G.; Madzhidov, T. I.; Varnek, A. Estimation of the Size of Drug-like Chemical Space Based on GDB-17 Data. J. Comput. Aided. Mol. Des. 2013, 27, 675-679.
- (15) Lipinski, C.; Hopkins, A. Navigating Chemical Space for Biology and Medicine. Nature 2004, 432, 855-861.
- (16) Reker, D.; Schneider, G. Active-Learning Strategies in Computer-Assisted Drug Discovery. Drug Discov.
Today 2015, 20, 458-465. - (17) Schneider, P.; Schneider, G. De Novo Design at the Edge of Chaos. J. Med. Chem. 2016, 59, 4077-4086.
- (18) Brown, N.; McKay, B.; Gilardoni, F.; Gasteiger, J. A Graph-Based Genetic Algorithm and Its Application to the Multiobjective Evolution of Median Molecules. J. Chem. Inf. Comput. Sci. 2004, 44, 1079-1087.
- (19) Gómez-Bombarelli, R.; Duvenaud, D.; Hernández-Lobato, J. M.; Aguilera-Iparraguirre, J.; Hirzel, T. D.; Adams, R. P.; Aspuru-Guzik, A. Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules. arXiv ID 1610.02415 2016, 1-23.
- (20) Segler, M. H. S.; Kogej, T.; Tyrchan, C.; Waller, M. P. Generating Focussed Molecule Libraries for Drug Discovery with Recurrent Neural Networks. 2017.
- (21) De Asis, K.; Hernandez-Garcia, J. F.; Holland, G. Z.; Sutton, R. S. Multi-Step Reinforcement Learning: A Unifying Algorithm. 2017.
- (22) Krakovsky, M. Reinforcement Renaissance. Commun. ACM 2016, 59, 12-14.
- (23) Silver, D.; Huang, A.; Maddison, C. J.; Guez, A.; Sifre, L.; van den Driessche, G.; Schrittwieser, J.; Antonoglou, I.; Panneershelvam, V.; Lanctot, M.; et al. Mastering the Game of Go with Deep Neural Networks and Tree Search. Nature 2016, 529, 484-489.
- (24) van den Herik, H. J.; Uiterwijk, J. W. H. M.; van Rijswijck, J. Games Solved: Now and in the Future. Artif. Intell. 2002, 134, 277-311.
- (25) Willia, R. J. Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning. Mach. Learn. 1992, 8, 229-256.
- (26) Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative Adversarial Nets. Adv. Neural Inf. Process. Syst. 27 2014, 2672-2680.
- (27) Yu, L.; Zhang, W.; Wang, J.; Yu, Y. SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient. arXiv 2016.
- (28) Martens, J. Generating Text with Recurrent Neural Networks. Neural Networks 2011, 131, 1017-1024.
- (29) Weininger, D. SMILES, a Chemical Language and Information System. 1. Introduction to Methodology and Encoding Rules. J. Chem. Inf. Comput. Sci. 1988, 28, 31-36.
- (30) Joulin, A.; Mikolov, T. Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets. arXiv 2015, 1-10.
- (31) Hochreiter, S.; Schmidhuber, J. J. Long Short-Term Memory. Neural Comput. 1997, 9, 1-32.
- (32) Chung, J.; Gulcehre, C.; Cho, K.; Bengio, Y. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. arXiv 2014, 1-9.
- (33) Deleu, T.; Dureau, J. Learning Operations on a Stack with Neural Turing Machines. arXiv 2016, 1-6.
- (34) Berstel, J. Transductions and Context-Free Languages; Vieweg+Teubner Verlag: Wiesbaden, 1979.
- (35) Grefenstette, E.; Hermann, K. M.; Suleyman, M.; Blunsom, P. Learning to Transduce with Unbounded Memory.
arXiv 2015, 12. - (36) Mikolov, T.; Chen, K.; Corrado, G.; Dean, J. Distributed Representations of Words and Phrases and Their Compositionality. Nips 2013, 1-9.
- (37) ChemAxon. Marvin Sketch.
- (38) Irwin, J. J.; Shoichet, B. K. ZINC—A Free Database of Commercially Available Compounds for Virtual Screening. J. Chem. Inf. Model. 2005, 45, 177-182.
- (39) Bemis, G. W.; Murcko, M. A. The Properties of Known Drugs. 1. Molecular Frameworks. J. Med. Chem. 1996, 39, 2887-2893.
- (40) Sakatsume, M.; Igarashi, K.; Winestock, K. D.; Garotta, G.; Larner, A. C.; Finbloom, D. S. The Jak Kinases Differentially Associate with the Alpha and Beta (Accessory Factor) Chains of the Interferon Gamma Receptor to Form a Functional Receptor Unit Capable of Activating STAT Transcription Factors. J. Biol. Chem. 1995, 270, 17528-17534.
- (41) Kralovics, R.; Passamonti, F.; Buser, A. S.; Teo, S.-S.; Tiedt, R.; Passweg, J. R.; Tichelli, A.; Cazzola, M.; Skoda, R. C. A Gain-of-Function Mutation of JAK2 in Myeloproliferative Disorders. N. Engl. J. Med. 2005, 352, 1779-1790.
- (42) Stumpfe, D.; Bajorath, J. Similarity Searching. Wiley Interdiscip. Rev. Comput. Mol. Sci. 2011, 1, 260-282.
- (43) Bento, A. P.; Gaulton, A.; Hersey, A.; Bellis, L. J.; Chambers, J.; Davies, M.; Kruger, F. A.; Light, Y.; Mak, L.; McGlinchey, S.; et al. The ChEMBL Bioactivity Database: An Update. Nucleic Acids Res. 2014, 42.
- (44) Van Der Maaten, L. J. P.; Hinton, G. E. Visualizing High-Dimensional Data Using T-Sne. J. Mach. Learn. Res. 2008, 9, 2579-2605.
- (45) Tetko, I. V; Sushko, Y.; Novotarskyi, S.; Patiny, L.; Kondratov, I.; Petrenko, A. E.; Charochkina, L.; Asiri, A. M. How Accurately Can We Predict the Melting Points of Drug-like Compounds? J. Chem. Inf. Model. 2014, 54, 3320-3329.
- (46) Beauman, J. A.; Howard, P. H. Physprop Database. Syracuse Res. Syracuse, N.Y., USA 1995.
- (47) Wang, Y.; Bryant, S. H.; Cheng, T.; Wang, J.; Gindulyte, A.; Shoemaker, B. A.; Thiessen, P. A.; He, S.; Zhang, J. PubChem BioAssay: 2017 Update. Nucleic Acids Res. 2017, 45, D955-D963.
- (48) Fourches, D.; Muratov, E.; Tropsha, A. Trust, but Verify: On the Importance of Chemical Structure Curation in Cheminformatics and QSAR Modeling Research. J
Chem Inf Model 2010, 50, 1189-1204. - It will be understood that various details of the subject matter described herein may be changed without departing from the scope of the subject matter described herein. Furthermore, the foregoing description is for the purpose of illustration only, and not for the purpose of limitation, as the subject matter described herein is defined by the claims as set forth hereinafter.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/632,328 US20200168302A1 (en) | 2017-07-20 | 2018-07-20 | Methods, systems and non-transitory computer readable media for automated design of molecules with desired properties using artificial intelligence |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201762535069P | 2017-07-20 | 2017-07-20 | |
US16/632,328 US20200168302A1 (en) | 2017-07-20 | 2018-07-20 | Methods, systems and non-transitory computer readable media for automated design of molecules with desired properties using artificial intelligence |
PCT/US2018/043114 WO2019018780A1 (en) | 2017-07-20 | 2018-07-20 | Methods, systems and non-transitory computer readable media for automated design of molecules with desired properties using artificial intelligence |
Publications (1)
Publication Number | Publication Date |
---|---|
US20200168302A1 true US20200168302A1 (en) | 2020-05-28 |
Family
ID=65015823
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/632,328 Pending US20200168302A1 (en) | 2017-07-20 | 2018-07-20 | Methods, systems and non-transitory computer readable media for automated design of molecules with desired properties using artificial intelligence |
Country Status (2)
Country | Link |
---|---|
US (1) | US20200168302A1 (en) |
WO (1) | WO2019018780A1 (en) |
Cited By (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111370074A (en) * | 2020-02-27 | 2020-07-03 | 北京晶派科技有限公司 | Method and device for generating molecular sequence and computing equipment |
CN112086144A (en) * | 2020-08-28 | 2020-12-15 | 深圳先进技术研究院 | Molecule generation method, molecule generation device, electronic device, and storage medium |
CN112151127A (en) * | 2020-09-04 | 2020-12-29 | 牛张明 | Unsupervised learning drug virtual screening method and system based on molecular semantic vector |
CN112164428A (en) * | 2020-09-23 | 2021-01-01 | 常州微亿智造科技有限公司 | Method and device for predicting properties of small drug molecules based on deep learning |
CN113223637A (en) * | 2021-05-07 | 2021-08-06 | 中国科学院自动化研究所 | Drug molecule generator training method based on domain knowledge and deep reinforcement learning |
CN113255770A (en) * | 2021-05-26 | 2021-08-13 | 北京百度网讯科技有限公司 | Compound attribute prediction model training method and compound attribute prediction method |
US11132621B2 (en) * | 2017-11-15 | 2021-09-28 | International Business Machines Corporation | Correction of reaction rules databases by active learning |
US20210319098A1 (en) * | 2018-12-31 | 2021-10-14 | Intel Corporation | Securing systems employing artificial intelligence |
CN113707235A (en) * | 2021-08-30 | 2021-11-26 | 平安科技(深圳)有限公司 | Method, device and equipment for predicting properties of small drug molecules based on self-supervision learning |
US20220059192A1 (en) * | 2020-08-18 | 2022-02-24 | International Business Machines Corporation | Running multiple experiments simultaneously on an array of chemical reactors |
CN114937478A (en) * | 2022-05-18 | 2022-08-23 | 北京百度网讯科技有限公司 | Method for training a model, method and apparatus for generating molecules |
WO2023029352A1 (en) * | 2021-08-30 | 2023-03-09 | 平安科技(深圳)有限公司 | Drug small molecule property prediction method and apparatus based on graph neural network, and device |
WO2023077522A1 (en) * | 2021-11-08 | 2023-05-11 | 深圳晶泰科技有限公司 | Compound design method and apparatus, device, and computer readable storage medium |
Families Citing this family (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11651841B2 (en) | 2019-05-15 | 2023-05-16 | International Business Machines Corporation | Drug compound identification for target tissue cells |
US11651860B2 (en) | 2019-05-15 | 2023-05-16 | International Business Machines Corporation | Drug efficacy prediction for treatment of genetic disease |
CN110276403B (en) * | 2019-06-25 | 2022-02-18 | 北京百度网讯科技有限公司 | Model building method and device |
KR20210012730A (en) | 2019-07-26 | 2021-02-03 | 삼성전자주식회사 | Learning method of artificial intelligence model and electronic apparatus |
CN110459275B (en) * | 2019-08-13 | 2022-10-21 | 广东工业大学 | Graph model drug generation method, device and medium based on reinforcement learning |
US20210057050A1 (en) | 2019-08-23 | 2021-02-25 | Insilico Medicine Ip Limited | Workflow for generating compounds with biological activity against a specific biological target |
WO2021044365A1 (en) * | 2019-09-05 | 2021-03-11 | 10736406 Canada Inc. | Method and system for generating synthetically accessible molecules with chemical reaction trajectories using reinforcement learning |
CA3169830A1 (en) * | 2020-01-30 | 2021-08-05 | 99Andbeyond Inc. | System and method for learning to generate chemical compounds with desired properties |
CN113488116B (en) * | 2021-07-09 | 2023-03-10 | 中国海洋大学 | Drug molecule intelligent generation method based on reinforcement learning and docking |
CN113707234B (en) * | 2021-08-27 | 2023-09-05 | 中南大学 | Lead compound patent drug property optimization method based on machine translation model |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100010946A1 (en) * | 2006-08-31 | 2010-01-14 | Silicos Nv | Method for evolving molecules and computer program for implementing the same |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6587845B1 (en) * | 2000-02-15 | 2003-07-01 | Benjamin B. Braunheim | Method and apparatus for identification and optimization of bioactive compounds using a neural network |
US7856321B2 (en) * | 2004-12-16 | 2010-12-21 | Numerate, Inc. | Modeling biological effects of molecules using molecular property models |
US20130252280A1 (en) * | 2012-03-07 | 2013-09-26 | Genformatic, Llc | Method and apparatus for identification of biomolecules |
-
2018
- 2018-07-20 WO PCT/US2018/043114 patent/WO2019018780A1/en active Application Filing
- 2018-07-20 US US16/632,328 patent/US20200168302A1/en active Pending
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100010946A1 (en) * | 2006-08-31 | 2010-01-14 | Silicos Nv | Method for evolving molecules and computer program for implementing the same |
Non-Patent Citations (12)
Title |
---|
Bjerrum, Esben Jannik, and Richard Threlfall. "Molecular generation with recurrent neural networks (RNNs)." arXiv preprint arXiv:1705.04612 (2017). (Year: 2017) * |
Duvenaud, David K., et al. "Convolutional networks on graphs for learning molecular fingerprints." Advances in neural information processing systems 28 (2015). (Year: 2015) * |
Gómez-Bombarelli, Rafael, et al. "Automatic chemical design using a data-driven continuous representation of molecules." arXiv:1610.02415, 2016 (Year: 2016) * |
Guimaraes GL, Sanchez-Lengeling B, Outeiral C, Farias PL, Aspuru-Guzik A. Objective-reinforced generative adversarial networks (organ) for sequence generation models. arXiv preprint arXiv:1705.10843. 2017 May 30. (Year: 2017) * |
Jaques, Natasha, et al. "Sequence tutor: Conservative fine-tuning of sequence generation models with kl-control." International Conference on Machine Learning. PMLR, 2017. (Year: 2017) * |
Jaques, Natasha, et al. "Tuning recurrent neural networks with reinforcement learning." (2017). (Year: 2017) * |
Joulin, Armand, and Tomas Mikolov. "Inferring algorithmic patterns with stack-augmented recurrent nets." Advances in neural information processing systems 28 (2015). (Year: 2015) * |
Lei, Tao, et al. "Deriving neural architectures from sequence and graph kernels." International Conference on Machine Learning. PMLR, 2017. (Year: 2017) * |
Olivecrona, Marcus, et al. "Molecular de-novo design through deep reinforcement learning." Journal of cheminformatics 9.1 (2017): 1-14. (Year: 2017) * |
Segler MH, Kogej T, Tyrchan C, Waller MP. Generating Focussed Molecule Libraries for Drug Discovery with Recurrent Neural Networks. arXiv preprint arXiv:1701.01329. 2017 Jan 5. (Year: 2017) * |
Yu, Lantao, et al. "Seqgan: Sequence generative adversarial nets with policy gradient." Proceedings of the AAAI conference on artificial intelligence. Vol. 31. No. 1. 2017. (Year: 2017) * |
Yuan, William, et al. "Chemical space mimicry for drug discovery." Journal of chemical information and modeling 57.4 (2017): 875-882. (Year: 2017) * |
Cited By (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11132621B2 (en) * | 2017-11-15 | 2021-09-28 | International Business Machines Corporation | Correction of reaction rules databases by active learning |
US20210319098A1 (en) * | 2018-12-31 | 2021-10-14 | Intel Corporation | Securing systems employing artificial intelligence |
CN111370074A (en) * | 2020-02-27 | 2020-07-03 | 北京晶派科技有限公司 | Method and device for generating molecular sequence and computing equipment |
US20220059192A1 (en) * | 2020-08-18 | 2022-02-24 | International Business Machines Corporation | Running multiple experiments simultaneously on an array of chemical reactors |
US11854670B2 (en) * | 2020-08-18 | 2023-12-26 | International Business Machines Corporation | Running multiple experiments simultaneously on an array of chemical reactors |
CN112086144A (en) * | 2020-08-28 | 2020-12-15 | 深圳先进技术研究院 | Molecule generation method, molecule generation device, electronic device, and storage medium |
CN112151127A (en) * | 2020-09-04 | 2020-12-29 | 牛张明 | Unsupervised learning drug virtual screening method and system based on molecular semantic vector |
CN112164428A (en) * | 2020-09-23 | 2021-01-01 | 常州微亿智造科技有限公司 | Method and device for predicting properties of small drug molecules based on deep learning |
CN113223637A (en) * | 2021-05-07 | 2021-08-06 | 中国科学院自动化研究所 | Drug molecule generator training method based on domain knowledge and deep reinforcement learning |
CN113255770A (en) * | 2021-05-26 | 2021-08-13 | 北京百度网讯科技有限公司 | Compound attribute prediction model training method and compound attribute prediction method |
CN113707235A (en) * | 2021-08-30 | 2021-11-26 | 平安科技(深圳)有限公司 | Method, device and equipment for predicting properties of small drug molecules based on self-supervision learning |
WO2023029352A1 (en) * | 2021-08-30 | 2023-03-09 | 平安科技(深圳)有限公司 | Drug small molecule property prediction method and apparatus based on graph neural network, and device |
WO2023077522A1 (en) * | 2021-11-08 | 2023-05-11 | 深圳晶泰科技有限公司 | Compound design method and apparatus, device, and computer readable storage medium |
CN114937478A (en) * | 2022-05-18 | 2022-08-23 | 北京百度网讯科技有限公司 | Method for training a model, method and apparatus for generating molecules |
Also Published As
Publication number | Publication date |
---|---|
WO2019018780A1 (en) | 2019-01-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20200168302A1 (en) | Methods, systems and non-transitory computer readable media for automated design of molecules with desired properties using artificial intelligence | |
Popova et al. | Deep reinforcement learning for de novo drug design | |
Coley et al. | Autonomous discovery in the chemical sciences part I: Progress | |
Yang et al. | Analyzing learned molecular representations for property prediction | |
Heid et al. | Machine learning of reaction properties via learned representations of the condensed graph of reaction | |
Gómez-Bombarelli et al. | Automatic chemical design using a data-driven continuous representation of molecules | |
Chuang et al. | Learning molecular representations for medicinal chemistry: miniperspective | |
Segler et al. | Generating focused molecule libraries for drug discovery with recurrent neural networks | |
Coley et al. | Autonomous discovery in the chemical sciences part II: outlook | |
Schneider et al. | De novo design at the edge of chaos: Miniperspective | |
McComb et al. | Machine learning in pharmacometrics: Opportunities and challenges | |
Galvelis et al. | Neural network and nearest neighbor algorithms for enhancing sampling of molecular dynamics | |
Born et al. | PaccMannRL: De novo generation of hit-like anticancer molecules from transcriptomic data via reinforcement learning | |
Harel et al. | Prototype-based compound discovery using deep generative models | |
Born et al. | Regression transformer enables concurrent sequence regression and generation for molecular language modelling | |
Hogg et al. | Exact hybrid particle/population simulation of rule-based models of biochemical systems | |
Sun et al. | Toward fast and reliable potential energy surfaces for metallic Pt clusters by hierarchical delta neural networks | |
Ramsundar | Molecular machine learning with DeepChem | |
Goel et al. | Molegular: Molecule generation using reinforcement learning with alternating rewards | |
Pillai et al. | Machine learning guided early drug discovery of small molecules | |
Druchok et al. | Ensembling machine learning models to boost molecular affinity prediction | |
Sulaimany et al. | Link prediction potentials for biological networks | |
Tropsha et al. | Integrating QSAR modelling and deep learning in drug discovery: the emergence of deep QSAR | |
Kim et al. | Open macromolecular genome: Generative design of synthetically accessible polymers | |
Stanley et al. | Fake it until you make it? generative de novo design and virtual screening of synthesizable molecules |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: APPLICATION DISPATCHED FROM PREEXAM, NOT YET DOCKETED |
|
AS | Assignment |
Owner name: NATIONAL SCIENCE FOUNDATION, VIRGINIA Free format text: CONFIRMATORY LICENSE;ASSIGNOR:UNIVERSITY OF NORTH CAROLINA, CHAPEL HILL;REEL/FRAME:054645/0494 Effective date: 20200723 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
AS | Assignment |
Owner name: NATIONAL SCIENCE FOUNDATION, VIRGINIA Free format text: CONFIRMATORY LICENSE;ASSIGNOR:UNIVERSITY OF NORTH CAROLINA AT CHAPEL HILL;REEL/FRAME:063601/0252 Effective date: 20200723 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |