US20230154630A1 - Realizing private and practical pharmacological collaboration using a neural network architecture configured for reduced computation overhead - Google Patents
Realizing private and practical pharmacological collaboration using a neural network architecture configured for reduced computation overhead Download PDFInfo
- Publication number
- US20230154630A1 US20230154630A1 US17/948,388 US202217948388A US2023154630A1 US 20230154630 A1 US20230154630 A1 US 20230154630A1 US 202217948388 A US202217948388 A US 202217948388A US 2023154630 A1 US2023154630 A1 US 2023154630A1
- Authority
- US
- United States
- Prior art keywords
- dti
- data
- training
- neural network
- drug
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000000144 pharmacologic effect Effects 0.000 title claims abstract description 9
- 238000013528 artificial neural network Methods 0.000 title claims description 22
- 238000000034 method Methods 0.000 claims abstract description 51
- 230000003993 interaction Effects 0.000 claims abstract description 27
- 239000003596 drug target Substances 0.000 claims abstract description 12
- 238000012549 training Methods 0.000 claims description 29
- 230000006870 function Effects 0.000 claims description 17
- 239000003814 drug Substances 0.000 claims description 12
- 229940079593 drug Drugs 0.000 claims description 12
- 230000004913 activation Effects 0.000 claims description 6
- 239000013598 vector Substances 0.000 claims description 5
- 102000004169 proteins and genes Human genes 0.000 claims description 3
- 108090000623 proteins and genes Proteins 0.000 claims description 3
- 239000000126 substance Substances 0.000 claims description 3
- 238000007876 drug discovery Methods 0.000 claims 1
- 238000003062 neural network model Methods 0.000 abstract description 6
- 241000499489 Castor canadensis Species 0.000 description 10
- 235000011779 Menyanthes trifoliata Nutrition 0.000 description 10
- 238000013459 approach Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 9
- 239000013256 coordination polymer Substances 0.000 description 9
- 239000011159 matrix material Substances 0.000 description 7
- 230000008569 process Effects 0.000 description 6
- 238000011160 research Methods 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 150000001875 compounds Chemical class 0.000 description 4
- 238000004590 computer program Methods 0.000 description 4
- 210000002569 neuron Anatomy 0.000 description 4
- 238000007781 pre-processing Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000012706 support-vector machine Methods 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000013537 high throughput screening Methods 0.000 description 3
- 238000000513 principal component analysis Methods 0.000 description 3
- 238000001994 activation Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000002790 cross-validation Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 210000004027 cell Anatomy 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000002860 competitive effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009792 diffusion process Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000000126 in silico method Methods 0.000 description 1
- 238000012886 linear function Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000000116 mitigating effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000000135 prohibitive effect Effects 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 238000013515 script Methods 0.000 description 1
- 238000001629 sign test Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000013517 stratification Methods 0.000 description 1
- 230000001225 therapeutic effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16C—COMPUTATIONAL CHEMISTRY; CHEMOINFORMATICS; COMPUTATIONAL MATERIALS SCIENCE
- G16C20/00—Chemoinformatics, i.e. ICT specially adapted for the handling of physicochemical or structural data of chemical particles, elements, compounds or mixtures
- G16C20/90—Programming languages; Computing architectures; Database systems; Data warehousing
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H10/00—ICT specially adapted for the handling or processing of patient-related medical or healthcare data
- G16H10/40—ICT specially adapted for the handling or processing of patient-related medical or healthcare data for data related to laboratory analysis, e.g. patient specimen analysis
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/50—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for simulation or modelling of medical disorders
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H70/00—ICT specially adapted for the handling or processing of medical references
- G16H70/40—ICT specially adapted for the handling or processing of medical references relating to drugs, e.g. their side effects or intended usage
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H80/00—ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
- G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16B—BIOINFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR GENETIC OR PROTEIN-RELATED DATA PROCESSING IN COMPUTATIONAL MOLECULAR BIOLOGY
- G16B15/00—ICT specially adapted for analysing two-dimensional or three-dimensional molecular structures, e.g. structural or functional relations or structure alignment
- G16B15/30—Drug targeting using structural data; Docking or binding prediction
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16C—COMPUTATIONAL CHEMISTRY; CHEMOINFORMATICS; COMPUTATIONAL MATERIALS SCIENCE
- G16C20/00—Chemoinformatics, i.e. ICT specially adapted for the handling of physicochemical or structural data of chemical particles, elements, compounds or mixtures
- G16C20/50—Molecular design, e.g. of drugs
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16C—COMPUTATIONAL CHEMISTRY; CHEMOINFORMATICS; COMPUTATIONAL MATERIALS SCIENCE
- G16C20/00—Chemoinformatics, i.e. ICT specially adapted for the handling of physicochemical or structural data of chemical particles, elements, compounds or mixtures
- G16C20/70—Machine learning, data mining or chemometrics
Definitions
- This application relates generally to data sharing and collaboration in biomedicine.
- DTIs drug-target interactions
- HTS high-throughput screening
- This disclosure provides for a secure and scalable cryptographic pipeline for pharmacological collaboration where multiple entities securely combine their data to jointly predict drug-target interactions with state-of-the-art accuracy.
- the subject matter herein is a computational protocol for drug-target interaction (DTI) prediction that is privacy-preserving, scales to millions of interactions, and achieves state-of-the-art prediction accuracy.
- cryptographic tools are used to enable DTI data sharing that provably ensures the confidentiality of the underlying drugs, targets, and observed interactions.
- this joint dataset is used to securely train a neural network model with predictive capabilities beyond that of any individual entity, without disclosing the original data to any of the involved parties.
- the end-to-end protocol identifies novel DTIs with strong clinical or experimental support and can train within days over a wide area network on a dataset with millions of observed interactions.
- the secure DTI prediction protocol lays the foundation for a transformational paradigm that enables more effective and collaborative biomedical research.
- FIG. 1 is a multi-computing entity implementation environment in which the techniques of this disclosed may be practiced
- FIGS. 2 A, 2 B and 2 C are graphs depicting how the secure neural network trained as described herein accurately predicts drug-target interactions within practical and scalable runtimes.
- FIG. 1 depicts the basic framework.
- the protocol divides computation across collaborating entities while ensuring that none of the entities has any knowledge about the private data.
- secret a private value
- shares a random number
- any strict subset of entities cannot extract any information about the underlying secret using their shares.
- the input data preferably is represented as a list of observed DTIs, where each instance typically includes interaction score and feature vectors representing the drug and target side information, preferably all of which is then secretly-shared.
- a predictive model (which takes the feature vectors as input and outputs a predicted interaction score) is then trained, preferably by making linear passes over the DTIs, using each data instance to securely update the model parameters.
- This approach scales linearly with the number of observed DTIs, which is typically much smaller than the number of possible DTIs.
- the described approach results in a predictive model that naturally generalizes to previously unseen drugs or targets, unlike matrix factorization.
- a neural network model is utilized due to its effectiveness in capturing complex patterns in the data.
- a na ⁇ ve implementation of neural network training still incurs a high cryptographic overhead in MPC.
- a preferred architectural design is provided that specifically takes advantage of MPC operations that benefit from a reduced amount of computation.
- Computing neuron activations using a highly non-linear function like sigmoid or tanh is difficult to efficiently approximate in MPC; instead, the approach herein preferably uses a rectified linear unit (ReLU), because evaluating the ReLU and its derivative requires just a single data-oblivious comparison.
- ReLU rectified linear unit
- training scores are evaluated with hinge loss, which requires a single data-oblivious comparison.
- training is accelerated using a mini-batch stochastic gradient descent, e.g., with Nesterov momentum.
- a mini-batch stochastic gradient descent e.g., with Nesterov momentum.
- MPC protocols as described herein readily generalize to other machine learning models, e.g., a secure support vector machine.
- multiple participating entities contribute secret-shared data to train a neural network on a joint DTI dataset, such as depicted in FIG. 1 .
- the model is made available to one or more (or all) participants, or the model could remain private such that entities receive a number of predictions commensurate with the amount of data they contribute, thereby further incentivizing participation.
- the pipeline is secure under an “honest-but-curious” security model in which the collaborating entities follow the correct protocol and do not collude to reconstruct the data.
- This approach is a substantial improvement over the current state of biomedical research where privacy concerns hinder any collaboration, and the framework can be extended to achieve even stronger security guarantees.
- the security guarantee holds as long as at least one entity is honest during the main computation
- the no-collusion requirement can be related by introducing additional entities into the protocol.
- This alternative embodiment does not substantially increase total computation time but does increase communication linearly in the number of entities.
- MAC message authentication code
- differential privacy methods currently being developed for differentially private neural networks
- An alternative strategy for collaborative deep learning is to train local models in plaintext and to use secure protocols only when periodically averaging over these models, thus minimizing the amount of cryptographic overhead.
- the latter approach is vulnerable to reverse engineering-based attacks in which a malicious collaborator jointly trains a local model (e.g., a generative adversarial network) that uncovers information about private data owned by honest collaborators, even when differential privacy techniques are applied.
- a local model e.g., a generative adversarial network
- securely training a single model over a decentralized network of computing parties is not vulnerable to such attacks.
- the privacy-preserving protocols described herein generalize to other large-scale data sharing problems beyond DTI prediction, with the highest potential for impact in areas that suffer from a lack of collaboration due to privacy concerns, such as predictive analyses of electronic health records.
- the secure DTI protocol as described herein enables greater scientific collaboration that realizes immense biological, commercial, and clinical benefit.
- collaborating entities e.g., pharmaceutical companies, research laboratories, and the like
- DTIs drug-target interactions
- the entities first use secret sharing 104 to pool their data 105 in a way that reveals no information about the underlying drugs, targets, or interactions (step 1 ).
- the collaborating entities then jointly execute a cryptographic protocol 106 that trains a predictive model 108 (e.g., a neural network) on the pooled dataset (step 2 ).
- the final model can be made available to participating entities, or be used to distribute novel DTI predictions 110 to participants in a way that encourages greater data sharing (step 3 ).
- FIG. 2 A is a graph that depicts the secure neural network (called “Secure NN”) outperforming previous methods for DTI prediction on the DrugBank 3.0 dataset using 10-fold cross validation (CV) on balanced training and test sets.
- bar height corresponds to mean AUPR (area under the precision-recall curve) and error bars correspond to the standard deviation.
- the methods compared include BLMNII, NetLapRLS, HNM, CMF, and DTINet.
- FIG. 2 B is a graph that depicts that on a large-scale benchmark dataset with 1,357,742 drug-target pairs (the STITCH dataset), the Secure NN produces high quality predictions even when the test set consisted only of previously unseen chemicals and also outperforms plaintext CMF—the only baseline method which could be feasibly run on a dataset of this size.
- FIG. 2 C is a graph that depicts the runtime of the training protocol, over a local area network (LAN), for different dataset sizes and observe a linear dependence, as expected. Box height represents the standard deviation. Even training on two million interactions, the total runtime for one epoch (one linear pass over the full, shuffled training set) is around 2.2 days. In practice, the model achieves high accuracy in only a few training epochs, e.g., after 1.5 epochs.
- the Secure NN is trained on all human drug-target interactions from the STITCH 5 database.
- the neural network is then used to score and rank the drug-target pairs that make up a remaining interaction space. Out of the top 20 interactions, excluding pairs with overrepresented drugs and targets, almost all have strong clinical or experimental support for interactivity, and some top-ranked pairs are considered to represent novel interactions.
- CP 1 and CP 2 are collaborating entities
- the protocol also involves a third auxiliary entity CP 0 that is involved, but preferably only during an offline pre-computation phase. Entity CP 0 cannot collude with other entities in the protocol (for the security guarantee to hold), thus CP 0 is a trusted party.
- secret sharing a cryptographic technique called secret sharing, each of CP 1 and CP 2 shares its DTI data (i.e., drug- and target-specific input features and observed interaction scores) with the other participant in such a way that enables privacy-preserving computation over the pooled data.
- CP 1 and CP 2 preferably leverage pre-computed data from CP 0 (which is input-agnostic) to greatly speed up the computation.
- CP 0 which is input-agnostic
- a preferred technique to this end is generalized Beaver partitioning.
- CP 1 and CP 2 then combine their outputs to reconstruct the final results (e.g., neural network weights or predicted DTIs).
- the approach herein adopts the “honest-but-curious” security model in which the protocol participants are assumed to follow the protocol exactly as specified, but at the end of the protocol execution, a party may try to infer additional information about other parties' private inputs based on their view of the protocol.
- the protocol herein is secure as long as CP 0 and at least one of the other CPs remains honest.
- all communication occurs over a secure and authenticated channel (e.g., over the TLS protocol).
- Adding two secret shared values [x] and [y] can be done by having each party add their own shares, i.e., [x] 1 +[y] 1 , [x] 2 +[y] 2 .
- Adding by a public field element a ⁇ q can be written as [x] 1 +a, [x] 2 .
- Multiplying by a public field element is also simple and can be written as a[x] 1 , a[x] 2 .
- Multiplying two secret shared values is more involved but preferably leverages Beaver multiplication triples for efficiently computing many multiplications, including matrix multiplications.
- the protocol uses a fixed-point representation of signed real numbers that uses k total bits, of which f is the number of bits allocated to the fractional domain, referred to as the “precision.”
- a secret shared fixed-point encoding of x ⁇ as [x] (f) .
- Multiplication of two fixed point numbers outputs a result with precision of 2f instead of f, so preferably a truncation routine is used to rescale the precision, which is denoted
- the protocol uses a data oblivious sign test (i.e., a comparison with zero) that takes the form
- a preferred neural network model as used herein is now defined.
- the technique assumes a feature matrix X ⁇ , where each row corresponds to a single training example and each column corresponds to a single data feature.
- the neural network model is a standard multilayer perceptron, consisting of real-valued weight matrices W (1) , . . . , W (L+1) and column vector biases b (1) , . . .
- each hidden layer has the same number of neurons, denoted H, where Z (l) ⁇ .
- the predictive performance of the model is evaluated using the hinge loss function
- model weights and biases are updated, e.g., using Nesterov momentum updates
- ⁇ and ⁇ which are the momentum and learning rates, respectively.
- Nesterov momentum is used instead of standard momentum because it has better convergence guarantees for convex functions.
- X batch and y batch are sampled randomly without replacement until all training examples have been considered, after which all training data is restored, and the process repeated.
- these randomly sampled mini-batches are used to iteratively compute the unbiased estimate of the gradient and update the model parameters, a procedure referred to as stochastic gradient descent (SGD).
- SGD stochastic gradient descent
- a GradientDescent protocol is implemented, which protocol takes as input [X batch ] (f) , [y batch ] (f) ,
- [ W ] (f) ⁇ [ W (1) ] (f) , . . . , [ W (L+1) ] (f) , [ W v (1) ] (f) , . . . , [ W v (L+1) ] (f) ⁇ , and
- [ b ] (f) ⁇ [ b (1) ] (f) , . . . , [ b (L+1) ] (f) , [ b v (1) ] (f) , . . . , [ b v (L+1) ] (f) ⁇ ,
- FIG. 3 together with FIGS. 4 A and 4 B , depict a preferred GradientDescent protocol in detail.
- This protocol preferably is implemented in software executing in hardware.
- multiple rounds of GradientDescent are invoked in which a new X batch and corresponding y batch are randomly sampled from the full dataset, which is repeated until reaching the max number of iterations T.
- the two collaborating entities CP 1 and CP 2 preferably combine their shares to jointly reconstruct W (1) , . . . , W (L+1) and b (1) , . . . , b (L+1) in plaintext.
- Hyperparameters preferably are chosen based on a grid search (in plaintext) of around 10 different hyperparameter combinations. The model performance is robust to a wide range of hyperparameter settings.
- the tuning phase can be sped up with parallelism and other coordinated tuning strategies.
- a linear-kernel Support Vector Machine may be used as an alternative to a neural network.
- the MPC protocol for the secure neural network and SVM may be implemented in C++ based on the number theory package NTL version 10.3.0 for finite field operations.
- Other data processing scripts are implemented in Python and the NumPy package.
- Representative computing systems include CP 1 : 2.40 GHz Intel Xeon E5-2695v2 CPU with 384 GB RAM, CP 2 : 2.30 GHz Intel Xeon E5-2650v3 CPU with 384 GB RAM; and CP 0 : 3.33 GHz Intel Xeon X5680 CPU with 96 GB RAM.
- cloud computing is a model of service delivery for enabling on-demand network access to a shared pool of configurable computing resources (e.g. networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a provider of the service.
- configurable computing resources e.g. networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services
- SaaS Software as a Service
- PaaS Platform as a service
- IaaS Infrastructure as a Service
- the platform may comprise co-located hardware and software resources, or resources that are physically, logically, virtually and/or geographically distinct.
- Communication networks used to communicate to and from the platform services may be packet-based, non-packet based, and secure or non-secure, or some combination thereof.
- a representative machine on which the software executes comprises commodity hardware, an operating system, an application runtime environment, and a set of applications or processes and associated data, that provide the functionality of a given system or subsystem.
- the functionality may be implemented in a standalone machine, or across a distributed set of machines.
- a computing entity herein receives the secretly-shared data from a client device, which may even be an end user device.
- a client device is a mobile device, such as a smartphone, tablet, or wearable computing device.
- a client device comprises a CPU (central processing unit), computer memory, such as RAM, and a drive.
- the device software includes an operating system (e.g., Google® AndroidTM, or the like), and generic support applications and utilities.
- the underlying network transport may be any communication medium including, without limitation, packet-based, cellular, wireless, Wi-Fi, small cell, and combinations thereof.
- Each above-described process preferably is implemented in computer software as a set of computer program instructions executable in one or more processors, as a special-purpose machine.
- Representative machines on which the subject matter herein is provided may be hardware processor-based computers running an operating system and one or more applications to carry out the described functionality.
- This apparatus may be a particular machine that is specially constructed for the required purposes, or it may comprise a computer otherwise selectively activated or reconfigured by a computer program stored in the computer.
- a computer program may be stored in a computer readable storage medium, such as, but is not limited to, any type of disk including an optical disk, a CD-ROM, and a magnetic-optical disk, a read-only memory (ROM), a random access memory (RAM), a magnetic or optical card, or any type of media suitable for storing electronic instructions, and each coupled to a computer system bus.
- a given implementation of the computing platform is software that executes on a hardware platform running an operating system, such as Linux.
- a machine implementing the techniques herein comprises a hardware processor, and non-transitory computer memory holding computer program instructions that are executed by the processor to perform the above-described methods.
- Communications herein e.g., secret sharing of data, sharing of PRGs, etc.
- Communications herein preferably take place over secure connections.
- information typically is communicated over SSL/TLS links, or using any other protocol, although if significant trust is in place (e.g., machines being operated in a secure environment, or between entities that have a trust relationship, etc.) information may be transmitted in the clear.
- computations herein may be carried out within a cluster comprises a set of computing entities (e.g., processors).
- a grid computing architecture may also be utilized.
- Any computing entity may act as a client or a server.
- the functionality may be co-located or various parts/components may be separately and run as distinct functions, perhaps in one or more locations (over a distributed network).
- the entity that performs pre-processing (CP 0 ) may also be a computing entity (e.g., CP 1 ) in the MPC computation to generate the DIT output.
- the pre-processing may be provided as a service.
- the secure computation may be provided as a service.
- Computing entities herein may be independent from one another, or associated with one another. Multiple computing entities may be associated with a single enterprise entity, but are separate and distinct from one another with respect to the MPC secure computation itself over their respective secret shares.
- the protocol may leverage several advanced computational techniques, which are now generally described. Details of these techniques are provided in U.S. Ser. No. 16/020,058, filed Jun. 27, 2018, the disclosure of which is hereby incorporated by reference.
- One technique improves upon an existing building block for secure computation, known as Beaver triples. Beaver triples were originally developed for carrying out secure pairwise multiplications; they are extended to arbitrary arithmetic circuits to thereby obtain more efficient subroutines for various operations such as matrix multiplication and exponentiation.
- Another technique involves using a random projection-based algorithm for Principal Component Analysis (PCA) so that the scale of the data is greatly reduced.
- PCA Principal Component Analysis
- PRGs pseudorandom number generators
- the approach generalizes Beaver triples so that auxiliary random numbers, which enable the secure evaluation of nonlinear functions, are generated for groups of operations at a time, rather than for each individual multiplication.
- the generalized Beaver tuple preferably includes a random number for each input value, and some function of these random numbers for each of the output values of a desired computation.
- a third party CP 0
- the third-party preferably does not observe the input data in any way and is able to finish its task before the main protocol, as a pre-processing step.
- the technical advances in achieving a scalable MPC protocol for realizing private and practical pharmacological collaboration as described herein may be extended for use with diverse applications in biomedicine and other disciplines.
- the computational approach herein allows pharmaceutical companies and academic researchers (and other interested parties) to pool their data for more accurate predictions while keeping the raw data private. Further, the methods for secure computation removes obstacles for data sharing and, as a result, enable new workflows and opportunities for improved scientific collaboration.
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Public Health (AREA)
- Epidemiology (AREA)
- Primary Health Care (AREA)
- Physics & Mathematics (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Pathology (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Databases & Information Systems (AREA)
- Chemical & Material Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Crystallography & Structural Chemistry (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medicinal Chemistry (AREA)
- Pharmacology & Pharmacy (AREA)
- Toxicology (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Storage Device Security (AREA)
- Medical Treatment And Welfare Office Work (AREA)
Abstract
Computationally-efficient techniques facilitate secure pharmacological collaboration with respect to private drug target interaction (DTI) data. In one embodiment, a method begins by receiving, via a secret sharing protocol, observed DTI data from individual participating entities. A secure computation then is executed against the secretly-shared data to generate a pooled DTI dataset. For increased computational efficiency, at least a part of the computation is executed over dimensionality-reduced data. The resulting pooled DTI dataset is then used to train a neural network model. The model is then used to provide one or more DTI predictions that are then returned to the participating entities (or other interested parties).
Description
- This application relates generally to data sharing and collaboration in biomedicine.
- Pharmaceutical companies and other biomedical researchers are painstakingly generating large datasets of drug compounds and linking them to potential targets of interest. While combining an unprecedented amount of data from multiple entities would power innovation and life-saving breakthroughs, open sharing of pharmacological data is generally not viable due to data privacy and intellectual property concerns.
- In particular, computational prediction of drug-target interactions (DTIs) allows biomedical researchers to prioritize downstream experiments and accelerate pharmaceutical research. Recent advances in high-throughput screening (HTS) technologies have given rise to unprecedented amounts of DTI data which offer new opportunities for in silico DTI prediction. However, the space of chemical compounds with potential therapeutic effects and the space of potential targets in the post-genomic era are far too expansive to be experimentally interrogated in an exhaustive manner, even with HTS. In practice, individual laboratories observe only a small fraction of the landscape of possible DTIs. As pharmacological research pipelines face declining productivity and increasing calls for greater collaboration, jointly leveraging such data from across the pharmaceutical industry and academia could enable more accurate predictive models of DTIs and spur life-saving biomedical discoveries. However, pharmacological data is often tightly coupled with financial interests and therefore rarely shared. In particular, researchers may wish to maintain the confidentiality of new compounds under development or even the set of potential targets being tested, both of which may also contain sensitive information about underlying experimental strategies.
- This disclosure provides for a secure and scalable cryptographic pipeline for pharmacological collaboration where multiple entities securely combine their data to jointly predict drug-target interactions with state-of-the-art accuracy. To this end, the subject matter herein is a computational protocol for drug-target interaction (DTI) prediction that is privacy-preserving, scales to millions of interactions, and achieves state-of-the-art prediction accuracy. Preferably, cryptographic tools are used to enable DTI data sharing that provably ensures the confidentiality of the underlying drugs, targets, and observed interactions. Preferably, this joint dataset is used to securely train a neural network model with predictive capabilities beyond that of any individual entity, without disclosing the original data to any of the involved parties. The end-to-end protocol identifies novel DTIs with strong clinical or experimental support and can train within days over a wide area network on a dataset with millions of observed interactions. The secure DTI prediction protocol lays the foundation for a transformational paradigm that enables more effective and collaborative biomedical research.
- The foregoing has outlined some of the more pertinent features of the subject matter. These features should be construed to be merely illustrative. Many other beneficial results can be attained by applying the disclosed subject matter in a different manner or by modifying the subject matter as will be described.
- For a more complete understanding of the subject matter and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:
-
FIG. 1 is a multi-computing entity implementation environment in which the techniques of this disclosed may be practiced; -
FIGS. 2A, 2B and 2C are graphs depicting how the secure neural network trained as described herein accurately predicts drug-target interactions within practical and scalable runtimes; and -
FIG. 3 , andFIGS. 4A and 4B together depict a preferred Gradient Descent algorithm for training a neural network according to an embodiment of this disclosure. - Modern cryptography offers techniques to encourage pharmacological collaboration by greatly mitigating privacy concerns associated with data sharing. For instance, secure multiparty computation (MPC) protocols allow multiple entities to securely compute over their private datasets without revealing any information about the underlying raw data, except for the final computational output. Unfortunately, the promise of privacy-preserving collaboration has been severely hindered by the inability of existing secure computation frameworks to scale to complex computations over large datasets, and in particular to DTI prediction on datasets with millions of interactions. This disclosure addresses this problem.
- In particular, this disclosure provides for an end-to-end pipeline for collaborative DTI prediction based on secure MPC that newly enables privacy-preserving collaboration within practical runtimes even for million-interaction datasets.
FIG. 1 depicts the basic framework. Conceptually, the protocol divides computation across collaborating entities while ensuring that none of the entities has any knowledge about the private data. This is achieved using a cryptographic framework known as secret sharing in which a private value (“secret”) is collectively represented by multiple entities, where preferably each entity is given a random number (“share”) in a finite field (i.e., integers modulo some prime number p) such that the sum of all shares modulo p equals the secret. Importantly, any strict subset of entities cannot extract any information about the underlying secret using their shares. Various protocols have been developed for performing elementary operations (e.g., addition, multiplication) over secret-shared inputs, which taken together form the building blocks for a general purpose MPC framework that performs arbitrary computation over secret-shared data without leaking any information about the private input. - While secret sharing-based MPC typically requires overwhelming amounts of data communication between entities for complex and large-scale computations, very recent optimizations have leveraged techniques, such as generalized Beaver triples and shared pseudorandom number generators (PRGs), to significantly reduce communication cost, enabling practical secure computation for challenging problems such as genome-wide association studies for a million individuals. Even with these advances, however, secure MPC may not be feasible for most DTI prediction methods that use matrix factorization or network diffusion to generalize the topology of the DTI network. This is primarily because these prediction methods scale quadratically with the number of drugs (n) and the number of targets (m) in the dataset (e.g., n2 or nm), which is prohibitive for realistic datasets with millions of compounds. Although a recently proposed method for privacy-preserving matrix factorization obtains linear complexity in the number of observed interactions, it does not admit additional features known as “side information,” which is important achieving competitive prediction performance. Moreover, without a sophisticated consideration of side information, matrix factorization has difficulty generalizing to drugs or proteins not in its training set, referred to as the “cold start problem.”
- To achieve scalable computation while admitting side information, according to this disclosure the input data preferably is represented as a list of observed DTIs, where each instance typically includes interaction score and feature vectors representing the drug and target side information, preferably all of which is then secretly-shared. A predictive model (which takes the feature vectors as input and outputs a predicted interaction score) is then trained, preferably by making linear passes over the DTIs, using each data instance to securely update the model parameters. This approach scales linearly with the number of observed DTIs, which is typically much smaller than the number of possible DTIs. Furthermore, the described approach results in a predictive model that naturally generalizes to previously unseen drugs or targets, unlike matrix factorization. In this protocol, preferably a neural network model is utilized due to its effectiveness in capturing complex patterns in the data.
- Despite linear-time asymptotics, a naïve implementation of neural network training still incurs a high cryptographic overhead in MPC. To achieve practical runtimes, and according to this disclosure, a preferred architectural design is provided that specifically takes advantage of MPC operations that benefit from a reduced amount of computation. Computing neuron activations using a highly non-linear function like sigmoid or tanh is difficult to efficiently approximate in MPC; instead, the approach herein preferably uses a rectified linear unit (ReLU), because evaluating the ReLU and its derivative requires just a single data-oblivious comparison. Similarly, instead of less-efficient but more common alternatives such as cross entropy loss, preferably training scores are evaluated with hinge loss, which requires a single data-oblivious comparison. Preferably, and as will be described, training is accelerated using a mini-batch stochastic gradient descent, e.g., with Nesterov momentum. These techniques allow the secure neural network to train, preferably over a wide area network (WAN) in a relatively short time period (e.g., measured in hours or days), even with respect to a large dataset (e.g., with more than a million training instances). The MPC protocols as described herein readily generalize to other machine learning models, e.g., a secure support vector machine.
- In one embodiment, multiple participating entities contribute secret-shared data to train a neural network on a joint DTI dataset, such as depicted in
FIG. 1 . After training, the model is made available to one or more (or all) participants, or the model could remain private such that entities receive a number of predictions commensurate with the amount of data they contribute, thereby further incentivizing participation. The more training data used, the better the performance, thereby entities are incentivized to share information in a way that is mutually beneficial but that still maintains privacy guarantees. - The pipeline is secure under an “honest-but-curious” security model in which the collaborating entities follow the correct protocol and do not collude to reconstruct the data. This approach is a substantial improvement over the current state of biomedical research where privacy concerns hinder any collaboration, and the framework can be extended to achieve even stronger security guarantees. In particular, because (as described below) the security guarantee holds as long as at least one entity is honest during the main computation, the no-collusion requirement can be related by introducing additional entities into the protocol. This alternative embodiment does not substantially increase total computation time but does increase communication linearly in the number of entities. Further, if additional security against malicious entities who deviate from the protocol during the online computation is required, a message authentication code (MAC) is included with each message, where at the end of the protocol the MAC is verified to ensure that each step was performed according to the protocol specification, a known technique. This alternative approach roughly doubles computation and communication, offering a tradeoff between security and performance that can be adjusted according to specific study requirements.
- While the pipeline does not consider adding noise to the final computation output to limit information leakage, a technique known as differential privacy (methods currently being developed for differentially private neural networks) can be used in conjunction with the protocol. An alternative strategy for collaborative deep learning is to train local models in plaintext and to use secure protocols only when periodically averaging over these models, thus minimizing the amount of cryptographic overhead. The latter approach, however, is vulnerable to reverse engineering-based attacks in which a malicious collaborator jointly trains a local model (e.g., a generative adversarial network) that uncovers information about private data owned by honest collaborators, even when differential privacy techniques are applied. In contrast, securely training a single model over a decentralized network of computing parties, as in the preferred embodiment of the below-described pipeline, is not vulnerable to such attacks.
- The privacy-preserving protocols described herein generalize to other large-scale data sharing problems beyond DTI prediction, with the highest potential for impact in areas that suffer from a lack of collaboration due to privacy concerns, such as predictive analyses of electronic health records. The secure DTI protocol as described herein enables greater scientific collaboration that realizes immense biological, commercial, and clinical benefit.
- As depicted in
FIG. 1 , collaborating entities (e.g., pharmaceutical companies, research laboratories, and the like) 100 have largeprivate datasets 102 of drug-target interactions (DTIs), as well as corresponding chemical structures and protein sequences. In one embodiment of the protocol, the entities first usesecret sharing 104 to pool theirdata 105 in a way that reveals no information about the underlying drugs, targets, or interactions (step 1). The collaborating entities then jointly execute acryptographic protocol 106 that trains a predictive model 108 (e.g., a neural network) on the pooled dataset (step 2). The final model can be made available to participating entities, or be used to distributenovel DTI predictions 110 to participants in a way that encourages greater data sharing (step 3). - As depicted in -
FIGS. 2A, 2B and 2C , a secure neural network trained as described herein accurately predicts drug-target interactions within practical and scalable runtimes. In particular,FIG. 2A is a graph that depicts the secure neural network (called “Secure NN”) outperforming previous methods for DTI prediction on the DrugBank 3.0 dataset using 10-fold cross validation (CV) on balanced training and test sets. In this graph, bar height corresponds to mean AUPR (area under the precision-recall curve) and error bars correspond to the standard deviation. The methods compared include BLMNII, NetLapRLS, HNM, CMF, and DTINet.FIG. 2B is a graph that depicts that on a large-scale benchmark dataset with 1,357,742 drug-target pairs (the STITCH dataset), the Secure NN produces high quality predictions even when the test set consisted only of previously unseen chemicals and also outperforms plaintext CMF—the only baseline method which could be feasibly run on a dataset of this size.FIG. 2C is a graph that depicts the runtime of the training protocol, over a local area network (LAN), for different dataset sizes and observe a linear dependence, as expected. Box height represents the standard deviation. Even training on two million interactions, the total runtime for one epoch (one linear pass over the full, shuffled training set) is around 2.2 days. In practice, the model achieves high accuracy in only a few training epochs, e.g., after 1.5 epochs. - In one embodiment, the Secure NN is trained on all human drug-target interactions from the STITCH 5 database. The neural network is then used to score and rank the drug-target pairs that make up a remaining interaction space. Out of the top 20 interactions, excluding pairs with overrepresented drugs and targets, almost all have strong clinical or experimental support for interactivity, and some top-ranked pairs are considered to represent novel interactions.
- For this description, consider the simplest setting with two collaborating entities (e.g., academic labs or pharmaceutical companies), denoted CP1 and CP2 (“computing parties”). Typically, the protocol also involves a third auxiliary entity CP0 that is involved, but preferably only during an offline pre-computation phase. Entity CP0 cannot collude with other entities in the protocol (for the security guarantee to hold), thus CP0 is a trusted party. Using a cryptographic technique called secret sharing, each of CP1 and CP2 shares its DTI data (i.e., drug- and target-specific input features and observed interaction scores) with the other participant in such a way that enables privacy-preserving computation over the pooled data. During this computation, CP1 and CP2 preferably leverage pre-computed data from CP0 (which is input-agnostic) to greatly speed up the computation. A preferred technique to this end is generalized Beaver partitioning. CP1 and CP2 then combine their outputs to reconstruct the final results (e.g., neural network weights or predicted DTIs).
- As noted above, the approach herein adopts the “honest-but-curious” security model in which the protocol participants are assumed to follow the protocol exactly as specified, but at the end of the protocol execution, a party may try to infer additional information about other parties' private inputs based on their view of the protocol. Under this setting, the protocol herein is secure as long as CP0 and at least one of the other CPs remains honest. In a preferred embodiment, all communication occurs over a secure and authenticated channel (e.g., over the TLS protocol).
- Preferably, the protocol relies on a two-party additive secret sharing scheme where a value x ∈ q is shared between CP1 and CP2, where a secret sharing of x between CP1 and CP2 is denoted as [x] [x]1, [x]2 , where the notation [x]1, [x]2 means that [x]1 and [x]2 are shares of x in q individually owned by CP1 and CP2, respectively, such that x=[x]1+[x]2. Adding two secret shared values [x] and [y] can be done by having each party add their own shares, i.e., [x]1+[y]1, [x]2+[y]2 . Adding by a public field element a ∈ q can be written as [x]1+a, [x]2 . Multiplying by a public field element is also simple and can be written as a[x]1, a[x]2 . Multiplying two secret shared values is more involved but preferably leverages Beaver multiplication triples for efficiently computing many multiplications, including matrix multiplications. Preferably, the protocol uses a fixed-point representation of signed real numbers that uses k total bits, of which f is the number of bits allocated to the fractional domain, referred to as the “precision.” We denote a secret shared fixed-point encoding of x ∈ as [x](f). Multiplication of two fixed point numbers outputs a result with precision of 2f instead of f, so preferably a truncation routine is used to rescale the precision, which is denoted
-
[xtrunc]←Truncate([x], b, s) - where b is the number of bits to mask, which is chosen such that a sufficient level of statistical security is guaranteed, and s is the number of least significant bits to truncate. Preferably, the protocol uses a data oblivious sign test (i.e., a comparison with zero) that takes the form
- A preferred neural network model as used herein is now defined. The technique assumes a feature matrix X ∈, where each row corresponds to a single training example and each column corresponds to a single data feature. The technique also assumes a label vector y ∈ {−1, +1}M where yi=+1 if X:,i is a positive training example and yi=−1 otherwise. While binary labels are assumed, the framework generalizes to continuous interaction scores. Preferably, the neural network model is a standard multilayer perceptron, consisting of real-valued weight matrices W(1), . . . , W(L+1) and column vector biases b(1), . . . , b(L+1), where L is the number of hidden layers and where each hidden layer consists of neurons Z(1), . . . , Z(L). During a forward-propagation phase, certain neurons are “activated” according to
-
Z :,i (1) =f act(W (1) X :,i +b (1)) and -
Z :,i (l) =f act(W (l) Z :,i (l−1) +b (l−1)) - for l=2, . . . , L and i=1, . . . , M. For present purposes, it is assumed that each hidden layer has the same number of neurons, denoted H, where Z(l) ∈. The function fact is known as an activation function, which in the neural network is the rectified linear unit (ReLU), which takes the form fact(x)=max{0,x}. After the final hidden layer, the model outputs scores s where
-
s i =W (L+1) Z :,i (L) +b (L+1) -
- where ⊙ is the component-wise (or Hadamard) product. Next, these errors are used to compute derivative updates to the weights and biases, starting with the output layer, where
-
- Note that a regularization term is added to the weight updates parameterized by the constant λ.
- Following a standard backpropagation algorithm for training neural networks, these derivatives are recursively propagated through each hidden layer using
-
- for hidden layers l=2, . . . , L and for the input layer l=1,
-
- Finally, the model weights and biases are updated, e.g., using Nesterov momentum updates
-
- for parameters Θ ∈{W(1), . . . , W(L+1), b(1), . . . , b(L+1)}, successive time steps t=1, . . . , T, and constants μ and α which are the momentum and learning rates, respectively. Preferably, Nesterov momentum is used instead of standard momentum because it has better convergence guarantees for convex functions.
- In practice, it is not required to use all training examples for each parameter update iteration; rather, it is sufficient to use a random subset referred to as a “mini-batch,” which is denoted Xbatch ∈ and the corresponding labels ybatch ∈{−1, +1}M
batch . In one embodiment of the protocol, Xbatch and ybatch are sampled randomly without replacement until all training examples have been considered, after which all training data is restored, and the process repeated. Preferably, these randomly sampled mini-batches are used to iteratively compute the unbiased estimate of the gradient and update the model parameters, a procedure referred to as stochastic gradient descent (SGD). - In particular, the following defines a secure protocol for neural network training. First, a GradientDescent protocol is implemented, which protocol takes as input [Xbatch](f), [ybatch](f),
-
[W](f)={[W (1)](f), . . . , [W (L+1)](f), [W v (1)](f), . . . , [W v (L+1)](f)}, and -
[b](f)={[b (1)](f), . . . , [b (L+1)](f), [b v (1)](f), . . . , [b v (L+1)](f)}, - and then outputs the updated model parameters [W](f) and [b](f) after performing a single stochastic gradient update.
-
FIG. 3 , together withFIGS. 4A and 4B , depict a preferred GradientDescent protocol in detail. This protocol preferably is implemented in software executing in hardware. Preferably, multiple rounds of GradientDescent are invoked in which a new Xbatch and corresponding ybatch are randomly sampled from the full dataset, which is repeated until reaching the max number of iterations T. Finally, after the training procedure is finished, the two collaborating entities CP1 and CP2 preferably combine their shares to jointly reconstruct W(1), . . . , W(L+1) and b(1), . . . , b(L+1) in plaintext. - In a representative embodiment, the neural network model (e.g., for the STITCH dataset experiments) has hyperparameters N=6903, Mbatch=100, L=2, H=50, T=20,000, λ=0.001, μ=0.9 and α=0.005. Hyperparameters preferably are chosen based on a grid search (in plaintext) of around 10 different hyperparameter combinations. The model performance is robust to a wide range of hyperparameter settings. In addition, with the same parameter tuning methodology in a secure WAN setting, the tuning phase can be sped up with parallelism and other coordinated tuning strategies.
- As an alternative to a neural network, a linear-kernel Support Vector Machine (SVM) may be used. The MPC protocol for the secure neural network and SVM may be implemented in C++ based on the number theory package NTL version 10.3.0 for finite field operations. Other data processing scripts are implemented in Python and the NumPy package. Representative computing systems include CP1: 2.40 GHz Intel Xeon E5-2695v2 CPU with 384 GB RAM, CP2: 2.30 GHz Intel Xeon E5-2650v3 CPU with 384 GB RAM; and CP0: 3.33 GHz Intel Xeon X5680 CPU with 96 GB RAM.
- One or more functions of the computing platform of this disclosure may be implemented in a cloud-based architecture. As is well-known, cloud computing is a model of service delivery for enabling on-demand network access to a shared pool of configurable computing resources (e.g. networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a provider of the service. Available services models that may be leveraged in whole or in part include: Software as a Service (SaaS) (the provider's applications running on cloud infrastructure); Platform as a service (PaaS) (the customer deploys applications that may be created using provider tools onto the cloud infrastructure); Infrastructure as a Service (IaaS) (customer provisions its own processing, storage, networks and other computing resources and can deploy and run operating systems and applications).
- The platform may comprise co-located hardware and software resources, or resources that are physically, logically, virtually and/or geographically distinct. Communication networks used to communicate to and from the platform services may be packet-based, non-packet based, and secure or non-secure, or some combination thereof.
- More generally, the techniques described herein are provided using a set of one or more computing-related entities (systems, machines, processes, programs, libraries, functions, or the like) that together facilitate or provide the described functionality described above. In a typical implementation, a representative machine on which the software executes comprises commodity hardware, an operating system, an application runtime environment, and a set of applications or processes and associated data, that provide the functionality of a given system or subsystem. As described, the functionality may be implemented in a standalone machine, or across a distributed set of machines.
- A computing entity herein receives the secretly-shared data from a client device, which may even be an end user device. Thus for example, but without limitation, a client device is a mobile device, such as a smartphone, tablet, or wearable computing device. Such a device comprises a CPU (central processing unit), computer memory, such as RAM, and a drive. The device software includes an operating system (e.g., Google® Android™, or the like), and generic support applications and utilities.
- The underlying network transport may be any communication medium including, without limitation, packet-based, cellular, wireless, Wi-Fi, small cell, and combinations thereof.
- Each above-described process (e.g., each of the protocols set forth in the drawings) preferably is implemented in computer software as a set of computer program instructions executable in one or more processors, as a special-purpose machine.
- Representative machines on which the subject matter herein is provided may be hardware processor-based computers running an operating system and one or more applications to carry out the described functionality.
- While the above describes a particular order of operations performed by certain embodiments of the invention, it should be understood that such order is exemplary, as alternative embodiments may perform the operations in a different order, combine certain operations, overlap certain operations, or the like. References in the specification to a given embodiment indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic.
- While the disclosed subject matter has been described in the context of a method or process, the subject matter also relates to apparatus for performing the operations herein. This apparatus may be a particular machine that is specially constructed for the required purposes, or it may comprise a computer otherwise selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer readable storage medium, such as, but is not limited to, any type of disk including an optical disk, a CD-ROM, and a magnetic-optical disk, a read-only memory (ROM), a random access memory (RAM), a magnetic or optical card, or any type of media suitable for storing electronic instructions, and each coupled to a computer system bus.
- A given implementation of the computing platform is software that executes on a hardware platform running an operating system, such as Linux. A machine implementing the techniques herein comprises a hardware processor, and non-transitory computer memory holding computer program instructions that are executed by the processor to perform the above-described methods.
- Communications herein (e.g., secret sharing of data, sharing of PRGs, etc.) preferably take place over secure connections. For machine-to-machine communications over a network, information typically is communicated over SSL/TLS links, or using any other protocol, although if significant trust is in place (e.g., machines being operated in a secure environment, or between entities that have a trust relationship, etc.) information may be transmitted in the clear. In lieu of using physically-distinct computing entities, computations herein may be carried out within a cluster comprises a set of computing entities (e.g., processors). A grid computing architecture may also be utilized.
- There is no limitation on the type of computing entity that may implement any connection (e.g. client-to-server). Any computing entity (system, machine, device, program, process, utility, or the like) may act as a client or a server.
- While given components of the system have been described separately, one of ordinary skill will appreciate that some of the functions may be combined or shared in given instructions, program sequences, code portions, and the like. Any application or functionality described herein may be implemented as native code, by providing hooks into another application, by facilitating use of the mechanism as a plug-in, by linking to the mechanism, and the like.
- The functionality may be co-located or various parts/components may be separately and run as distinct functions, perhaps in one or more locations (over a distributed network). There may any number of computing parties that securely compute the DTI statistics using the secretly-shared data sets (namely, the data, and the random number data created during the pre-processing operation). The entity that performs pre-processing (CP0) may also be a computing entity (e.g., CP1) in the MPC computation to generate the DIT output.
- The pre-processing may be provided as a service.
- The secure computation may be provided as a service.
- Computing entities herein may be independent from one another, or associated with one another. Multiple computing entities may be associated with a single enterprise entity, but are separate and distinct from one another with respect to the MPC secure computation itself over their respective secret shares.
- The protocol may leverage several advanced computational techniques, which are now generally described. Details of these techniques are provided in U.S. Ser. No. 16/020,058, filed Jun. 27, 2018, the disclosure of which is hereby incorporated by reference. One technique improves upon an existing building block for secure computation, known as Beaver triples. Beaver triples were originally developed for carrying out secure pairwise multiplications; they are extended to arbitrary arithmetic circuits to thereby obtain more efficient subroutines for various operations such as matrix multiplication and exponentiation. Another technique involves using a random projection-based algorithm for Principal Component Analysis (PCA) so that the scale of the data is greatly reduced. An additional technique provides for the notion of shared pseudorandom number generators (PRGs) between pairs of computing entities; the use of shared PRGs obviates transferring a stream of random numbers, which is a frequent operation in the protocol, by enabling each party to independently draw the random numbers from the same PRG. Collectively, these techniques (namely, improved secure computation building blocks, randomized PCA for population stratification, and shared PRGs) enable the provision of a secure protocol that achieves practical scalability.
- These computational techniques leverages building blocks that enable arbitrary arithmetic functions, i.e., addition and multiplication, over the private input, to be securely evaluated. The preferred approach avoids the requirement of generating a Beaver triple for every pairwise multiplication, which is computationally-inefficient especially if the overall computation contains many multiplications. As mentioned above, preferably the MPC technique with elements in a finite field, and there are additional high-level routines for various bit operations like bit shift, bit comparison, etc., which preferably are also written as arithmetic functions so that they can be efficiently-implemented. To address the problem (of having to generate a Beaver triple for every pairwise multiplication), the approach generalizes Beaver triples so that auxiliary random numbers, which enable the secure evaluation of nonlinear functions, are generated for groups of operations at a time, rather than for each individual multiplication. In particular, the generalized Beaver tuple preferably includes a random number for each input value, and some function of these random numbers for each of the output values of a desired computation. Thus, in this operation-centric to data-centric view, the total amount of auxiliary data to be generated and secret-shared depends on the size of input and output, rather than the number of pairwise multiplications, and this leads to significant computational efficiency. Although there are many ways to generate the Beaver triple, according to the protocol (and as noted above) preferably a third party (CP0) is used to perform this task. As noted, the third-party preferably does not observe the input data in any way and is able to finish its task before the main protocol, as a pre-processing step.
- The technical advances in achieving a scalable MPC protocol for realizing private and practical pharmacological collaboration as described herein may be extended for use with diverse applications in biomedicine and other disciplines. The computational approach herein allows pharmaceutical companies and academic researchers (and other interested parties) to pool their data for more accurate predictions while keeping the raw data private. Further, the methods for secure computation removes obstacles for data sharing and, as a result, enable new workflows and opportunities for improved scientific collaboration.
Claims (9)
1. A method for pharmacologic collaboration involving drug target interaction (DTI) data, the DTI data having been secretly shared to generate a pooled DTI dataset, wherein the secret sharing preserves privacy of individual drugs, targets and interactions, comprising:
providing a neural network configured for reduced computation overhead by using an activation function, together with a hinge loss function that evaluates training scores, wherein at least one of the activation and hinge loss functions use a single data-oblivious comparison, thereby reducing the computation overhead;
training the neural network at least in part using gradient descent and with a random subset of training examples for at least one neural network parameter update iteration;
following training that occurs over a scalable runtime due at least in part to the reduced computation overhead, generating one or more DTI predictions for a drug discovery workflow using the neural network; and
outputting the one or more DTI predictions.
2. The method as described in claim 1 wherein the DTI data is associated with two or more collaborating entities.
3. The method as described in claim 1 wherein the DTI data comprises drug and target side information, the drug and target side information including an interaction score, and one or more feature vectors.
4. The method as described in claim 3 wherein the DTI data further include chemical structures and protein sequences describing a drug.
5. The method as described in claim 1 wherein the gradient descent is a stochastic gradient descent with Nesterov momentum.
6. The method as described in claim 1 wherein the training scales linearly with respect to a subset of the DTI data.
7. The method as described in claim 1 wherein the DTI data comprises greater than 106 drug target interactions and the training is executed over a time period measured in hours.
8. The method as described in claim 6 wherein the training is further executed over a wide area network (WAN).
9. The method as described in claim 1 wherein the activation function is a rectified linear unit (RELU) activation function.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US17/948,388 US20230154630A1 (en) | 2017-12-29 | 2022-09-20 | Realizing private and practical pharmacological collaboration using a neural network architecture configured for reduced computation overhead |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201762611678P | 2017-12-29 | 2017-12-29 | |
US16/020,058 US10910087B2 (en) | 2017-06-27 | 2018-06-27 | Secure secret-sharing-based crowdsourcing for large-scale association studies of genomic and phenotypic data |
US16/235,606 US11450439B2 (en) | 2017-12-29 | 2018-12-28 | Realizing private and practical pharmacological collaboration using a neural network architecture configured for reduced computation overhead |
US17/948,388 US20230154630A1 (en) | 2017-12-29 | 2022-09-20 | Realizing private and practical pharmacological collaboration using a neural network architecture configured for reduced computation overhead |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/235,606 Continuation US11450439B2 (en) | 2017-12-29 | 2018-12-28 | Realizing private and practical pharmacological collaboration using a neural network architecture configured for reduced computation overhead |
Publications (1)
Publication Number | Publication Date |
---|---|
US20230154630A1 true US20230154630A1 (en) | 2023-05-18 |
Family
ID=67068168
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/235,606 Active 2038-07-11 US11450439B2 (en) | 2017-12-29 | 2018-12-28 | Realizing private and practical pharmacological collaboration using a neural network architecture configured for reduced computation overhead |
US17/948,388 Pending US20230154630A1 (en) | 2017-12-29 | 2022-09-20 | Realizing private and practical pharmacological collaboration using a neural network architecture configured for reduced computation overhead |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/235,606 Active 2038-07-11 US11450439B2 (en) | 2017-12-29 | 2018-12-28 | Realizing private and practical pharmacological collaboration using a neural network architecture configured for reduced computation overhead |
Country Status (2)
Country | Link |
---|---|
US (2) | US11450439B2 (en) |
WO (1) | WO2019133858A1 (en) |
Families Citing this family (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CA3072638A1 (en) | 2017-08-30 | 2019-03-07 | Inpher, Inc. | High-precision privacy-preserving real-valued function evaluation |
US11593634B2 (en) * | 2018-06-19 | 2023-02-28 | Adobe Inc. | Asynchronously training machine learning models across client devices for adaptive intelligence |
KR20210127168A (en) * | 2019-02-22 | 2021-10-21 | 인퍼, 인코포레이티드 | Arithmetic for secure multiparty computation with modular integers |
US10878950B1 (en) * | 2019-08-09 | 2020-12-29 | HealthBlock, Inc. | Verifying data accuracy in privacy-preserving computations |
CN110457936B (en) * | 2019-07-01 | 2020-08-14 | 阿里巴巴集团控股有限公司 | Data interaction method and device and electronic equipment |
CN113821824B (en) * | 2021-08-27 | 2024-05-24 | 交通银行股份有限公司 | Triplet generation method and system based on careless linear evaluation of OLE |
US11829239B2 (en) | 2021-11-17 | 2023-11-28 | Adobe Inc. | Managing machine learning model reconstruction |
US20230269090A1 (en) * | 2022-02-18 | 2023-08-24 | Onai Inc. | Apparatus for secure multiparty computations for machine-learning |
JP7307429B1 (en) | 2022-03-25 | 2023-07-12 | 株式会社リーディングエッジ | Secret sharing method |
CN116663064B (en) * | 2023-07-25 | 2023-10-20 | 武汉大学 | Privacy protection neural network prediction method and system |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8776250B2 (en) * | 2011-07-08 | 2014-07-08 | Research Foundation Of The City University Of New York | Method of comparing private data without revealing the data |
EP2864916A4 (en) * | 2012-06-21 | 2015-12-23 | Univ Georgetown | Method for predicting drug-target interactions and uses for drug repositioning |
EP3101645B1 (en) | 2014-01-28 | 2019-09-04 | Nippon Telegraph and Telephone Corporation | Secure computation method, secure computation system, secure computation server, registrant terminal, user terminal and program |
WO2019005946A2 (en) | 2017-06-27 | 2019-01-03 | Leighton Bonnie Berger | Secure genome crowdsourcing for large-scale association studies |
-
2018
- 2018-12-28 WO PCT/US2018/067939 patent/WO2019133858A1/en unknown
- 2018-12-28 US US16/235,606 patent/US11450439B2/en active Active
-
2022
- 2022-09-20 US US17/948,388 patent/US20230154630A1/en active Pending
Also Published As
Publication number | Publication date |
---|---|
US11450439B2 (en) | 2022-09-20 |
WO2019133858A1 (en) | 2019-07-04 |
US20190311813A1 (en) | 2019-10-10 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20230154630A1 (en) | Realizing private and practical pharmacological collaboration using a neural network architecture configured for reduced computation overhead | |
Sav et al. | POSEIDON: Privacy-preserving federated neural network learning | |
Patra et al. | BLAZE: blazing fast privacy-preserving machine learning | |
Zheng et al. | Helen: Maliciously secure coopetitive learning for linear models | |
Tan et al. | CryptGPU: Fast privacy-preserving machine learning on the GPU | |
JP6921233B2 (en) | Logistic regression modeling method using secret sharing | |
Wagh et al. | Falcon: Honest-majority maliciously secure framework for private deep learning | |
Bogdanov et al. | High-performance secure multi-party computation for data mining applications | |
Hu et al. | FDML: A collaborative machine learning framework for distributed features | |
Zhu et al. | From federated learning to federated neural architecture search: a survey | |
CN110998579B (en) | Privacy-preserving distributed multi-party security model training framework | |
Froelicher et al. | Scalable privacy-preserving distributed learning | |
Shi et al. | Secure multiparty quantum computation for summation and multiplication | |
Dong et al. | FLOD: Oblivious defender for private Byzantine-robust federated learning with dishonest-majority | |
Xie et al. | BAYHENN: Combining Bayesian deep learning and homomorphic encryption for secure DNN inference | |
Zhang et al. | Privcoll: Practical privacy-preserving collaborative machine learning | |
Semenov et al. | Algorithm for finding partitionings of hard variants of boolean satisfiability problem with application to inversion of some cryptographic functions | |
Hayward et al. | Parallelizing fully homomorphic encryption for a cloud environment | |
Tian et al. | Sphinx: Enabling privacy-preserving online learning over the cloud | |
Gao et al. | SecureRC: a system for privacy-preserving relation classification using secure multi-party computation | |
Khan et al. | Vertical federated learning: A structured literature review | |
Deng et al. | Non-interactive and privacy-preserving neural network learning using functional encryption | |
Dong et al. | Meteor: improved secure 3-party neural network inference with reducing online communication costs | |
Müller-Hermes et al. | Operator Schmidt ranks of bipartite unitary matrices | |
EP3732688A1 (en) | Realizing private and practical pharmacological collaboration |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |