CN111598253A - Training machine learning models using teacher annealing - Google Patents
Training machine learning models using teacher annealing Download PDFInfo
- Publication number
- CN111598253A CN111598253A CN202010404322.8A CN202010404322A CN111598253A CN 111598253 A CN111598253 A CN 111598253A CN 202010404322 A CN202010404322 A CN 202010404322A CN 111598253 A CN111598253 A CN 111598253A
- Authority
- CN
- China
- Prior art keywords
- machine learning
- student
- output
- training
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000010801 machine learning Methods 0.000 title claims abstract description 138
- 238000012549 training Methods 0.000 title claims abstract description 126
- 238000000137 annealing Methods 0.000 title abstract description 10
- 238000000034 method Methods 0.000 claims abstract description 43
- 238000003860 storage Methods 0.000 claims abstract description 13
- 238000012545 processing Methods 0.000 claims description 27
- 230000008569 process Effects 0.000 claims description 21
- 230000006870 function Effects 0.000 claims description 16
- 238000013528 artificial neural network Methods 0.000 claims description 12
- 238000003058 natural language processing Methods 0.000 claims description 5
- 238000009826 distribution Methods 0.000 claims description 3
- 238000004590 computer program Methods 0.000 abstract description 14
- 238000004891 communication Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000036541 health Effects 0.000 description 4
- 230000015654 memory Effects 0.000 description 4
- 230000003993 interaction Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 238000013515 script Methods 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 230000026676 system process Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 238000012935 Averaging Methods 0.000 description 1
- 241000009334 Singa Species 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 230000008451 emotion Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/088—Non-supervised learning, e.g. competitive learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N7/00—Computing arrangements based on specific mathematical models
- G06N7/01—Probabilistic graphical models, e.g. probabilistic networks
Abstract
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a machine learning model using teacher annealing.
Description
Technical Field
This specification relates to training machine learning models.
Background
The machine learning model receives input and generates output, e.g., predicted output, based on the received input and model-based parameter values.
Neural networks are machine learning models that employ one or more layers of nonlinear units to predict output for received inputs. Some neural networks include one or more hidden layers in addition to an output layer. The output of each hidden layer is used as input to the next layer in the network, i.e. the next hidden layer or output layer. Each layer of the network generates an output from the received input in accordance with the current values of the respective parameter set.
Disclosure of Invention
This specification broadly describes a system implemented as one or more computer programs on one or more computers in one or more locations or multiple locations that use teacher annealing (anealing) to train machine learning models.
The machine learning model being trained will be referred to as "student machine learning model" in this specification, and the parameters of the student machine learning model, i.e., the parameters updated by the training, will be referred to as "student parameters".
In particular, during training, the system uses both truth outputs and teacher outputs generated by one or more trained teacher machine learning models. By performing teacher annealing, the system repeatedly adjusts weight values during training that define weights between teacher output and true value output for use in calculating an objective function for training.
Particular embodiments of the subject matter described in this specification can be implemented to realize one or more of the following advantages.
This specification describes using teacher annealing to improve training of student machine learning models when output generated by already trained teacher models is available. By using teacher annealing early in training, the student model is mainly refined, i.e., mainly learned from the output of the teacher model, to get as useful training signals as possible. Near the end of training, student models rely primarily on true value outputs. This progression allows the student machine learning model to achieve performance on any given task that exceeds the teacher machine learning model for that task, even though the student machine learning model is a multitasking machine learning model and the teacher model is a single-tasking machine learning model that is specific to that task. In particular, using these techniques, a student model may exceed the performance of a teacher without training any more training data that is used to train the teacher. In addition, in a multitasking setting, the described techniques allow the student machine learning model to achieve robust multitasking gains across multiple tasks at once, i.e., relative to conventional training techniques for those tasks.
In addition, by utilizing the teacher model during training as described in this specification, the student model may be trained to perform equally well or even better than the teacher model even when the student model consumes less computing resources than the teacher model to generate output. For example, where the teacher model and the student models are both single-tasking models, the student models may have fewer parameters than the teacher model, or generate student outputs with fewer iterations than required by the teacher model. As a particular example, the student model and the teacher model may have similar architectures, but the student model has fewer neural network layers, and thus fewer parameters. As another particular example, the teacher model may be an auto-regression model that generates output through multiple iterations, while the student model is a feed-forward model that generates student output through a single pass-forward of the student model. As another example, when the teacher model is a single-tasking model and the student model is a multitasking model, for example, the student model may have much fewer parameters than the total number of parameter combinations in the teacher model due to having certain parameters shared between all of the multitasking.
The details of one or more embodiments of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.
Drawings
FIG. 1 illustrates an example machine learning model training system.
FIG. 2 is a flow diagram of an example process for training a student machine learning model.
FIG. 3 is a flow diagram of another example process for determining updates to student parameters using current weight values.
Like reference numbers and designations in the various drawings indicate like elements.
Detailed Description
FIG. 1 illustrates an example machine learning model training system 100. The machine learning model training system 100 is an example of a system implemented as a computer program on one or more computers in one or more locations in which the systems, components, and techniques described below may be implemented.
This training system 100 trains a machine learning model 110. The machine learning model 110 being trained will be referred to in this specification as a "student machine learning model" and the parameters 118 of the student machine learning model, i.e., the parameters updated by the training, will be referred to as "student parameters".
In some implementations, the student machine learning model 110 is a single-task machine learning model that is only configured to perform a single machine learning task, and the system 100 trains the student machine learning model 110 to perform the single machine learning task.
For example, a single task may be a natural language processing task, such as an implication task, a paraphrase task, a text similarity task, an emotion task, a grammar task, and the like.
As another example, a single task may be an image processing task, such as image classification, object detection, semantic segmentation, image enhancement, domain transfer, and the like.
As another example, a single task may be a health prediction task, where the input to the machine learning model is the electronic health record data of the patient, and the model output for a given patient may be a probability distribution over patient health related categories (e.g., possible diagnoses of the patient, possible future health events associated with the patient, etc.).
As another example, the single task may be an audio processing task, such as speech recognition, language recognition, hotword detection, and the like.
Thus, in these cases, the student machine learning model 110 receives input specific to a task and generates student output for that task.
In some other implementations, the student machine learning model 110 is a multi-tasking machine learning model configured to perform a plurality of machine learning tasks, and the system 100 trains the student machine learning model 110 to perform all of the plurality of machine learning tasks.
In other words, in these embodiments, the student machine learning model 110 receives inputs of a type common to all machine learning tasks and generates a model output comprising a plurality of student outputs, one for each of the machine learning tasks.
For example, multitasking may be a number of different natural language processing tasks that may be performed on the same input text sequence.
As another example, multitasking may be a plurality of different image processing tasks that may be performed on the same input image.
The machine learning model 110 may have any architecture suitable for the type of model input processed by the machine learning model 110. For example, when the model input is image or audio data, the machine learning model 110 may be a convolutional neural network. When the model input is a text sequence or a sequence of other features, such as electronic medical record features or audio features, the machine learning model 110 may be based on a self-attention neural network (e.g., a transformer) or a recurrent neural network (e.g., a Long Short Term Memory (LSTM) neural network). When the model inputs include multi-modal inputs such as images and text, the model 110 may include different types of neural networks, such as convolutional layers and self-attention or recursive layers.
When the model 110 is a multitasking model, the model 110 may include an initial layer shared among all tasks and a respective set of task-specific output layers for each of the multiple tasks. As another example, all layers of the model 110 may be shared between tasks, and the model inputs processed by the model 110 may each include an identifier or other data that identifies the task to be performed by the model 110 for that model input.
The system 100 receives training data 140 for training the student machine learning model 110. More specifically, for a given task on which system 100 is training student machine learning model 110, training data 140 includes training inputs 142 and, for each training input 142, a true value output 144 for the given task. The true value output 144 is the output that should be generated by the student machine learning model 110 for a given task by processing the training inputs 142. Stated differently, the true value output 144 is a known (presumed) accurate output for a given task.
System 100 then trains student machine learning model 110 using truth outputs 144 and one or more teacher machine learning models 120.
In general, each teacher machine learning model 120 is a machine learning model that has been trained to perform one or more tasks that student machine learning model 110 is being trained to perform.
In particular, when the student machine learning model 100 is a single-tasking model that is being trained for a single task, the system 100 uses a single teacher model 120 that has been trained to perform a single task (or equivalently, a collection of single-tasking teacher models whose outputs are combined to generate a teacher output).
In some of these cases, student machine learning model 110 may be a smaller model than teacher model 120, i.e., may have fewer parameters and a lower computational load (computational complexity) than teacher model 120, and system 100 may use a training process to generate a trained model that is more computationally efficient than a single teacher model 120, while having comparable or higher accuracy than teacher model 120.
For example, teacher model 120 may have a similar architecture as student model 110, but student model 110 may have fewer neural network layers, and thus fewer parameters.
In other of these cases, the teacher machine learning model 120 may be an autoregressive model that generates output over many time steps, while the student machine learning model 110 is a feedforward model that only requires a single time step to generate output. For example, the teacher model 120 may be an autoregressive convolutional, self-attentive, or recurrent neural network, while the student model 110 is a feedforward convolutional neural network.
In other of these cases, teacher model 120 and student model 110 may have the same architecture, and the system may use the training process to generate a trained model that performs better than teacher model 120.
In some cases, when student model 110 is a multitasking model, system 100 uses a single teacher model 120 that is also a multitasking model, and system 100 uses the same teacher model 120 for all tasks. That is, in these cases, the system may be training a model that is more computationally efficient than teacher model 120, or training a model that has the same architecture but improves the performance of teacher model 120, as described above.
In other cases, when student model 110 is a multitasking model, system 100 uses a plurality of different teacher models 120 for different tasks on which multitasking student model 110 is trained. For example, system 100 may use a respective single-task teacher model 120 for each of the multiple tasks being performed by training student models 110. Thus, in these cases, the system 100 is using a training process to train a single model 110, the single model 110 having much fewer parameters and less computation than the combined parameters and computation of all of the teacher models 120.
In general, during training, training engine 150 in system 100 iteratively uses truth output 144 and teacher output 124 generated by one or more teacher models 120 to determine errors in student output 114 generated by student machine learning model 110. The training engine 150 then uses the error to update the values of the model parameters 118.
More specifically, the training engine 150 repeatedly adjusts a weight value, which defines a weighting between the teacher output 124 (i.e., the output generated by one of the teacher models 120) and the true value output 144 used in calculating the objective function for training, using a technique that will be referred to as teacher annealing. Training model 110 using teacher annealing is described in more detail below with reference to fig. 2-3.
In some embodiments, training engine 150 pre-trains student machine learning model 110 on an unsupervised task before training student model 110 using teacher annealing. In some cases, pre-training the student model 110 may improve the performance of the final trained model without the need for additional labeled data. For example, when the task is a natural language processing task, the unsupervised task may be an unsupervised language modeling task, such as described in devin, Jacob; chang, Ming-Wei; lee, Kenton; toutanova, Kristina (10/11/2018). "BERT: pre-training of Deep Bidirectional transducer Language Understanding. arXiv: 1810.04805v 2.
Once the model 110 is trained, the system 100 may provide data specifying the trained model for processing new network inputs. That is, the system 100 may output trained values of the model parameters for later use in processing input using the trained model, such as by output to a user device or by storage in memory accessible to the system 100.
Alternatively or in addition to outputting the trained model data, the system 100 can instantiate an instance of the machine learning model with the trained values of the model parameters, e.g., receive input to be processed through an Application Programming Interface (API) provided by the system, process the received input using the trained model to generate model output, and then provide the generated model output, classification output, or both in response to the received input.
Fig. 2 is a flow diagram of an example process 200 for training a student machine learning model. For convenience, process 200 is described as being performed by a system of one or more computers located at one or more locations. For example, a machine learning model training system, such as the machine learning model training system 100 of FIG. 1, suitably programmed, may perform the process 200.
The system initializes a weight value that defines a weighting between the teacher output and the true output (step 202). In other words, the system sets the weight value to a fixed initial value before starting training.
When calculating the weighted combination of teacher outputs, the weight values determine the weights assigned to the teacher output for a given training input and the true value output for the given training input.
In some cases, the weight value λ may be the weight assigned to the true output in the weighted combination, and then the weight assigned to the teacher output is equal to 1- λ. In these cases, the system initializes the weight values to values equal to 0 or within a range of threshold 0, i.e., so that the teacher output is initially strongly supported in the weighted combination with respect to the true value output.
In other cases, the weight value λ may be the weight assigned to the teacher output in a weighted combination, and then the weight assigned to the true value output is equal to 1- λ. In these cases, the system initializes the weight value to a value equal to 1 or within a range of threshold 1, i.e., so that the teacher output is initially strongly supported in the weighted combination with respect to the true value output.
The system trains the student machine learning model on the training data until the criteria for updating the weight values are met (step 204).
In particular, the system repeatedly performs training iterations on small batches of training iterations to optimize an objective function that measures, for any given training input, the error between (i) a weighted combination of the teacher output and the true value output for the given training input and (ii) the student output generated by the student machine learning model for the training input.
For example, when λ is a weight assigned to the true value output, the objective function/for task T may be expressed as:
wherein the content of the first and second substances,
is a training input for task T
The true value of (a) is output,
is generated by a teacher model trained for the task T according to the parameters theta of the teacher modelTFor training input
Generated teacher output, and
is directed to the training input by the student model according to the current value of the student parameter theta
The generated student output.
The objective function measures the error in a manner that is appropriate for a given task T.
For example, when task T is a classification task and student output, teacher output, and truth output are probability distributions, the objective function may be to measure the cross-entropy loss of cross-entropy between the weighted combination and the student output.
As another example, when task T is a regression task and student output, teacher output, and true value output are all an ordered set of one or more regression values, the objective function may be a distance penalty measuring the distance between the weighted combination and the student output, e.g., an L2 distance penalty.
In each training iteration, the system uses a small batch of training data to compute an update to the current values of the student model parameters at the time of the cutoff iteration, and then applies the update to the current values of the student parameters. Updating the current value is described below with reference to fig. 3.
The system may determine that the criteria for updating the weight values are met at any suitable point during training. For example, the system may determine that the criteria are met every N training iterations, i.e., after every N updates applied to the student model. As another example, the system may determine that the criteria are met each time a certain amount of time elapses during training. As yet another example, the system may maintain a set of performance benchmarks and may determine that a criterion is met whenever, for example, performance of a student model as measured on a validation dataset reaches one of the performance benchmarks.
Once the criteria are met during training, the system updates the weight values to gradually support the true value output in the weighting (step 206).
For example, when a true value output is assigned a weight equal to a weight value, the system may linearly increase the weight at some point during training, i.e., each time a criterion is met, so that the weight value moves close to 1 from a zero or near-zero starting point.
That is, the system updates the weight values according to a linear schedule to gradually support true value output, e.g., linearly increasing the weight after every N updates are applied to the student parameter (when the true value output is assigned a weight equal to the weight value) or linearly decreasing the weight after every N updates have been applied (when the teacher output is assigned a weight equal to the weight value).
As another example, the system may update the weight values according to an exponential schedule to gradually support a true value output, for example, increasing the weight exponentially after every N updates have been applied to the student parameter (when the true value output is assigned a weight equal to the weight value), or decreasing the weight exponentially after every N updates have been applied (when the teacher output is assigned a weight equal to the weight value).
The process 200 then returns to step 204, i.e., the system continues to train the model until the criteria for updating the weight values are again met. The system may continue to repeat steps 204 and 206 until termination criteria for training are met, e.g., a specified number of iterations of step 204 have been performed, a specified amount of time has elapsed or the student model parameters have converged.
Thus, by repeatedly performing steps 204 and 206 during training, the system repeatedly updates the weight values to gradually support the true value output in the weighting. That is, as training progresses, the system will continue to adjust the weight values so that the true outputs are given more and more weight in the weighted combination relative to the teacher output.
Fig. 3 is a flow diagram of an example process 300 for determining updates to current values of parameters of a student machine learning model. For convenience, process 300 is described as being performed by a system of one or more computers located at one or more locations. For example, a machine learning model training system, such as the machine learning model training system 100 of fig. 1, suitably programmed, may perform the process 300 by adjusting weight values as described in this specification.
The system may perform process 300 on each of a small batch of training inputs to determine a respective update for each training input. The system may then combine the updates, for example by summing or averaging the updates, and apply the combined updates to the current values of the student parameters, for example by adding or subtracting the combined updates to or from the current values.
When the student model is a single-task model, the training inputs in the small batch will all be training inputs of a single task.
When the student model is a multitasking model, the small batch may include training inputs for different ones of the multitasking. For example, the system may sample each training input in the mini-batch from an overall training data set that includes training data for all tasks, or may sample a specified number of training inputs from each of the multi-tasks to generate the mini-batch.
The system obtains a training input for the first machine learning task and a true value output for the training input (step 302).
The system processes the training inputs using the trained first teacher machine learning model to generate teacher outputs for the first machine learning task (step 304). As described above, the trained first teacher machine learning model has been trained to perform the first machine learning task. In some cases, prior to starting training, the system preprocesses all training data for the first task using the trained first teacher machine learning model to generate teacher outputs for training inputs in the training data. In other cases, the system uses the trained first machine learning model to process training input online, i.e., when teacher output is needed for training during training.
The system determines a weighted combination of the teacher output for the first machine learning task and the true output for the first training input based on the weight values (step 306). In other words, the system calculates a weighted combination of the teacher output and the true output by weighting the true output and the teacher output according to the current weights at the time of the current iteration of process 300.
The system processes the training inputs using the student machine learning model and according to the student parameters to generate student outputs for the first machine learning task (step 308).
The system then determines a gradient of the student parameter relative to an objective function by back-propagation, the objective function measuring the error between the weighted combination and the student output (step 310), and determines an update to the student parameter based on the gradient (step 312). For example, the system may determine the updates according to update rules (e.g., random gradient descent, rmsProp, Adam, etc.) of an optimizer used to train the machine learning model, i.e., by applying the update rules of the optimizer to the gradients.
This specification uses the term "configured" in connection with system and computer program components. A system for one or more computers to be configured to perform particular operations or actions means that the system has installed thereon software, firmware, hardware or a combination of software, firmware, hardware that in operation causes the system to perform the operations or actions. By one or more computer programs to be configured to perform particular operations or actions is meant that the one or more programs include instructions which, when executed by a data processing apparatus, cause the apparatus to perform the operations or actions.
Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly embodied computer software or firmware, in computer hardware comprising the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on a tangible, non-transitory storage medium for execution by, or to control the operation of, data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access storage device, or a combination of one or more of them. Alternatively or in addition, the program instructions may be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by the data processing apparatus.
The term "data processing apparatus" refers to data processing hardware and encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. An apparatus may also be, or further comprise, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). The apparatus can optionally include, in addition to hardware, code that creates an execution environment for the computer program, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.
A computer program, which may also be referred to or described as a program, software application, app, module, software module, script, or code, may be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages; and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, such as one or more scripts stored in a markup language document; in a single file dedicated to the program or in multiple coordinated files, such as files storing one or more modules, sub programs, or portions of code. A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a data communication network.
In this specification, the term "database" is used broadly to refer to any set of data: the data need not be structured in any particular way, or at all, and it may be stored on a storage device in one or more locations. Thus, for example, an index database may include multiple data sets, each of which may be organized and accessed differently.
Similarly, the term "engine" is used broadly in this specification to refer to a software-based system, subsystem, or process that is programmed to perform one or more particular functions. Typically, the engine will be implemented as one or more software modules or components installed on one or more computers in one or more locations. In some cases, one or more computers will be dedicated to a particular engine; in other cases, multiple engines may be installed and run on the same computer or multiple computers.
The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and in combination with, special purpose logic circuitry, e.g., an FPGA or an ASIC.
A computer suitable for executing a computer program may be based on a general purpose microprocessor, or a special purpose microprocessor, or both, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a central processing unit for executing or carrying out instructions and one or more memory devices for storing instructions and data. The central processing unit and the memory can be supplemented by, or incorporated in, special purpose logic circuitry. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Further, the computer may be embedded in another device, e.g., a mobile telephone, a Personal Digital Assistant (PDA), a mobile audio or video player, a game controller, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a Universal Serial Bus (USB) flash drive, etc.
Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and storage devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks.
To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. Further, the computer may interact with the user by sending documents to and receiving documents from the device used by the user; for example, by sending a web page to a web browser on the user's device in response to receiving a request from the web browser. In addition, the computer may interact with the user by sending a text message or other form of message to a personal device, such as a smartphone that is running a messaging application, and then receiving a response message from the user.
The data processing apparatus for implementing a machine learning model may also include dedicated hardware accelerator units, e.g., for processing common and computationally intensive portions of machine learning training or production, i.e., reasoning, workload.
The machine learning model may be implemented and deployed using a machine learning framework. The machine learning framework is, for example, a TensorFlow framework, a Microsoft Cognitive Toolkit framework, an Apache Singa framework, or an Apache MXNet framework.
Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, e.g., as a data server; or include middleware components, such as application servers; or include a front-end component, such as a client computer having a graphical user interface, a web browser, or an app with which a user can interact with an implementation of the subject matter described in this specification; or any combination of one or more such back-end, intermediate, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a Local Area Network (LAN) and a Wide Area Network (WAN), such as the Internet.
The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, the server transmits data, e.g., HTML pages, to the user device, e.g., for the purpose of displaying data to a user interacting with the device as a client and receiving user input from the user. Data generated at the user device, e.g., results of user interactions, may be received at the server from the device.
While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.
Similarly, while operations are depicted in the drawings and described in the claims in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.
Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous.
Claims (15)
1. A method of training a student machine learning model having a plurality of student parameters to perform at least a first machine learning task, wherein the student machine learning model is configured to receive model inputs and process the model inputs in accordance with the student parameters to generate an output comprising student outputs for the first machine learning task, the method comprising:
initializing a weight value defining a weighting between the teacher output and the true value output;
training the student machine learning model on training data, the training data comprising a plurality of first training inputs and a respective true value output for each first training input of the first machine learning task, the training comprising, for each first training input:
processing the first training input using a trained first teacher machine learning model to generate a teacher output for the first machine learning task, wherein the trained first teacher machine learning model has been trained to perform the first machine learning task;
determining a weighted combination of the teacher output for the first machine learning task and the true value output for the first training input according to the weight values;
processing the first training input using the student machine learning model and in accordance with the student parameters to generate student output for the first machine learning task;
determining a gradient of a student parameter with respect to an objective function that measures an error between the weighted combination and the student output; and
determining an update to the student parameter based on the gradient; and
during the training, the weight values are repeatedly updated to gradually support the true value output in the weighting.
2. The method of claim 1, wherein repeatedly updating the weight values comprises:
repeatedly linearly increasing the weight value during the training.
3. The method of claim 1, wherein the student output, the teacher output, and the truth output are probability distributions, and wherein the objective function is a cross-entropy loss that measures cross-entropy between the weighted combination and the student output.
4. The method of claim 1, wherein the first machine learning task is a regression task, wherein the student output, the teacher output, and the truth output are each an ordered set of one or more regression values, and wherein the objective function is a distance loss measuring a distance between the weighted combination and the student output.
5. The method of claim 1, wherein the student machine learning model is a single-task model trained only to perform the first machine learning task, and wherein the model output includes only student output for the first machine learning task.
6. The method of claim 1, wherein the first and second light sources are selected from the group consisting of,
wherein the student machine learning model is a multi-task model trained to perform a plurality of machine learning tasks including the first machine learning task,
wherein the model output comprises a respective student output for each of the plurality of machine learning tasks,
wherein the training data comprises a respective plurality of training inputs for each of the plurality of machine learning tasks and a respective true value output for the first machine learning task for each of the plurality of training inputs for that machine learning task, and
wherein the training comprises training the student machine learning model on the training data to perform all of the plurality of machine learning tasks.
7. The method of claim 6, wherein the trained first teacher machine learning model is a multi-tasking model that has been trained on the plurality of machine learning tasks, and
wherein, for each of the plurality of machine learning tasks and for each of the plurality of training inputs for that machine learning task, the training further comprises:
processing the training input using the trained first teacher machine learning model to generate a teacher output for the machine learning task;
determining a weighted combination of the teacher output for the machine learning task and the true value output for the training input according to the weight values;
processing the training input using the student machine learning model and in accordance with the student parameters to generate student output for the machine learning task;
determining a gradient of a student parameter with respect to an objective function that measures an error between the weighted combination and the student output; and
determining an update to the student parameter based on the gradient.
8. The method of claim 6, wherein the trained first teacher machine learning model is a single task model, and
wherein, for each of the plurality of tasks and for each training input for that task, the training further comprises:
processing the training input using a different trained teacher machine learning model specific to the machine learning task to generate a teacher output for the machine learning task;
determining a weighted combination of the teacher output for the machine learning task and the true value output for the training input according to the weight values;
processing the training input using the student machine learning model and in accordance with the student parameters to generate student output for the machine learning task;
determining a gradient of a student parameter with respect to an objective function that measures an error between the weighted combination and the student output; and
determining an update to the student parameter based on the gradient.
9. The method of claim 6, wherein the student machine learning model is a neural network comprising an encoder neural network shared among all of the plurality of machine learning tasks and a respective output neural network for each of the plurality of machine learning tasks.
10. The method of claim 6, wherein the model input of the student machine learning model is a text sequence of a natural language, and wherein the plurality of machine learning tasks are different natural language processing tasks executable on the text sequence.
11. The method according to any one of claims 1-10, further comprising:
pre-training the student machine learning model on an unsupervised task prior to the training.
12. The method of claim 11, wherein the first machine learning task is a natural language processing task and the unsupervised task is an unsupervised language modeling task.
13. The method according to any one of claims 1-10, further comprising:
after training the student machine learning model to perform at least the first machine learning task:
receiving a new network input for the first machine learning task; and
processing the new network inputs using the trained student machine learning model to generate new network outputs for the new network inputs of the first machine learning task.
14. A system comprising one or more computers and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations of the respective methods of any of claims 1-13.
15. One or more computer storage media storing instructions that, when executed by one or more computers, cause the one or more computers to perform operations of the respective methods of any of claims 1-13.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US201962847220P | 2019-05-13 | 2019-05-13 | |
| US62/847,220 | 2019-05-13 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111598253A true CN111598253A (en) | 2020-08-28 |
Family
ID=72185419
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010404322.8A Pending CN111598253A (en) | 2019-05-13 | 2020-05-13 | Training machine learning models using teacher annealing |
Country Status (2)
| Country | Link |
|---|---|
| US (2) | US11488067B2 (en) |
| CN (1) | CN111598253A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116542310A (en) * | 2023-07-01 | 2023-08-04 | 帕西尼感知科技(张家港)有限公司 | Model training and motion instruction prediction method, device and system for robot |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11488067B2 (en) * | 2019-05-13 | 2022-11-01 | Google Llc | Training machine learning models using teacher annealing |
| US11501081B1 (en) | 2019-12-31 | 2022-11-15 | Meta Platforms, Inc. | Methods, mediums, and systems for providing a model for an end-user device |
| CN112529153B (en) * | 2020-12-03 | 2023-12-22 | 平安科技（深圳）有限公司 | BERT model fine tuning method and device based on convolutional neural network |
| US20220293247A1 (en) * | 2021-03-12 | 2022-09-15 | Siemens Healthcare Gmbh | Machine learning for automatic detection of intracranial hemorrhages with uncertainty measures from ct images |
| WO2023158881A1 (en) * | 2022-02-18 | 2023-08-24 | Google Llc | Computationally efficient distillation using generative neural networks |
| KR20240014374A (en) | 2022-07-25 | 2024-02-01 | 삼성전자주식회사 | Computer system for compressing transformer model and quantization training method thereof |
| US20240070232A1 (en) * | 2022-08-22 | 2024-02-29 | Nec Laboratories America, Inc. | Snr detection with few-shot trained models |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017201511A1 (en) * | 2016-05-20 | 2017-11-23 | Google Llc | Training machine learning models |
| WO2018126213A1 (en) * | 2016-12-30 | 2018-07-05 | Google Llc | Multi-task learning using knowledge distillation |
| WO2018217635A1 (en) * | 2017-05-20 | 2018-11-29 | Google Llc | Application development platform and software development kits that provide comprehensive machine learning services |
| CN109598342A (en) * | 2018-11-23 | 2019-04-09 | 中国运载火箭技术研究院 | A kind of decision networks model is from game training method and system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170132528A1 (en) * | 2015-11-06 | 2017-05-11 | Microsoft Technology Licensing, Llc | Joint model training |
| US11488067B2 (en) * | 2019-05-13 | 2022-11-01 | Google Llc | Training machine learning models using teacher annealing |
-
2020
- 2020-05-11 US US16/872,271 patent/US11488067B2/en active Active
- 2020-05-13 CN CN202010404322.8A patent/CN111598253A/en active Pending
-
2022
- 2022-10-31 US US17/978,045 patent/US11922281B2/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017201511A1 (en) * | 2016-05-20 | 2017-11-23 | Google Llc | Training machine learning models |
| WO2018126213A1 (en) * | 2016-12-30 | 2018-07-05 | Google Llc | Multi-task learning using knowledge distillation |
| WO2018217635A1 (en) * | 2017-05-20 | 2018-11-29 | Google Llc | Application development platform and software development kits that provide comprehensive machine learning services |
| CN109598342A (en) * | 2018-11-23 | 2019-04-09 | 中国运载火箭技术研究院 | A kind of decision networks model is from game training method and system |
Non-Patent Citations (2)
| Title |
|---|
| 李丽娜;马俊;梁德;李文浩;: "基于相关向量机及模拟退火的损耗模型训练" * |
| 李丽娜;马俊;梁德;李文浩;: "基于相关向量机及模拟退火的损耗模型训练", 计算机工程与设计, no. 01, 16 January 2016 (2016-01-16), pages 139 - 145 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116542310A (en) * | 2023-07-01 | 2023-08-04 | 帕西尼感知科技(张家港)有限公司 | Model training and motion instruction prediction method, device and system for robot |
| CN116542310B (en) * | 2023-07-01 | 2023-09-22 | 帕西尼感知科技(张家港)有限公司 | Model training and motion instruction prediction method, device and system for robot |
Also Published As
| Publication number | Publication date |
|---|---|
| US20200364617A1 (en) | 2020-11-19 |
| US20230049747A1 (en) | 2023-02-16 |
| US11488067B2 (en) | 2022-11-01 |
| US11922281B2 (en) | 2024-03-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10748065B2 (en) | Multi-task neural networks with task-specific paths | |
| EP3602409B1 (en) | Selecting actions using multi-modal inputs | |
| CN111598253A (en) | Training machine learning models using teacher annealing | |
| CN111602148B (en) | Regularized neural network architecture search | |
| EP3696737B1 (en) | Training action selection neural networks | |
| US11790214B2 (en) | Mixture of experts neural networks | |
| US11875262B2 (en) | Learning neural network structure | |
| CN111406267A (en) | Neural architecture search using performance-predictive neural networks | |
| JP2020521205A (en) | Multi-task neural network system with task-specific and shared policies | |
| KR20200110400A (en) | Learning data augmentation policy | |
| KR20190028531A (en) | Training machine learning models for multiple machine learning tasks | |
| US20210271970A1 (en) | Neural network optimizer search | |
| US20220092416A1 (en) | Neural architecture search through a graph search space | |
| CN117709426A (en) | Method, system and computer storage medium for training machine learning model | |
| US20240127058A1 (en) | Training neural networks using priority queues | |
| US20200410365A1 (en) | Unsupervised neural network training using learned optimizers | |
| US11755879B2 (en) | Low-pass recurrent neural network systems with memory | |
| JP2020506488A (en) | Batch renormalization layer | |
| CN113826125A (en) | Training machine learning models using unsupervised data enhancement | |
| US20220230065A1 (en) | Semi-supervised training of machine learning models using label guessing | |
| CN113348472A (en) | Convolutional neural network with soft kernel selection | |
| CN114492758A (en) | Training neural networks using layer-by-layer losses | |
| CN114730380A (en) | Deep parallel training of neural networks | |
| US11886976B1 (en) | Efficient decoding of output sequences using adaptive early exiting | |
| US20230325658A1 (en) | Conditional output generation through data density gradient estimation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |