CN111985637A - Machine learning model with conditional execution of multiple processing tasks - Google Patents
Machine learning model with conditional execution of multiple processing tasks Download PDFInfo
- Publication number
- CN111985637A CN111985637A CN202010437644.2A CN202010437644A CN111985637A CN 111985637 A CN111985637 A CN 111985637A CN 202010437644 A CN202010437644 A CN 202010437644A CN 111985637 A CN111985637 A CN 111985637A
- Authority
- CN
- China
- Prior art keywords
- task
- machine learning
- learning model
- trained machine
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/10—Interfaces, programming languages or software development kits, e.g. for simulating neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Abstract
The present disclosure provides a method that includes receiving input data at a trained machine learning model that includes a common portion and a task-specific portion, receiving an execution instruction that identifies one or more processing tasks to be performed, processing the input data using the common portion of the trained machine learning model to generate intermediate data, and processing the intermediate data using one or more of the task-specific portions of the trained machine learning model based on the execution instruction to generate one or more outputs.
Description
Of the related applicationCross-referencing
This application claims the benefit of U.S. patent application serial No. 62/850,618 filed on 21/5/2019, which is hereby incorporated by reference in its entirety for all purposes.
Technical Field
The present disclosure relates to machine learning models with conditional execution of multiple processing tasks.
Background
Machine learning models, such as deep neural networks, are typically configured to perform a single task. For example, when three processing tasks are to be performed using a set of input data, the three tasks are typically performed by three neural networks, each processing one of the tasks.
Disclosure of Invention
One aspect of the disclosure is a method that includes receiving input data at a trained machine learning model that includes a common portion and a task-specific portion, receiving execution instructions that identify one or more processing tasks to be performed, processing the input data using the common portion of the trained machine learning model to generate intermediate data, and processing the intermediate data using one or more of the task-specific portions of the trained machine learning model based on the execution instructions to generate one or more outputs.
In some implementations of the method, processing the intermediate data using one or more of the task-specific portions of the trained machine learning model based on the execution instructions includes evaluating the conditional instructions for each of the task-specific portions of the trained machine learning model to determine whether to initiate execution of each of the task-specific portions of the trained machine learning model.
In some implementations of the method, processing the intermediate data using one or more of the task-specific portions of the trained machine learning model based on the execution instructions includes causing execution of one or more of the task-specific portions of the trained machine learning model based on the execution instructions.
In some implementations of the method, processing the intermediate data using one or more of the task-specific portions of the trained machine learning model based on the execution instructions includes suppressing execution of one or more of the task-specific portions of the trained machine learning model based on the execution instructions.
In some implementations of the method, processing the intermediate data using one or more of the task-specific portions of the trained machine learning model based on the execution instruction includes loading only one or more of the task-specific portions of the trained machine learning model identified by the execution instruction.
In some implementations of the method, processing the intermediate data using one or more of the task-specific portions of the trained machine learning model based on executing the instructions includes offloading at least a portion of the task-specific portions of the trained machine learning model.
The method may also include defining, based on the execution instructions, a linear execution order including one or more of a common portion of the trained machine learning model and a task-specific portion of the trained machine learning model. In such implementations, processing the input data using the common portion of the trained machine learning model and processing the intermediate data using one or more of the task-specific portions of the trained machine learning model are performed according to a linear execution order.
In some implementations of the method, one or more of the task-specific portions of the trained machine learning model include a first task-specific portion of the trained machine learning model and a second task-specific portion of the trained machine learning model, and processing the intermediate data using one or more of the task-specific portions of the trained machine learning model based on the execution instructions includes sequentially executing the first task-specific portion of the trained machine learning model and the second task-specific portion of the trained machine learning model.
In some implementations of the method, one or more of the task-specific portions of the trained machine learning model include a first task-specific portion of the trained machine learning model and a second task-specific portion of the trained machine learning model, and processing the intermediate data using one or more of the task-specific portions of the trained machine learning model based on the execution instructions includes executing the first task-specific portion of the trained machine learning model and the second task-specific portion of the trained machine learning model in parallel.
Another aspect of the disclosure is a non-transitory computer readable storage device comprising program instructions executable by one or more processors. The program instructions, when executed, cause one or more processors to perform operations. The operations include receiving input data at a trained machine learning model that includes a common portion and a task-specific portion, receiving execution instructions that identify one or more processing tasks to be performed, processing the input data using the common portion of the trained machine learning model to generate intermediate data, and processing the intermediate data using one or more of the task-specific portions of the trained machine learning model based on the execution instructions to generate one or more outputs.
Another aspect of the disclosure is a system that includes a memory and a processor. The processor is configured to execute instructions stored in the memory. The instructions, when executed by the processor, cause the processor to receive input data at a trained machine learning model that includes a common portion and a task-specific portion, receive execution instructions that identify one or more processing tasks to be performed, process the input data using the common portion of the trained machine learning model to generate intermediate data, and process the intermediate data using one or more of the task-specific portions of the trained machine learning model based on the execution instructions to generate one or more outputs.
Drawings
FIG. 1 is a block diagram illustrating a processing system including a trained machine learning model.
FIG. 2 is a diagram illustrating an example of functional relationships and data dependencies between portions of a particular implementation of a trained machine learning model.
Fig. 3 is a flowchart showing a first example of a process for executing a processing task.
Fig. 4 is a flowchart showing a second example of a process for executing a processing task.
Fig. 5 is a diagram illustrating an example of a hardware configuration for a computing device.
Detailed Description
When multiple processing tasks are performed using separate neural networks (e.g., deep neural networks), each processing task has its own memory footprint and serial processing of these tasks may be required. Some processing tasks may be redundant when the associated task uses a common set of input data or a large overlapping set of input data. For example, each network may perform a similar set of processing operations before performing different processing operations specific to the respective processing task.
The systems and methods described herein relate to neural networks capable of performing two or more processing tasks using a set of input data and a common set of initialization layers. The initial portion of the neural network performs processing operations that are common to all of the two or more processing tasks. The output of the initial portion of the input data is provided as input to two or more network portions, each network portion performing less than all of the processing operations required for the processing task. The network part is conditionally executed. If one of the processing tasks does not need to be processed with respect to a particular set of input data, that processing task may be skipped or otherwise excluded from execution.
Fig. 1 is a block diagram illustrating a processing system 100. The processing system 100 utilizes the input data 102 to generate output 104 using a trained machine learning model 106. The operation of the trained machine learning model 106 is controlled by the execution instructions 108. As will be explained herein, the execution instructions 108 control how the input data 102 is processed by the common portion 110 of the trained machine learning model 106 and the task-specific portion 112 of the trained machine learning model 106. During operation of the trained machine learning model 106, the common portion 110 of the trained machine learning model 106 and the task-specific portion 112 of the trained machine learning model 106 may generate intermediate data 114 that may be saved and loaded as needed by the trained machine learning model 106.
The input data 102 is a data item or set of data stored in a format that can be processed by the trained machine learning model 106. The input data 102 may be provided directly to the trained machine learning model 106, or may undergo one or more processing operations prior to use by the trained machine learning model 106. Many types of information may be used as input data 102 or included in the input data depending on the processing tasks that processing system 100 is configured to perform. For example, the input data 102 may be or include images in implementations in which the processing task includes an image analysis task such as object detection (e.g., a digital image defined by an array of pixels, each pixel having a color represented by a component value or by an index value).
The outputs 104 are results obtained from processing tasks performed by the processing system 100, and these results depend on the input data. The output 104 is information that can be understood by a human operator or a high-level processing task, and that is intended to be processed by other than the processing system 100 or utilized by a human operator. The output 104 may be verifiable against objective criteria to allow for an assessment of whether the output 104 is correct or incorrect. For example, in implementations in which the processing system 100 is configured to detect objects in an image, the output 104 may include a label (e.g., descriptive text) that identifies the type of object present in the image and the location (e.g., center point or bounding box) of the image relative to where the object is located.
The processing system 100 performs a plurality of processing tasks according to execution instructions 108 using the trained machine learning model 106. The trained machine learning model 106 includes a common portion 110 and a task-specific portion 112. Common portion 110 and task-specific portion 112 are each a group of one or more operations that are grouped according to their relevance to the processing task to be performed by processing system 100. The common portion 110 of the machine learning model 106 includes operations related to all tasks to be performed by the processing system 100. The task-specific portions 112 each include one or more operations related to a single processing task to be performed by the processing system 100 or related to two or more processing tasks to be performed by the processing system 100.
For example, the trained machine learning model 106 may be a deep neural network, where the common portion 110 of the trained machine learning model 106 has one or more layers of processing units (e.g., neurons) and the task-specific portions 112 of the trained machine learning model 106 each have one or more layers of processing units (e.g., neurons). One or more layers of the common portion 110 of the trained machine learning model 106 include an input layer configured to receive input data 102. At least a portion of the task-specific portion 112 of the trained machine learning model 106 includes an output layer configured to produce one or more outputs 104.
During operation of the processing system 100, the trained machine learning model 106 may generate intermediate data 114. The intermediate data 114 is an output produced by a portion of the trained machine learning model 106, such as the common portion 110 or the task-specific portion 112 of the trained machine learning model 106. However, the intermediate data 114 is distinguishable from the output 104 because the information contained in the intermediate data 114 is not readily understandable by humans or systems other than the trained machine learning model 106, and does not represent results intended for either processing task. Instead, the intermediate data 114 includes values produced by a portion of the trained machine learning model 106 (such as the common portion 110 or one of the task-specific portions 112) and is intended for further processing by one or more of the task-specific portions 112 of the trained machine learning model 106. For example, the intermediate data 114 may include values (e.g., values generated by neurons before or after being modified by an activation function) output by processing units of a layer (e.g., a hidden layer) included in the common portion 110 of the trained machine learning model 106 or in one of the task-specific portions 112 of the trained machine learning model 106.
Training of the trained machine learning model 106 may be performed using conventional training techniques. The common portion 110 and the task-specific portion 112 may be later defined by determining relevance boundaries between portions of the trained machine learning model 106.
Fig. 2 is a diagram illustrating an example of functional relationships and data dependencies between portions of an example of a specific implementation of the trained machine learning model 106. In the illustrated example, the trained machine learning model 106 is configured to perform three different processing tasks, referred to herein as a first processing task, a second processing task, and a third processing task. When the trained machine learning model 106 is executed (i.e., processing of input data is initiated), some or all of these processing tasks may be selected for use, and the unselected processing tasks are not executed.
In this example, the trained machine learning model 106 includes a common portion 110 and a task-specific portion 112, as previously described. The task-specific portions 112 of the trained machine learning model include a first task-specific portion 212a, a second task-specific portion 212b, a third task-specific portion 212c, and a fourth task-specific portion 212 d. The intermediate data 114 includes first intermediate data 214a and second intermediate data 214 b. The outputs 104 include a first output 204a associated with the first processing task, a second output 204b associated with the second processing task, and a third output 204c associated with the third processing task.
The common portion 110 receives the input data 102 as input and processes it. To process the input data 102, the common portion 110 may be configured as part of a deep neural network, including multiple layers (e.g., an input layer and one or more hidden layers) of processing elements (e.g., neurons). The output of the common portion 110 is first intermediate data 214 a.
The first task-specific part 212a receives as input and processes the first intermediate data 214 a. To process the first intermediate data 214a, the first task-specific portion 212a may be part of a deep neural network, including one or more layers (e.g., a hidden layer and an output layer) of processing elements (e.g., neurons). The output of the first task-specific portion 212a is the first output 204 a.
The second task specific part 212b receives as input and processes the first intermediate data 214 a. To process the first intermediate data 214a, the second task-specific portion 212b may be part of a deep neural network, including one or more layers (e.g., hidden layers) of processing elements (e.g., neurons). The output of the second task-specific part 212b is second intermediate data 214 b.
The third task-specific part 212c receives as input and processes the second intermediate data 214 b. To process the second intermediate data 214b, the third task-specific portion 212c may be part of a deep neural network, including one or more layers (e.g., a hidden layer and an output layer) of processing elements (e.g., neurons). The output of the third task-specific portion 212c is the second output 204 b.
The fourth task specific part 212d receives as input and processes the second intermediate data 214 b. To process the second intermediate data 214b, the fourth task-specific portion 212d may be part of a deep neural network, including one or more layers (e.g., a hidden layer and an output layer) of processing elements (e.g., neurons). The output of the fourth task-specific portion 212d is the third output 204 c.
As can be seen from the foregoing, the first processing task, the second processing task and the third processing task each comprise a different set of operations, but there is some overlap between them. The first processing operation entails executing the common portion 110 and the first task-specific portion 212a to produce the first output 204 a. The second processing operation entails executing the common portion 110, the second task-specific portion 212b, and the third task-specific portion 212c to produce the second output 204 b. The third processing operation entails executing the common portion 110, the second task-specific portion 212b, and the fourth task-specific portion 212d to produce a third output 204 c. Thus, the shared portion 110 is executed whenever any processing task is performed, the first task-specific portion 212a is executed only when the first processing task is performed, the second task-specific portion 212b is executed when either the second processing task or the third processing task is performed, the third task-specific portion 212c is executed only when the second processing task is performed, and the fourth task-specific portion 212d is executed only when the third processing task is performed.
The execution instructions 108 may include information indicating which processing tasks are to be performed. During operation of the trained machine learning model 106, the execution instructions 108 are used to control whether each task-specific portion 112 of the trained machine learning model 106 is executed. Executing the instructions need not include direct selection of any particular portion of the trained machine learning model 106. For example, the execution instructions 108 may describe the processing task to be performed according to the list of outputs of interest, and may control execution of task-specific portions of the trained machine learning model 106 based on the execution instructions 108 to perform those task-specific portions of the trained machine learning model 106 needed to produce the output specified by the execution instructions 108.
For example, the execution instructions 108 may indicate which processing tasks to execute, and conditional instructions (e.g., if statements or other conditional operations) may be evaluated to determine whether to execute each task-specific portion in a serial operation in which portions of the machine learning model 106 are executed one at a time in an order determined by the data dependencies. Thus, in the present example, the shared portion 110 is executed, the first conditional instruction causing execution of the first task-specific portion 212a if the first processing task is active, the second conditional instruction causing execution of the second task-specific portion 212b if either the second processing task or the third processing task is active, the third conditional instruction causing execution of the third task-specific portion 212c if the second processing task is active, and the fourth conditional instruction causing execution of the fourth task-specific portion 212d if the third processing task is active.
As another example, the execution instructions 108 may indicate which processing tasks are to be performed, and the execution instructions 108 are for configuring the trained machine learning model 106 prior to execution by loading and/or unloading portions of the trained machine learning model 106 based on the execution instructions 108. This results in a linear program for each possible combination of processing tasks and only requires one evaluation of the execution instructions 108 before executing the trained machine learning model 106.
Thus, in the current example, if only the first processing task is to be performed, the common portion 110 and the first task-specific portion 212a are loaded, all other portions are unloaded, and then the trained machine learning model is executed to process the input data 102 and produce the first output 204 a. If only the second processing task is to be performed, the common portion 110, the second task-specific portion 212b, and the third task-specific portion 212c are loaded, all other portions are unloaded, and then the trained machine learning model is executed to process the input data 102 and produce the second output 204 b. If only the third processing task is to be performed, the common portion 110, the second task-specific portion 212b, and the fourth task-specific portion 212d are loaded, all other portions are unloaded, and then the trained machine learning model is executed to process the input data 102 and generate a third output 204 c. If the first processing task and the second processing task are to be performed, the common portion 110, the first task-specific portion 212a, the second task-specific portion 212b, and the third task-specific portion 212c are loaded, all other portions are unloaded, and then the trained machine learning model is executed to process the input data 102 and produce the first output 204a and the second output 204 b. If the first processing task and the third processing task are to be performed, the common portion 110, the first task-specific portion 212a, the second task-specific portion 212b, and the fourth task-specific portion 212d are loaded, all other portions are unloaded, and then the trained machine learning model is executed to process the input data 102 and produce the first output 204a and the third output 204 c. If the second and third processing tasks are to be performed, the common portion 110, the second task-specific portion 212b, the third task-specific portion 212c, and the fourth task-specific portion 212d are loaded, all other portions are unloaded, and then the trained machine learning model is executed to process the input data 102 and produce the second output 204b and the third output 204 c. If the first, second, and third processing tasks are to be performed, the common portion 110, the first task-specific portion 212a, the second task-specific portion 212b, the third task-specific portion 212c, and the fourth task-specific portion 212d are loaded, and then the trained machine learning model is executed to process the input data 102 and generate the first, second, and third outputs 204a, 204b, and 204 c.
In some implementations, the portions of the machine learning model 106 are all executed sequentially. In other implementations, portions of the machine learning model 106 may be executed in parallel. Parallel execution may be performed using a single processor or multiple processors, and/or using a single computing device or multiple computing devices. In the present example, if the first processing task and the second processing task are active, the common portion 110 is executed and the first intermediate data 214a is generated and stored. As computing resources become available, tasks associated with each of the first processing task and the second processing task may be executed in parallel. In particular, when a first processing resource (e.g., one or more processors or processing cores) becomes available, the first processing task may continue by initiating execution of the first task-specific portion 212a using the first intermediate data 214 a. When the second processing resource becomes available, the second processing task may be continued by executing the second task-specific part 212b and the third task-specific part 212c in parallel with the execution of the first task-specific part 212 a.
Fig. 3 is a flow diagram illustrating a first example of a process 320 for performing a processing task. Process 320 may be implemented using a computing device. For example, the computing device may include a processor, a memory, and computer-interpretable instructions stored in the memory and accessible by the processor, wherein the instructions, when executed by the processor, cause the processor to perform the operations of process 320. In some implementations, the process 320 is implemented in the form of a computer readable storage device including computer interpretable program instructions which, when executed, cause the operations of the process 320.
By advancing the process to operation 325, the evaluation made in operation 324 causes one or more of the task-specific portions 112 of the trained machine learning model 106 to be executed based on executing the instructions 108. The process proceeds to operation 326 by skipping operation 325, the evaluation performed in operation 324 refrains from executing one or more of the task-specific portions 112 of the trained machine learning model 106 based on executing the instructions 108.
By advancing the process to operation 325, the evaluation made in operation 324 causes one or more of the task-specific portions 112 of the trained machine learning model 106 to be executed based on executing the instructions 108. The process proceeds to operation 326 by skipping operation 325, the evaluation performed in operation 324 refrains from executing one or more of the task-specific portions 112 of the trained machine learning model 106 based on executing the instructions 108.
In some implementations of operations 324 and 325, processing the intermediate data 114 using one or more of the task-specific portions 112 of the trained machine learning model 106 based on executing the instructions 108 includes loading only one or more of the task-specific portions 112 of the trained machine learning model 106 identified by executing the instructions 108. The identification may explicitly identify the portion of the model (e.g., by an identification code) or may implicitly identify the portion of the model (such as by identifying a processing task that requires execution of the portion of the model).
In some implementations of operations 324 and 325, processing the intermediate data 114 using one or more of the task-specific portions 112 of the trained machine learning model 106 based on executing the instructions 108 includes offloading at least a portion of the task-specific portions 112 of the trained machine learning model 106.
In operation 326, it is determined whether there are more task-specific portions 112 to be processed. If there are more task-specific portions 112 to process, then the next task-specific portion is selected in operation 327 and the process returns to operation 324. If no more task-specific parts are pending, the process proceeds to operation 328, where the output 104 is returned to the process that initiated execution of the trained machine learning model 106. The return output 104 may include, for example, a store, display, transmit, or further process output 104.
In some implementations of the process 320, one or more of the task-specific portions 112 of the trained machine learning model 106 include a first task-specific portion of the trained machine learning model 106 and a second task-specific portion of the trained machine learning model 106, and processing the intermediate data 114 using one or more of the task-specific portions of the trained machine learning model 106 based on executing the instructions 108 includes sequentially executing the first task-specific portion of the trained machine learning model 106 and the second task-specific portion of the trained machine learning model 106.
In some implementations of the process 320, one or more of the task-specific portions 112 of the trained machine learning model 106 include a first task-specific portion of the trained machine learning model 106 and a second task-specific portion of the trained machine learning model 106, and processing the intermediate data 114 using one or more of the task-specific portions 112 of the trained machine learning model 106 based on executing the instructions 108 includes executing the first task-specific portion of the trained machine learning model 106 and the second task-specific portion of the trained machine learning model 106 in parallel.
Fig. 4 is a flow diagram illustrating a process 430 for performing processing tasks. Process 430 may be implemented using a computing device. For example, the computing device may include a processor, a memory, and computer-interpretable instructions stored in the memory and accessible by the processor, wherein the instructions, when executed by the processor, cause the processor to perform the operations of process 430. In some implementations, the process 430 is implemented in the form of a computer readable storage device including computer interpretable program instructions which, when executed, cause the operations of the process 430.
In some implementations of operations 433 and 435, processing the intermediate data 114 using one or more of the task-specific portions 112 of the trained machine learning model 106 based on executing the instructions 108 includes loading only one or more of the task-specific portions 112 of the trained machine learning model 106 identified by executing the instructions 108. In operation 433, loading is performed during configuration of the trained machine learning model 106. The identification may explicitly identify the portion of the model (e.g., by an identification code) or may implicitly identify the portion of the model (such as by identifying a processing task that requires execution of the portion of the model).
In some implementations of operations 433 and 435, processing the intermediate data 114 using one or more of the task-specific portions 112 of the trained machine learning model 106 based on executing the instructions 108 includes offloading at least a portion of the task-specific portions 112 of the trained machine learning model 106. In operation 433, offloading is performed during configuration of the trained machine learning model 106.
In some implementations, configuring the trained machine learning model 106 in operation 433 may further include defining a linear execution order including one or more of the common portion 110 of the trained machine learning model 106 and the task-specific portion 112 of the trained machine learning model 106 based on the execution instructions. In such implementations, processing the input data 102 using the common portion of the trained machine learning model and processing the intermediate data 114 using one or more of the task-specific portions 112 of the trained machine learning model 106 in operation 434 are performed according to a linear execution order.
In operation 436, it is determined whether there are more task-specific portions 112 to be processed. If there are more task-specific parts 112 to process, then the next task-specific part is selected in operation 437 and the process returns to operation 435. If no more task-specific parts are pending, the process proceeds to operation 438 where the output 104 is returned to the process that initiated execution of the trained machine learning model 106. The return output 104 may include, for example, a store, display, transmit, or further process output 104.
In some implementations of the process 430, one or more of the task-specific portions 112 of the trained machine learning model 106 include a first task-specific portion of the trained machine learning model 106 and a second task-specific portion of the trained machine learning model 106, and processing the intermediate data 114 using one or more of the task-specific portions 112 of the trained machine learning model 106 based on executing the instructions 108 includes sequentially executing the first task-specific portion of the trained machine learning model 106 and the second task-specific portion of the trained machine learning model 106.
In some implementations of the process 430, one or more of the task-specific portions 112 of the trained machine learning model 106 include a first task-specific portion of the trained machine learning model 106 and a second task-specific portion of the trained machine learning model 106, and processing the intermediate data 114 using one or more of the task-specific portions 112 of the trained machine learning model 106 based on executing the instructions 108 includes executing the first task-specific portion of the trained machine learning model 106 and the second task-specific portion of the trained machine learning model 106 in parallel.
Fig. 5 is a diagram illustrating an example of a hardware configuration for a computing device that may be used to implement the systems described herein. Computing device 540 may include a processor 541, a memory 542, a storage device 543, one or more input devices 544, and one or more output devices 545. Computing device 540 may include a bus 546 or similar device to interconnect components for communication. The processor 541 is operable to execute computer program instructions and perform operations described by the computer program instructions. For example, processor 541 may be a conventional device such as a central processing unit. The memory 542 may be a volatile, high-speed, short-term information storage device such as a random access memory module. The storage device 543 may be a non-volatile information storage device such as a hard disk or solid state drive. The input devices 544 may include any type of human-machine interface, such as buttons, switches, a keyboard, a mouse, a touch screen input device, a gesture input device, or an audio input device. Output device 545 may include any type of device operable to provide an indication to a user regarding the operational status, such as a display screen or an audio output.
As described above, one aspect of the present technology is to collect and use data available from various sources for processing by neural networks at training time and at runtime. The present disclosure contemplates that, in some instances, such collected data may include personal information data that uniquely identifies or may be used to contact or locate a particular person. Such personal information data may include demographic data, location-based data, phone numbers, email addresses, twitter IDs, home addresses, data or records related to the user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), date of birth, or any other identifying or personal information.
The present disclosure recognizes that the use of such personal information data in the present technology may be useful to benefit the user. For example, personal information data may be used to train a neural network to perform a plurality of processing tasks. In addition, the present disclosure also contemplates other uses for which personal information data is beneficial to a user. For example, health and fitness data may be used to provide insight into the overall health condition of a user, or may be used as positive feedback for individuals using technology to pursue health goals.
The present disclosure contemplates that entities responsible for collecting, analyzing, disclosing, transmitting, storing, or otherwise using such personal information data will comply with established privacy policies and/or privacy practices. In particular, such entities should enforce and adhere to the use of privacy policies and practices that are recognized as meeting or exceeding industry or government requirements for maintaining privacy and security of personal information data. Such policies should be easily accessible to users and should be updated as data is collected and/or used. Personal information from the user should be collected for legitimate and legitimate uses by the entity and not shared or sold outside of these legitimate uses. Furthermore, such acquisition/sharing should be performed after receiving user informed consent. Furthermore, such entities should consider taking any necessary steps to defend and secure access to such personal information data, and to ensure that others who have access to the personal information data comply with their privacy policies and procedures. In addition, such entities may subject themselves to third party evaluations to prove compliance with widely accepted privacy policies and practices. In addition, policies and practices should be adjusted to the particular type of personal information data collected and/or accessed, and to applicable laws and standards including specific considerations of jurisdiction. For example, in the united states, the collection or acquisition of certain health data may be governed by federal and/or state laws, such as the health insurance association and accountability act (HIPAA); while other countries may have health data subject to other regulations and policies and should be treated accordingly. Therefore, different privacy practices should be maintained for different personal data types in each country.
Regardless of the foregoing, the present disclosure also contemplates embodiments in which a user selectively prevents use or access to personal information data. That is, the present disclosure contemplates that hardware elements and/or software elements may be provided to prevent or block access to such personal information data. For example, a system employing the present technology may be configured to allow a user to opt-in or opt-out of collecting personal information data at any time during or after registration service. As another example, a user may choose not to provide personal information to a service using the present technology. As another example, the user may choose to limit the length of time that personal information is maintained by a service using the present technology, or the user may simply prohibit personal information from being used by a system using the present technology. In addition to providing "opt-in" and "opt-out" options, the present disclosure contemplates providing notifications related to accessing or using personal information. For example, the user may be notified that their personal information data is to be accessed when the application is downloaded, and then be reminded again just before the personal information data is accessed by the application.
Further, it is an object of the present disclosure that personal information data should be managed and processed to minimize the risk of inadvertent or unauthorized access or use. Once the data is no longer needed, the risk can be minimized by limiting data collection and deleting data. In addition, and when applicable, including in certain health-related applications, data de-identification may be used to protect the privacy of the user. De-identification may be facilitated by removing particular identifiers (e.g., date of birth, etc.), controlling the amount or specificity of stored data (e.g., collecting location data at a city level rather than at an address level), controlling how data is stored (e.g., aggregating data among users), and/or other methods, as appropriate.
Thus, while the present disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, the present disclosure also contemplates that various embodiments may be implemented without the need to access such personal information data. That is, various embodiments of the present technology do not fail to function properly due to the lack of all or a portion of such personal information data. For example, a neural network may be trained and used based on non-personal information data or absolutely minimal personal information (e.g., content requested by a device associated with the user, other non-personal information available to a service using the present technology, or publicly available information).
Claims (18)
1. A method, comprising:
receiving input data at a trained machine learning model comprising a common portion and a task-specific portion;
receiving an execution instruction identifying one or more processing tasks to be performed;
processing the input data using the common portion of the trained machine learning model to generate intermediate data; and
processing the intermediate data using one or more of the task-specific portions of the trained machine learning model based on the execution instructions to generate one or more outputs.
2. The method of claim 1, wherein processing the intermediate data using the one or more of the task-specific portions of the trained machine learning model based on the execution instructions comprises evaluating conditional instructions for each of the task-specific portions of the trained machine learning model to determine whether to initiate execution of each of the task-specific portions of the trained machine learning model.
3. The method of claim 1, wherein processing the intermediate data using the one or more of the task-specific portions of the trained machine learning model based on the execution instructions comprises causing execution of one or more of the task-specific portions of the trained machine learning model based on the execution instructions.
4. The method of claim 1, wherein processing the intermediate data using the one or more of the task-specific portions of the trained machine learning model based on the execution instructions comprises inhibiting execution of one or more of the task-specific portions of the trained machine learning model based on the execution instructions.
5. The method of claim 1, wherein processing the intermediate data using the one or more of the task-specific portions of the trained machine learning model based on the execution instructions comprises loading only the one or more of the task-specific portions of the trained machine learning model identified by the execution instructions.
6. The method of claim 1, wherein processing the intermediate data using the one or more of the task-specific portions of the trained machine learning model based on the execution instructions comprises offloading at least a portion of the task-specific portions of the trained machine learning model.
7. The method of claim 1, further comprising:
defining, based on the execution instructions, a linear execution order that includes the common portion of the trained machine learning model and the one or more of the task-specific portions of the trained machine learning model, wherein processing the input data using the common portion of the trained machine learning model and processing the intermediate data using the one or more of the task-specific portions of the trained machine learning model are performed according to the linear execution order.
8. The method of claim 1, wherein the one or more of the task-specific portions of the trained machine learning model comprise a first task-specific portion of the trained machine learning model and a second task-specific portion of the trained machine learning model, and processing the intermediate data using the one or more of the task-specific portions of the trained machine learning model based on the execution instructions comprises sequentially executing the first task-specific portion of the trained machine learning model and the second task-specific portion of the trained machine learning model.
9. The method of claim 1, wherein the one or more of the task-specific portions of the trained machine learning model comprise a first task-specific portion of the trained machine learning model and a second task-specific portion of the trained machine learning model, and processing the intermediate data using the one or more of the task-specific portions of the trained machine learning model based on the execution instructions comprises executing the first task-specific portion of the trained machine learning model and the second task-specific portion of the trained machine learning model in parallel.
10. A system, comprising:
a memory; and
a processor configured to execute instructions stored in the memory, wherein the instructions, when executed by the processor, cause the processor to:
receiving input data at a trained machine learning model comprising a common portion and a task-specific portion;
receiving an execution instruction identifying one or more processing tasks to be performed;
processing the input data using the common portion of the trained machine learning model to generate intermediate data; and
processing the intermediate data using one or more of the task-specific portions of the trained machine learning model based on the execution instructions to generate one or more outputs.
11. The system of claim 10, wherein processing the intermediate data using the one or more of the task-specific portions of the trained machine learning model based on the execution instructions comprises evaluating conditional instructions for each of the task-specific portions of the trained machine learning model to determine whether to initiate execution of each of the task-specific portions of the trained machine learning model.
12. The system of claim 10, wherein processing the intermediate data using the one or more of the task-specific portions of the trained machine learning model based on the execution instructions comprises causing execution of one or more of the task-specific portions of the trained machine learning model based on the execution instructions.
13. The system of claim 10, wherein processing the intermediate data using the one or more of the task-specific portions of the trained machine learning model based on the execution instructions comprises inhibiting execution of one or more of the task-specific portions of the trained machine learning model based on the execution instructions.
14. The system of claim 10, wherein processing the intermediate data using the one or more of the task-specific portions of the trained machine learning model based on the execution instructions comprises loading only the one or more of the task-specific portions of the trained machine learning model identified by the execution instructions.
15. The system of claim 10, wherein processing the intermediate data using the one or more of the task-specific portions of the trained machine learning model based on the execution instructions comprises offloading at least a portion of the task-specific portions of the trained machine learning model.
16. The system of claim 10, further comprising:
defining, based on the execution instructions, a linear execution order that includes the common portion of the trained machine learning model and the one or more of the task-specific portions of the trained machine learning model, wherein processing the input data using the common portion of the trained machine learning model and processing the intermediate data using the one or more of the task-specific portions of the trained machine learning model are performed according to the linear execution order.
17. The system of claim 10, wherein the one or more of the task-specific portions of the trained machine learning model comprise a first task-specific portion of the trained machine learning model and a second task-specific portion of the trained machine learning model, and processing the intermediate data using the one or more of the task-specific portions of the trained machine learning model based on the execution instructions comprises sequentially executing the first task-specific portion of the trained machine learning model and the second task-specific portion of the trained machine learning model.
18. The system of claim 10, wherein the one or more of the task-specific portions of the trained machine learning model comprise a first task-specific portion of the trained machine learning model and a second task-specific portion of the trained machine learning model, and processing the intermediate data using the one or more of the task-specific portions of the trained machine learning model based on the execution instructions comprises executing the first task-specific portion of the trained machine learning model and the second task-specific portion of the trained machine learning model in parallel.
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US201962850618P | 2019-05-21 | 2019-05-21 | |
| US62/850,618 | 2019-05-21 | ||
| US16/878,254 US11699097B2 (en) | 2019-05-21 | 2020-05-19 | Machine learning model with conditional execution of multiple processing tasks |
| US16/878,254 | 2020-05-19 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111985637A true CN111985637A (en) | 2020-11-24 |
Family
ID=71016695
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010437644.2A Pending CN111985637A (en) | 2019-05-21 | 2020-05-21 | Machine learning model with conditional execution of multiple processing tasks |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111985637A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120239598A1 (en) * | 2011-03-15 | 2012-09-20 | Cascaval Gheorghe C | Machine Learning Method to Identify Independent Tasks for Parallel Layout in Web Browsers |
| CN105493095A (en) * | 2013-09-24 | 2016-04-13 | 迈克菲股份有限公司 | Adaptive and recursive filtering for sample submission |
| US20170132528A1 (en) * | 2015-11-06 | 2017-05-11 | Microsoft Technology Licensing, Llc | Joint model training |
| CN107273979A (en) * | 2017-06-08 | 2017-10-20 | 第四范式（北京）技术有限公司 | The method and system of machine learning prediction are performed based on service class |
| CN107660284A (en) * | 2014-05-30 | 2018-02-02 | 苹果公司 | Search based on machine learning improves |
| US20180129972A1 (en) * | 2016-11-04 | 2018-05-10 | Google Inc. | Implicit bridging of machine learning tasks |
| US20180189228A1 (en) * | 2017-01-04 | 2018-07-05 | Qualcomm Incorporated | Guided machine-learning training using a third party cloud-based system |
| WO2018140969A1 (en) * | 2017-01-30 | 2018-08-02 | Google Llc | Multi-task neural networks with task-specific paths |
| US20180336463A1 (en) * | 2017-05-18 | 2018-11-22 | General Electric Company | Systems and methods for domain-specific obscured data transport |
| CN109690576A (en) * | 2016-07-18 | 2019-04-26 | 渊慧科技有限公司 | The training machine learning model in multiple machine learning tasks |
-
2020
- 2020-05-21 CN CN202010437644.2A patent/CN111985637A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120239598A1 (en) * | 2011-03-15 | 2012-09-20 | Cascaval Gheorghe C | Machine Learning Method to Identify Independent Tasks for Parallel Layout in Web Browsers |
| CN105493095A (en) * | 2013-09-24 | 2016-04-13 | 迈克菲股份有限公司 | Adaptive and recursive filtering for sample submission |
| CN107660284A (en) * | 2014-05-30 | 2018-02-02 | 苹果公司 | Search based on machine learning improves |
| US20170132528A1 (en) * | 2015-11-06 | 2017-05-11 | Microsoft Technology Licensing, Llc | Joint model training |
| CN109690576A (en) * | 2016-07-18 | 2019-04-26 | 渊慧科技有限公司 | The training machine learning model in multiple machine learning tasks |
| US20180129972A1 (en) * | 2016-11-04 | 2018-05-10 | Google Inc. | Implicit bridging of machine learning tasks |
| US20180189228A1 (en) * | 2017-01-04 | 2018-07-05 | Qualcomm Incorporated | Guided machine-learning training using a third party cloud-based system |
| WO2018140969A1 (en) * | 2017-01-30 | 2018-08-02 | Google Llc | Multi-task neural networks with task-specific paths |
| US20180336463A1 (en) * | 2017-05-18 | 2018-11-22 | General Electric Company | Systems and methods for domain-specific obscured data transport |
| CN107273979A (en) * | 2017-06-08 | 2017-10-20 | 第四范式（北京）技术有限公司 | The method and system of machine learning prediction are performed based on service class |
Non-Patent Citations (1)
| Title |
|---|
| LANLAN LIU ETAL: "Dynamic deep neural networks：optimizing accuracy-efficiency trade-offs by selective execution", 《HTTPS://ARXIV.ORG/PDF/1701.00299.PDF》, pages 1 - 12 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Adrion et al. | Health care costs, utilization and patterns of care following Lyme disease | |
| Zhang et al. | Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making | |
| US10970641B1 (en) | Heuristic context prediction engine | |
| US10236079B2 (en) | Managing user information—authorization masking | |
| EP3547604A1 (en) | Method and apparatus for security question generation and identity verification | |
| US20150095267A1 (en) | Techniques to dynamically generate real time frequently asked questions from forum data | |
| KR102490529B1 (en) | Total periodic non-identification management apparatus and method | |
| US20190138749A1 (en) | Total periodic de-identification management apparatus and method | |
| US11631041B2 (en) | Methods and systems for creating and managing a research study and deploying via mobile and web utilizing a research module | |
| Gallo et al. | Alternative views on setting clinical trial futility criteria | |
| US11699097B2 (en) | Machine learning model with conditional execution of multiple processing tasks | |
| CN109522705B (en) | Authority management method, device, electronic equipment and medium | |
| CN113822438A (en) | Machine learning model training checkpoint | |
| US10977285B2 (en) | Using observations of a person to determine if data corresponds to the person | |
| WO2016033744A1 (en) | Screen-image based classification | |
| CN116055102A (en) | Method for updating necessary repair loopholes, method for scanning necessary repair loopholes and related equipment | |
| Wickramasinghe et al. | A survey-based exploration of users’ awareness and their willingness to protect their data with smart objects | |
| CN108062401B (en) | Application recommendation method and device and storage medium | |
| CN105718462B (en) | Cheating detection method and device for application operation | |
| Moniz et al. | Intimate Data: Exploring Perceptions of Privacy and Privacy-Seeking Behaviors Through the Story Completion Method | |
| CN109215794B (en) | Abnormal user identification method and device, storage medium and electronic equipment | |
| Stöver et al. | Investigating how users imagine their personal privacy assistant | |
| CN111985637A (en) | Machine learning model with conditional execution of multiple processing tasks | |
| US10282876B1 (en) | Tracking environmental contaminants | |
| Girdler-Brown | Evaluation of the notifiable diseases surveillance system in South Africa |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |