JP2023552048A - Neural architecture scaling for hardware acceleration - Google Patents

Abstract

A method, system, and apparatus, including a computer-readable medium, for scaling a neural network architecture on a hardware accelerator. The method includes receiving training data and information specifying target computing resources, and using the training data to perform a neural architecture search of a search space to identify a base neural network architecture. Multiple scaling parameter values may be identified for scaling the basic neural network. This identification may include iteratively selecting a plurality of candidate scaling parameter values and determining a performance evaluation metric for the scaled basic neural network in response to the plurality of candidate scaling parameter values, The evaluation index is a performance evaluation index according to a plurality of secondary objectives including a latency objective. A scaled neural network architecture may be determined using the base neural network architecture scaled according to the plurality of scaling parameter values.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Patent Application No. 63/137,926, filed January 15, 2021, under 35 U.S.C. 119(e) February 12, 2021 It is a continuation of U.S. Patent Application No. 17,175,029, filed on July 1, 1999, the disclosure of which is incorporated herein by reference.

Background Neural networks are machine learning models that include one or more layers of nonlinear operations to predict outputs given inputs received. Some neural networks include one or more hidden layers in addition to input and output layers. The output of each hidden layer may be input to another hidden layer or output layer of the neural network. Each layer of the neural network can generate a respective output from received inputs depending on the value of one or more model parameters for that layer. Model parameters can be weights or biases determined by a training algorithm to force the neural network to produce accurate outputs.

SUMMARY A system implemented in accordance with aspects of the present invention reduces the latency of neural network architectures by exploring neural network architecture candidates according to each candidate's computational requirements (e.g., FLOPS), computational intensity, and execution efficiency. Can be done. As discussed herein, whereas computational requirements alone affect latency, including inference-time latency, computational requirements, computational intensity, and execution efficiency affect neural networks on target computing resources. was found to be the root cause of latency. Aspects of the present disclosure develop neural architectures based on this observed latency-computation relationship, computational intensity, and execution efficiency through latency-aware composite scaling and by expanding the space in which to search for neural network candidates. Enabling techniques to perform exploration and scaling.

Furthermore, the system can perform compound scaling to uniformly scale multiple parameters of the neural network according to multiple objectives. This can improve the performance of scaled neural networks over approaches where one objective is considered or where the scaling parameters of the neural network are searched separately. Latency-aware compound scaling can be used to quickly build a family of neural network architectures that can be scaled according to different values from an initial scaled neural network architecture and suitable for different use cases.

According to aspects of the present disclosure, a computer-implemented method includes a method for determining the architecture of a neural network. The method includes: one or more processors receiving training data corresponding to a neural network task and information specifying target computing resources; and one or more processors receiving training data corresponding to a neural network task and information specifying target computing resources; performing a neural architecture search of the search space using the training data to identify the architecture of the base neural network; and identifying a plurality of scaling parameter values for scaling the base neural network in response to a parameter. The identifying includes iteratively selecting a plurality of scaling parameter value candidates and determining a performance evaluation metric for the scaled basic neural network in response to the plurality of scaling parameter value candidates, and determining the performance of the scaled basic neural network. The evaluation index is determined according to a plurality of secondary objectives, including a latency objective. The method may further include the one or more processors generating a scaled neural network architecture using the scaled base neural network architecture in response to the plurality of scaling parameter values.

Each of the above and other embodiments may optionally include one or more of the following features alone or in combination.

The first objectives for performing the neural architecture search may be the same as the second objectives for identifying scaling parameter values.

The plurality of first objectives and the plurality of second objectives may include an accuracy rate objective corresponding to an accuracy rate of the output of the basic neural network.

The performance metric is the difference between the base neural network accepting input and producing output when the base neural network is scaled according to multiple scaling parameter value candidates and deployed on a target computing resource. may correspond at least in part to a latency metric.

The latency objective may correspond to a minimum latency between the base neural network accepting input and producing output when the base neural network is deployed on the target computing resource.

The search space may include candidate neural network layers, with each candidate neural network layer configured to perform one or more operations. The search space may include neural network layer candidates that include different activation functions.

The basic neural network architecture may include multiple component candidates, each component having multiple neural network layers. The search space includes a first component of the network layer candidate that includes a first activation function, and a second component of the network layer candidate that includes a second activation function that is different from the first activation function. A network layer candidate may include multiple component candidates.

The information specifying target computing resources may specify one or more hardware accelerators, and the method includes executing the scaled neural network on the one or more hardware accelerators to perform the neural network task. further including.

The target computing resource may include a first target computing resource, the plurality of scaling parameter values are a plurality of first scaling parameter values, and the method includes: receiving information specifying a second target computing resource different from the resource; and a plurality of second scaling parameter values for scaling the base neural network in response to the information specifying the second target computing resource. the plurality of second scaling parameter values being different from the plurality of first scaling parameter values.

The plurality of scaling parameter values are a plurality of first scaling parameter values, and the method includes generating a scaled neural network architecture from a scaled base neural network architecture using a plurality of second scaling parameter values. Further including, a second scaling parameter value is generated in response to the plurality of first scaling parameter values and one or more composite coefficients that uniformly modify the value of each of the first scaling parameter values.

The base neural network may be a convolutional neural network, and the plurality of scaling parameters may include one or more of base neural network depth, base neural network width, and base neural network input resolution.

According to another aspect, a method for determining an architecture of a neural network includes: one or more processors receiving information specifying target computing resources; and one or more processors determining an architecture of a base neural network. and a plurality of scalings for the one or more processors to scale the base neural network in response to information specifying target computing resources and a plurality of scaling parameters for the base neural network. and identifying parameter values. The identifying includes iteratively selecting a plurality of scaling parameter value candidates and determining a performance evaluation metric for the scaled basic neural network in response to the plurality of scaling parameter value candidates, and determining the performance of the scaled basic neural network. The evaluation metric is determined according to multiple objectives, including a latency objective, and the method uses a basic neural network architecture in which one or more processors are scaled according to multiple scaling parameter values. Further comprising generating an architecture for the neural network.

The plurality of objectives may be a plurality of secondary objectives, and receiving data specifying the architecture of the underlying neural network comprises: the one or more processors receiving training data corresponding to the neural network task; The one or more processors may include performing a neural architecture search of the search space using the training data to identify an architecture of the base neural network according to the plurality of first objectives.

Other embodiments include a computer system, an apparatus, and a computer program recorded on one or more computer storage devices, each configured to perform the operations of the method.

FIG. 2 is a block diagram illustrating a family of scaled neural network architectures for deployment in a data center containing hardware accelerators on which the deployed neural networks operate. FIG. 2 is a flow diagram of an example process for generating a scaled neural network architecture for execution on target computing resources. An exemplary process for latency-aware compound scaling of basic neural network architectures. 1 is a block diagram of a NAS-LACS (Neural Architecture Search-Latency Aware Complex Scaling) system according to aspects of the present disclosure. FIG. 1 is a block diagram of an example environment for implementing a NAS-LACS system. FIG.

DETAILED DESCRIPTION Overview The technology described herein generally relates to scaling neural networks for execution on different target computing resources, such as different hardware accelerators. Neural networks can be scaled for a number of different performance objectives. These performance objectives are to minimize processing time (referred to herein as latency) and to maximize the accuracy rate of the neural network when scaled to run on the target computing resources. It can include a separate purpose: purpose.

In general, a NAS (Neural Architecture Search) system may be deployed to select a neural network architecture from a given search space comprised of architecture candidates according to one or more objectives. One common objective is the accuracy rate of neural networks. In general, systems implementing NAS technology prefer networks that have a high accuracy rate after training over networks that have a low accuracy rate. After the base neural network is selected following the NAS, the base neural network may be scaled according to one or more scaling parameters. Scaling may include searching for one or more scaling parameter values for scaling the basic neural network, for example, by searching for scaling parameters in a coefficient search space made up of numbers. Scaling may involve increasing or decreasing the number of layers a neural network has or the size of each layer to make better use of computational and/or memory resources available for deploying the neural network.

A commonly held idea regarding neural architecture exploration and scaling is that the network computations, e.g. measured in FLOPS (floating point operations per second), are required to process the inputs passed through the neural network. The idea is that the requirement is proportional to the latency between sending input to the network and accepting output. That is, it is believed that a neural network with low computational requirements (low FLOPS) will produce output faster than if the network has high computational requirements (high FLOPS). This is because fewer operations are executed overall. Therefore, many NAS systems choose neural networks because of their low computational requirements. However, it is clear that the relationship between computational requirements and latency is not proportional, as other characteristics of neural networks such as computational intensity, parallelism, and execution efficiency can affect the overall latency of neural networks. It has become.

The technology described herein enables LACS (Latency Aware Composite Scaling) and expansion of the neural network candidate search space from which neural networks are selected. In the context of extending the search space, operations and architectures may be included in the search space. The computation and architecture are ``hardware accelerator friendly'' in that they together provide higher computational intensity, execution efficiency, and parallelism suitable for deployment on various types of hardware accelerators. ” operations and architecture. Such operations include space-to-depth operations, space-to-batch operations, fused convolution structures, and per-component search activation functions. may be included.

Latency-aware composite scaling of neural networks can improve the scaling of neural networks over traditional methods that do not perform latency-aware optimizations. Alternatively, LACS may be used to identify scaled parameter values for scaled neural networks that operate accurately and with low latency on target computing resources.

This technology further enables multi-objective scaling that shares the purpose of exploring neural architectures using NAS or similar technology. The scaled neural network is distinguishable according to the same objectives used in searching the basic neural network. As a result, the scaled neural network can be optimized for performance at the basic architecture exploration and scaling stages, rather than treating each stage as tasks with separate objectives.

LACS can be integrated with existing NAS systems. This is because the same objectives can be used for both exploration and scaling, at least to build an end-to-end system for determining scaled neural network architectures. Furthermore, families of scaled neural network architectures can be identified faster than searching-without-scaling techniques. Searching-without-scaling techniques explore neural network architectures but do not scale them for deployment on target computing resources.

The technology described herein may enable neural networks that are improved over those identified with traditional search-and-scaling techniques that do not use LACS. In addition to this, families of neural networks with different trade-offs between objectives such as model accuracy and inference latency can be quickly generated to be applied to various use cases. This technology may also identify neural networks to perform a particular task faster, and the identified neural networks may have an improved accuracy rate than neural networks identified using other techniques. It can work. This means that, at a minimum, the neural networks identified as a result of performing the exploration and scaling described herein do not simply consider the computational requirements of the network, but also the computational intensity and This is to consider characteristics such as execution efficiency. In this way, the identified neural network can operate faster during inference without sacrificing the accuracy of the network.

This technology may also enable a generally applicable framework for quickly migrating existing neural networks to improved computing resource environments. For example, the LACS and NAS described herein can be applied when migrating an existing neural network implementation selected for a data center with particular hardware to a data center with different hardware. In this regard, families of neural networks to perform the tasks of existing neural networks can be quickly identified and deployed on new data center hardware. This application method can be particularly useful in areas of rapid deployment, such as when networks perform tasks in computer vision or other image processing tasks, which require state-of-the-art hardware to perform efficiently.

FIG. 1 is a block diagram of a family 103 of scaled neural network architectures 104A-104N for deployment in a data center 115 in which a hardware accelerator 116 is housed. The deployed neural network operates on the hardware accelerator 116. Hardware accelerator 116 may be any type of processor such as a CPU, GPU, FGPA, or an ASIC such as a TPU. According to aspects of the present disclosure, family of scaled neural network architectures 103 may be generated from base neural network architecture 101.

Neural network architecture refers to the characteristics that identify a neural network. For example, the architecture may include the characteristics of the different neural network layers that make up a network, how these layers process input, how the layers interact with each other, and so on. For example, the ConvNet (convolutional neural network) architecture defines a discrete convolutional layer that receives input image data, then a pooling layer, and then a neural network that classifies the content of the input image data. Fully connected layers can be defined that produce outputs according to network tasks. The architecture of a neural network can also define the types of operations performed within each layer. For example, the ConvNet architecture may specify the use of ReLU activation functions in the fully connected layer of the network.

Using the NAS, basic neural network architectures 101 can be identified according to a set of objectives. From a search space composed of neural network architecture candidates, basic neural network architectures 101 can be identified using the NAS according to a desired set. As described in further detail herein, the search space comprised of candidate neural network architectures is expanded to include different network components, different operations, and different layers from which a basic network satisfying the objective can be identified. can.

The set of objectives used to identify the base neural network architecture 101 may also be applied to identify scaling parameter values for each neural network 104A-104N in the family 103. Basic neural network architecture 101 and scaled neural network architectures 104A-104N may be characterized by a number of parameters. These parameters are scaled to varying degrees in scaled neural network architectures 104A-104N. In FIG. 1, the neural networks 101, 104A are shown in FIG. It is shown to have three scaling parameters: R, which indicates the size of the input being processed.

As described in further detail herein, a system configured to perform LACS can search coefficient search space 108 to identify multiple sets of scaling parameter values. Each scaling parameter value is a coefficient in a coefficient search space, and may be, for example, a set of positive real numbers. Each network candidate 107A-107N is scaled from base neural network 101 according to the coefficient value candidates identified as part of the search in coefficient search space 108. The system can apply any of a variety of search techniques to identify coefficient candidates, such as Pareto frontier search or grid search. For each network candidate 107A-107N, the system can evaluate the performance metrics of that network candidate in performing the neural network task. The performance metrics may be based on multiple objectives, including a latency objective of measuring the latency of the network candidate between accepting input and producing a corresponding output as part of executing the neural network task. .

After the scaling parameter value search is performed on the coefficient search space 108, the system may accept a scaled neural network architecture 109. Scaled neural network architecture 109 is scaled from base neural network 101 using scaling parameter values that result in the maximum performance metrics of network candidates 107A-107N identified during the search in coefficient search space.

From the scaled neural network architecture 109, the system can generate a family 103 of scaled neural network architectures 104A-104N. Families 103 can be generated by scaling each scaled neural network architecture 109 according to a different value. Each scaling parameter value of scaled neural network architecture 109 can be uniformly scaled to generate other scaled neural network architectures in family 103. For example, each scaling parameter value of scaled neural network architecture 109 can be scaled by increasing each scaling parameter value by a factor of two. The scaled neural network architecture 109 can be scaled by different values--or "compound coefficients"--that are uniformly applied to each scaling parameter value of the scaled neural network architecture 109. In some implementations, scaled neural network architecture 109 is scaled in another manner, such as by scaling each scaling parameter value separately, to produce the scaled neural network architectures in family 103. do.

By scaling the scaled neural network architectures 109 according to different values, different neural network architectures can be quickly generated to perform tasks according to different use cases. The different use cases may be specified as different trade-offs between objectives used to identify the scaled neural network architecture 109. For example, certain scaled neural network architectures may be identified as meeting higher accuracy thresholds at the cost of greater latency during execution. Alternative scaled neural network architectures may be identified that meet lower accuracy rate thresholds, but can be run with lower latency on the hardware accelerator 116. Another scaled neural network architecture may be identified that balances the trade-off between accuracy and latency on the hardware accelerator 116.

As an example, to perform a computer vision task such as object recognition, an application may sequentially accept video or image data as well as a neural network as part of the task of identifying a particular class of objects in the received data. The architecture will need to generate output in real time or near real time. For this example task, the accuracy rate tolerance may be low, so a scaled neural network architecture with an appropriate tradeoff for low latency and low accuracy rate may be deployed to perform the task.

As another example, a neural network architecture may be tasked with classifying all objects in a scene received from image or video data. In this example, if the latency in performing this illustrative task is not considered as important as performing said task with high accuracy, then a scaled neural network with a high accuracy rate at the expense of latency. A network may be deployed. In other examples, scaled neural network architectures may be deployed that balance tradeoffs between accuracy rate, latency, and other objectives. Here, no particular trade-offs are identified or desired when performing neural network tasks.

Scaled neural network architecture 104N is scaled using scaling parameter values different from the scaling parameter values used to scale scaled neural network architecture 109 to obtain scaled neural network 104A; It may represent a different use case, for example a use case where accuracy rate is desired over latency during inference.

The LACS and NAS technologies described herein generate a family 103 of hardware accelerators 116 that may receive additional training data and information specifying multiple different computing resources, such as different types of hardware accelerators. . In addition to generating family 103 of hardware accelerators 116, the system may explore basic neural network architectures and generate families of scaled neural network architectures for other hardware accelerators. For example, for GPUs and TPUs, the system may generate separate model families optimized for accuracy rate and latency trade-offs for GPUs and TPUs, respectively. In some implementations, the system may generate multiple scaled families from the same basic neural network architecture.

Exemplary Method FIG. 2 is a flow diagram of an exemplary process 200 for generating a scaled neural network architecture for execution on target computing resources. Exemplary process 200 may be executed on a system comprised of one or more processors at one or more locations. For example, the NAS-LACS (Neural Architecture Exploration-Latency Aware Composite Scaling) system described herein may perform process 200.

As shown at block 210, the system receives training data corresponding to a neural network task. Neural network tasks are machine learning tasks that can be performed by neural networks. A scaled neural network may be configured to accept any type of data input and produce output to perform a neural network task. By way of example, the output may be any type of score, classification, or regression output based on the input. Correspondingly, the neural network task may be a scoring task, a classification task, and/or a regression task to predict an output given input. These tasks may correspond to various applications in processing images, video, text, audio, or other types of data.

The received training data may be in any format suitable for training a neural network, depending on one of a variety of learning techniques. Learning techniques for training neural networks may include supervised learning techniques, unsupervised learning techniques, and semi-supervised learning techniques. For example, the training data may include multiple training examples that the neural network may accept as input. Training examples may be labeled with known outputs that correspond to the outputs that are supposed to be produced by a properly trained neural network to perform a particular neural network task. For example, if the neural network task is a classification task, the training example may be an image labeled with one or more classes that classify the objects depicted in the image.

As shown at block 220, the system receives information specifying target computing resources. The target computing resource data may specify characteristics of the computing resource on which at least a portion of the neural network may be deployed. Computing resources may be housed in one or more data centers or other physical locations hosting various types of hardware devices. Types of hardware include CPUs (Central Processing Units), GPUs (Graphics Processing Units), edge or mobile computing devices, FGPAs (Field Programmable Gate Arrays), and various types of ASICs ( Application-Specific Circuit) Examples include.

Some devices may be configured for hardware acceleration and may include devices configured to efficiently perform certain types of operations. These hardware accelerators, including for example GPUs and TPUs (Tensor Processing Units), may perform specialized functions of hardware acceleration. Examples of hardware acceleration functions include configurations for executing operations commonly associated with the execution of machine learning models, such as matrix multiplication. Also, by way of example, these special functions may include matrix multiply-accumulate units available in different types of GPUs, and matrix multiply units available in TPUs.

Target computing resource data may include data about one or more target computing resource sets. The target computing resource set refers to the collection of computing devices on which you want to deploy your neural network. Information specifying a target set of computing resources may refer to the type and amount of hardware accelerators or other computing devices included in the target set. A target set may include devices of the same type or different types. For example, the target computing resource set may define hardware characteristics and quantities of particular types of hardware accelerators, including processing power, throughput, and memory capacity. As described herein, the system may generate a family of scaled neural network architectures for each device specified in the target computing resource set.

In addition, the target computing resource data may specify different sets of target computing resources, eg, reflecting different possible configurations of computing resources housed in the data center. From this training and target computing resource data, the system may generate a family of neural network architectures. Each architecture may be generated from the basic neural networks identified by the system.

As shown at block 230, the system may use the training data to perform a neural architecture search on the search space to identify the architecture of the base neural network. The system may use any of a variety of NAS techniques, such as techniques based on reinforcement learning, evolutionary search, or differentiable search. In some implementations, the system may directly accept data specifying the architecture of the basic neural network, without, for example, accepting training data and running the NAS as described herein. For example, an initial candidate neural network architecture can be predetermined based on an analysis of the types of neural network architectures that are suitable for performing a particular neural network task. As another example, an initial neural network architecture candidate can be randomly selected from a search space comprised of neural network candidates. As another example, characteristics of at least some of the initial neural network candidates can be selected based on similar characteristics of previously trained neural networks, such as, for example, similar or the same number of layers, weight values, etc.

Search space refers to neural network candidates or portions of neural network candidates that may be selected as part of the basic neural network architecture. Some of the candidate neural network architectures may refer to components of a neural network. The architecture of a neural network may be defined according to multiple components of the neural network. In a neural network, each component includes one or more neural network layers. Characteristics of the neural network layers can be specified in a component-level architecture, which means that the architecture can specify specific operations on the component, such that each neural network in the component Perform the same operation. Also, a component may be defined in the architecture by the number of layers it has.

As part of performing NAS, the system identifies neural network candidates, obtains performance metrics that address multiple objectives, and evaluates neural network candidates according to each of these performance metrics. , may be executed repeatedly. As part of obtaining performance metrics, such as accuracy rate and latency metrics, for the neural network candidate, the system may train the neural network candidate using the received training data. Once trained, the system may evaluate candidate neural network architectures, determine their performance metrics, and compare these performance metrics according to the currently best candidate.

The system may iteratively perform this search process by selecting neural network candidates, training the networks, and comparing their performance metrics until a stopping metric is reached. The outage metric may be a predetermined minimum threshold of performance that the current network candidate meets. Additionally or alternatively, the stopping metric may be a maximum number of search iterations or a maximum period of time allocated for performing the search. The stopping metric may be a condition under which the performance of the neural network converges, for example, the performance of a subsequent iteration is below a different threshold than the performance of a previous iteration.

In the context of optimizing various performance metrics of a neural network, such as accuracy rate and latency, the stopping metric may specify a predetermined threshold range that is "optimal." For example, the optimal latency threshold range may be a threshold range from a theoretical or measured minimum latency that the target computing resource achieves. The theoretical or measured minimum latency may be based on physical characteristics of the computing resource, such as the minimum amount of time required for a component of the computing resource to be physically capable of reading and processing received data. In some implementations, the latency is maintained as a minimum value, eg, as close to zero delay as physically possible, and is not based on a target latency measured or calculated from the target computing resource.

The system may be configured to use machine learning models or other techniques to select the next candidate neural network architecture. Here, the selection may be based at least in part on learned characteristics of different neural network candidates that are likely to perform well given the objectives of the particular neural network task.

In some examples, the system may use a multi-objective reward mechanism to identify the basic neural network architecture as follows.

To measure the accuracy rate of a neural network candidate, the system may use a training set to train the neural network candidate to perform a neural network task. The system may split the training data into a training set and a validation set, for example, by an 80/20 split. For example, the system may apply supervised learning techniques to calculate the error between the output produced by the candidate neural network and the ground truth labels of the training examples processed by the network. The system can utilize any variety of loss or error functions appropriate to the type of task the neural network is being trained on, such as cross-entropy error for classification tasks and mean squared error for regression tasks. The gradient of the error when changing the weights of the neural network candidates may be calculated using, for example, a backpropagation algorithm, and the weights of the neural network may be updated. The system may be trained to train the neural network candidate until a stopping metric is met, such as when a minimum threshold of number of training iterations, maximum duration, convergence, or accuracy rate is met.

In addition to other performance metrics, including (1) the computational intensity of the base neural network candidate when deployed on the target computing resource, and/or the execution efficiency of the base neural network candidate on the target computing resource, the system , may generate accuracy rate and latency performance metrics for the candidate neural network architecture on the target computing resource. In some implementations, in addition to accuracy rate and latency, performance metrics for base neural network candidates are based at least in part on computational intensity and/or execution efficiency.

Latency, computational intensity, and execution efficiency may be defined as follows.

The system explores basic neural network architectures by simultaneously exploring multiple neural network architecture candidates with improved computational intensity, execution efficiency, and computational requirements, rather than searching only for networks with fewer computations. , the latency of the final neural network can be improved. The system can be configured to operate in this manner to reduce the overall latency of the final underlying neural network architecture.

In addition to this, especially when the target computing resource is a data center hardware accelerator, the system expands the architecture candidate search space from which to select a base neural network architecture to improve accuracy with less inference latency on the target computing resource. It is possible to expand the variety of available neural network candidates that are likely to perform well.

Expanding the search space as described can increase the number of neural network architecture candidates that are more suitable for data center accelerator deployments, resulting in a search space that is not expanded in accordance with aspects of this disclosure. Be able to identify basic neural network architectures that otherwise would not have been possible. In embodiments where the target computing resources specify hardware accelerators such as GPUs and TPUs, candidate architectures or portions of the architecture, such as components or operations that improve computational strength, parallelism, and/or execution efficiency, can be used to expand the search space.

In one exemplary expansion method, the search space can be expanded to include neural network architecture components having layers that implement one of various types of activation functions. It has been found that for TPUs and GPUs, activation functions such as ReLU or swish typically have low computational intensity and are typically memory constrained on these types of hardware accelerators. Executing activation functions in neural networks is generally constrained by the amount of memory available on the target computing resources, so executing these functions has a huge impact on the inference speed performance of the end-to-end network. may have a negative impact.

One exemplary extension of the search space relative to the activation function is to introduce an activation function fused with an associated discrete convolution into the search space. Activation functions are generally element-by-element operations and operate on a hardware accelerator configured for vector operations, so these activation functions can be performed in parallel with discrete convolution. Discrete convolution is a matrix-based operation that typically operates on matrix units of hardware accelerators. These fused activation function-convolution operations can be selected as neural network component candidates by the system as part of the basic neural network architecture search described herein. Any of a variety of activation functions can be utilized, including Swish, Rectified Linear Unit (ReLU), Sigmoid, Tanh, and Softmax.

Fusion Activation Functions - Various components can be added to the search space, including layers of convolution operations, and the components can vary depending on the type of activation function used. For example, one component of the activation function-convolution layer may include a ReLU activation function, while another component may include a Swish activation function. Since different hardware accelerators can operate more efficiently using different activation functions, expanding the search space to include a fused activation function-convolution of multiple types of activation functions can improve the target neural network task. We found that we can further improve our ability to identify the most suitable basic neural network architecture to perform.

In addition to the components with various activation functions described herein, other fused convolutional structures can be used to expand the search space, and convolutions of different shapes, types, and sizes can be used to expand the search space. You can also make it even richer. Different convolutional structures may be components added as part of a candidate neural network architecture, such as an enhancement layer consisting of 1×1 convolutions, a depth-wise convolution, a projection layer consisting of 1×1 convolutions, and other operations such as activation functions, batch normalization functions, and/or skip connections.

Therefore, the search space of the NAS can be expanded in other ways to include operations that can take advantage of the parallelism available on hardware accelerators. The search space may include one or more operations to fuse the depth-wise convolution with adjacent 1×1 convolutions and operations to shape the input of the neural network. For example, the input to a neural network candidate may be a tensor. A tensor is a data structure that can represent multiple values according to different ranks. For example, a first order tensor may be a vector, a second order tensor may be a matrix, a third order matrix may be a three-dimensional matrix, etc. There may be merit in fusing depth-wise convolutions. This is because depth unit operations are generally low-integration operations, and by merging this operation with adjacent convolutions, the operation intensity can be increased to near the maximum capacity of the hardware accelerator. It's for a reason.

The search space may also include operations that shape the input tensor by moving elements of the tensor to various locations in memory on the target computing resource. Additionally or alternatively, operations may duplicate elements at various locations in memory.

In some implementations, the system is configured to receive the base neural network for scaling directly and not perform a NAS or other search to identify the base neural network. In some implementations, multiple devices individually perform the process 200 by identifying a base neural network on one device and scaling the base neural network on another device as described herein. Do at least some of it.

As shown in block 240 of FIG. 2, the system may identify multiple scaling parameter values for scaling the base neural network in response to information specifying target computing resources and the multiple scaling parameters. The system can use the latency-aware composite scaling described here in order to use the accuracy rate and latency of the scaled neural network candidate for the purpose of searching the scaling parameter value of the basic neural network. For example, the system may apply a latency-aware composite scaling process 300 and identify multiple scaling parameter values in step 240 of FIG.

Generally, scaling techniques are applied in conjunction with NAS to identify neural networks to be scaled for deployment on target computing resources. Model scaling can be used in conjunction with NAS to more efficiently explore families of neural networks that support a variety of use cases. Under scaling techniques, different techniques can be used to explore different values for scaling parameters such as depth, width, and resolution of the neural network. Scaling can be performed by searching for values for each scaling parameter separately or by searching for a uniform set of values to adjust multiple scaling parameters together. The former is sometimes referred to as simple scaling, and the latter is sometimes referred to as complex scaling.

Scaling techniques that use accuracy as the sole objective do not allow neural networks to be scaled with adequate consideration of performance/speed implications when deployed on specialized hardware such as data center accelerators. As explained in further detail in FIG. 3, LACS may use both accuracy and latency objectives, which may be shared as the same objective used to identify the basic neural network architecture.

FIG. 3 is an example process 300 for latency-aware compound scaling of basic neural network architectures. Exemplary process 300 may be executed on a system or device comprised of one or more processors at one or more locations. For example, process 300 may be performed on a NAS-LACS system described herein. For example, the system may perform process 300 as part of step 240 to identify multiple scaling parameter values for scaling the base neural network.

In the composite scaling method, scaling coefficients for each scaling parameter are searched for at the same time. The system can apply any of a variety of search techniques to identify tuples of coefficients, such as Pareto frontier search or grid search. The system may search for tuples of coefficients according to the same objectives used to identify the basic neural network architecture described herein with reference to FIG. In addition to this, multiple objectives can include both accuracy rate and latency, and can be expressed as:

As part of determining the performance metrics, the system may use the received training data to further train and tune the scaled neural network architecture candidate. The system may determine a performance metric for the trained scaled neural network and, in accordance with block 330, determine whether the performance metric meets a performance threshold. A performance metric and a performance threshold may be a composite of multiple performance metrics and multiple performance thresholds, respectively. For example, the system may determine one performance metric from both the scaled neural network's accuracy rate and inference latency metrics, or it may determine separate performance metrics for different purposes and associate each metric with a corresponding performance Can be compared to a threshold.

If the performance metrics meet the performance threshold, process 300 ends. Otherwise, the process continues and the system selects a new plurality of potential scaling parameter values according to block 310. For example, the system may select new tuples comprised of scaling coefficients from the coefficient search space based at least in part on previously selected tuple candidates and their corresponding performance metrics. In some implementations, the system searches multiple tuples of coefficients and performs a finer search depending on multiple objectives in the vicinity of each candidate tuple.

The system may implement any of a variety of techniques to iteratively explore the coefficient candidate space, such as using grid search, reinforcement learning, evolutionary search, for example. As mentioned above, the system may continue to explore scaling parameter values until a stopping metric is reached, such as convergence or number of iterations.

In some implementations, the system may be adjusted in response to one or more controller parameter values. Controller parameter values may be adjusted manually, learned by machine learning techniques, or a combination thereof. Controller parameters may affect the relative effect of each objective on the overall performance metrics of the tuple candidates. In some examples, particular values or particular relationships between values in the candidate tuples are favored or It may not be liked.

According to block 340, the system generates one or more scaling parameter value groups from the selected scaling parameter value candidates according to one or more objective tradeoffs. Objective trade-offs may represent different thresholds for each objective, such as accuracy rate and latency, and may be satisfied by various scaled neural networks. For example, one objective tradeoff is to have a high threshold for network accuracy, but a low threshold for inference latency (ie, high accuracy networks with high latency). As another example, the objective tradeoff is to have a low threshold for network accuracy, but a high threshold for inference latency (i.e., a network with low accuracy and low latency). As another example, an objective tradeoff is a balance between accuracy rate and latency performance.

For each objective tradeoff, the system may identify a group of scaling parameter values that the system can use to scale the base neural network architecture to meet the objective tradeoff. That is, the system may iterate the selection shown in block 310, the determination of a performance metric shown in block 320, and the determination of whether the performance metric meets a performance threshold shown in block 330. is determined by objective trade-offs). In some implementations, rather than exploring tuples of basic neural network architectures, the system scales according to the selected scaling parameter value candidate that first satisfies multiple objective performance metrics according to block 330. Candidate scaling factor tuples of the basic neural network architecture can be searched.

Returning to FIG. 2, as shown at block 250, the NAS-LACS system uses one or more architectures of the scaled neural network using the architecture of the scaled base neural network according to the plurality of scaling parameter values. can be generated. A scaled neural network architecture may be a family of neural networks generated from a base neural network architecture and various scaling parameter values.

If the information specifying a target computing resource includes one or more sets of multiple target computing resources, e.g., multiple hardware accelerators of various types, the system specifies a process for each hardware accelerator. 200 and process 300 may be repeated to generate scaled neural network architectures, each corresponding to a target set. For each hardware accelerator, the system determines the relationship between latency and accuracy rate, or between latency and accuracy rate and other objectives (including, inter alia, computational intensity and execution efficiency as discussed with reference to (3)). A family of scaled neural network architectures may be generated depending on various objective trade-offs between

In some implementations, the system may generate multiple families of scaled neural network architectures from the same basic neural network architecture. This approach may be useful in situations where various target devices share similar hardware characteristics, allowing faster identification of the corresponding scaled family for each device. This is because at least the search for the basic neural network architecture, such as the one shown in the process of FIG. 2, is performed only once.

Exemplary System FIG. 4 is a block diagram of a NAS-LACS (Neural Architecture Discovery-Latency Aware Composite Scaling) system 400, in accordance with aspects of the present disclosure. System 400 is configured to receive training data 401 for performing a neural network task and target computing resource data 402 specifying a target computing resource. As described herein with reference to FIGS. 1-3, system 400 may be configured to implement techniques for generating a family of scaled neural network architectures.

System 400 may be configured to receive input data in response to a user interface. For example, system 400 may receive data as part of a call to an API (application program interface) that exposes system 400. As described herein with reference to FIG. 5, system 400 can be implemented on one or more computing devices. For example, input to system 400 may be made through a storage medium, including remote storage, connected to one or more computing devices in a network, or through a user interface on a client computing device coupled to system 400. can be done.

System 400 may be configured to output a scaled neural network architecture 409, such as a family of scaled neural network architectures. The scaled neural network architecture 409 may be transmitted as an output for display on a user display, for example, and optionally visualized according to the shape and size of each neural network layer defined in the architecture. In some implementations, system 400 may be configured to provide scaled neural network architecture 409 as a set of computer readable instructions, such as one or more computer programs. The computer readable instruction set may be executed by target computing resources to implement scaled neural network architecture 409.

Computer programs may be written in any type of programming language according to any programming paradigm, such as declarative, procedural, assembly, object-oriented, data-oriented, functional, or imperative. A computer program may be written to perform one or more different functions and to operate within a computing environment, such as on a physical device, on a virtual machine, or across multiple devices. A computer program also performs the functions described herein, such as those performed by a system, engine, module, or model.

In some implementations, system 400 is configured to convert the architecture into an executable program written in a computer programming language (optionally as part of a framework for generating machine learning models). The scaled neural network architecture 409 is configured to transfer data for the scaled neural network architecture 409 to one or more other configured devices. System 400 may also be configured to send data corresponding to scaled neural network architecture 409 to a storage device for storage and later retrieval.

System 400 may include a NAS engine 405. NAS engine 405 and other components of system 400 may be implemented as one or more computer programs, specially configured electronic circuits, or any combination thereof. NAS engine 405 may be configured to receive training data 401 and target computing resource data 402 and generate basic neural network architecture 407. Basic neural network architecture 407 may be sent to LACS engine 415. NAS engine 405 may implement any of the various techniques of neural architecture exploration described herein with reference to FIGS. 1-3. A system may be configured in accordance with aspects of the present disclosure to perform NAS with multiple objectives, including inference latency and accuracy rate of neural network candidates when executed on target computing resources. As part of determining performance metrics that the NAS engine 405 can utilize to explore the underlying neural network architecture, the system 400 may include a performance measurement engine 410.

Performance measurement engine 410 may be configured to receive the base neural network candidate architecture and generate performance metrics in response to the purpose used by NAS engine 405 to perform the NAS. Performance metrics may provide an overall performance evaluation of a neural network candidate for multiple purposes. In order to determine the accuracy rate of the basic neural network candidate, the performance measurement engine 410 obtains a training example set for verification by leaving a part of the training data 401, and performs a test on the verification set. Basic neural network candidates can be implemented.

To measure latency, performance measurement engine 410 may communicate with a computing resource corresponding to the target computing resource specified by data 402. For example, if target computing resource data 402 specifies a TPU as the target resource, performance measurement engine 410 may send the base neural network candidate to run on the corresponding TPU. The TPU may be housed in a data center in communication (e.g., in a network described in further detail with reference to FIG. 5) with one or more processors implementing system 400.

Performance measurement engine 410 may receive latency information indicating the latency between a target computing resource accepting input and producing output. Latency information may be measured locally on the target computing resource directly and sent to performance measurement engine 410, or may be measured by performance measurement engine 410 itself. If performance measurement engine 410 measures latency, engine 410 may be configured to compensate for latency that is not due to processing of the underlying neural network candidate, e.g., network latency for communicating with the target computing resource. As another example, performance measurement engine 410 may estimate the latency of processing the input through the base neural network candidate based on previous measurements of the target computing resource and hardware characteristics of the target computing resource. .

Performance measurement engine 410 may generate performance metrics for other characteristics of the candidate neural network architecture, such as computational intensity and execution efficiency. As described herein with reference to FIGS. 1-3, inference latency may be determined as a function of FLOPS (computational requirements), execution efficiency, and computational intensity, and in some implementations, the system 400 explores and scales neural networks directly or indirectly based on these additional properties.

Once the performance metrics are generated, performance measurement engine 410 may send the metrics to NAS engine 405. NAS engine 405 may then iterate new searches for new base neural network architecture candidates until a stopping metric is reached, as described herein with reference to FIG. 2.

In some examples, the NAS engine 405 is adjusted in response to one or more controller parameters to adjust how the NAS engine 405 selects the next base neural network architecture candidate. Controller parameters can be adjusted manually depending on the desired characteristics of the neural network for a particular neural network task. In some examples, the controller parameters can be learned by various machine learning techniques, and the NAS engine 405 can be one trained to select the base neural network architecture according to multiple objectives, such as latency and accuracy rate. The above machine learning models can be implemented. For example, the NAS engine 405 implements a recurrent neural network that is trained to use features and multiple objectives of previous base neural network candidates to select base network candidates that are more likely to meet those objectives. Predictable. Develop a neural network using a training data set associated with the neural network task, and the training data and performance metrics labeled to indicate the final base neural architecture selected given the target computing resource data. Can be trained.

LACS engine 415 may be configured to perform latency-aware composite scaling as described in accordance with aspects of this disclosure. LACS engine 415 is configured to receive data 407 from NAS engine 405 specifying a basic neural network architecture. Similar to NAS engine 405, LACS engine 415 may communicate with performance measurement engine 410 to obtain performance metrics for the scaled neural network architecture candidate. As described herein with reference to FIGS. 1-3, LACS engine 415 may maintain a search space in memory of different tuples composed of scaling factors and It may be configured to scale to quickly obtain a family of scaled neural network architectures. In some implementations, LACS engine 415 is configured to perform other forms of scaling, such as simple scaling, but uses multiple purposes, including the latency used by NAS engine 405.

FIG. 5 is a block diagram of an example environment 500 for implementing NAS-LACS system 400. System 400 may be implemented on one or more devices having one or more processors in one or more locations, such as within a server computing device 515. Client computing device 512 and server computing device 515 may be communicatively coupled to one or more storage devices 530 in network 560. Storage device(s) 530 may be a combination of volatile and non-volatile memory and may be in the same physical location as the computing devices 512, 515 or in a different physical location. good. For example, storage device(s) 530 may store information such as hard drives, solid state drives, tape drives, optical storage devices, memory cards, ROMs, RAM, DVDs, CD-ROMs, writable memory, and read-only memory. It may include any type of non-transitory computer-readable medium capable of being stored.

Server computing device 515 may include one or more processors 513 and memory 514. Memory 514 may store information that may be accessed by processor(s) 513, including instructions 521 that may be executed by processor(s) 513. Memory 514 may also include data 523 that may be retrieved, manipulated, and stored by processor(s) 513 . Memory 514 can be any type of non-transitory computer-readable medium capable of storing information that can be accessed by processor(s) 513, such as volatile memory and non-volatile memory. The processor(s) 513 includes one or more CPUs (Central Processing Units), one or more GPUs (Graphic Processing Units), one or more FGPAs (Field Programmable Gate Arrays), and/or T. PU (Tensor Processing) may include one or more ASICs (Application Specific Integrated Circuits), such as

Instructions 521 may include one or more instructions. The one or more instructions, when executed by processor(s) 513, cause the one or more processors to perform the operations specified by the instructions. Instructions 521 may be in the form of object code that is processed directly by processor(s) 513 or may include an interpretable script or collection of independent source code modules that are interpreted or precompiled on demand. , and can be stored in other formats. Instructions 521 may include instructions for implementing system 400 consistent with aspects of this disclosure. System 400 may be executed using processor(s) 513 and/or other processors located remotely from server computing device 515.

Data 523 may be retrieved, stored, or modified by processor(s) 513 according to instructions 521. Data 523 can be stored in computer registers, in a relational or non-relational database as a table with multiple different fields and records, or as a JSON, YAML, proto, or XML document. Additionally, data 523 may be formatted in a computer readable format, such as, but not limited to, binary values, ASCII, or Unicode. Data 523 also includes information sufficient to identify the relevant information, such as numbers, descriptive text, proprietary codes, pointers, references to data stored in other memory, such as other network locations; Or, it may contain information that the function uses to calculate relevant data.

Client computing device 512 may also be configured with one or more processors 516, memory 517, instructions 518, and data 519, similar to server computing device 515. Client computing device 512 may also include user outputs 526 and user inputs 524. User input 524 may include any suitable mechanism or technology for receiving input from a user, such as a keyboard, mouse, mechanical actuator, soft actuator, touch screen, microphone, and sensor.

Server computing device 515 may be configured to transmit data to client computing device 512, and client computing device 512 may transmit at least a portion of the received data to a display implemented as part of user output 526. It can be configured to display on. User output 526 may also be used to display the interface between client computing device 512 and server computing device 515. Alternatively or additionally, user output 526 may include one or more speakers, transducers, or other audio outputs that provide non-visual and non-audible information to a platform user of client computing device 512. may include a haptic interface or other tactile feedback.

Although processors 513, 516 and memories 514, 517 are illustrated in FIG. 5 as being within computing devices 515, 512, the components described herein including processors 513, 516 and memories 514, 517 are , may include multiple processors and multiple memories, each of which can operate in different physical locations, and which are not within the same computing device. For example, some of the instructions 521, 518 and data 523, 519 may be stored on a removable SD card, and the remainder may be stored within a read-only computer chip. Some or all of the instructions and data may be stored in a location that is physically remote from the processors 513, 516 but accessible to the processors 513, 516. Similarly, processors 513, 516 may include a collection of processors that can operate simultaneously and/or sequentially. Computing devices 515, 512 may each include one or more internal clocks that provide timing information. Timing information may be used to time operations and programs executed by computing devices 515, 512.

Server computing device 515 may be connected by network 560 to data center 550 where hardware accelerators 551A-551N are housed. Data center 550 may be one of a plurality of data centers or other facilities in which various types of computing devices are located, such as hardware accelerators. As described herein, computing resources housed in data center 550 may be designated as part of the target computing resources for deploying a scaled neural network architecture.

Server computing device 515 may be configured to accept requests to process data from client computing devices 512 on computing resources located at data center 550. For example, environment 500 may be part of a computing platform configured to provide various services to users through various user interfaces and/or APIs exposing platform services. The one or more services may be a machine learning framework or a toolset for generating neural networks or other machine learning models in response to specified tasks and training data. Client computing device 512 may receive and transmit data specifying target computing resources to be allocated to execute a neural network trained to perform a particular neural network task. According to aspects of the disclosure described herein with reference to FIGS. 1-4, the NAS-LACS system 400 receives data specifying target computing resources and training data, and in response: A family of scaled neural network architectures may be generated for deployment on target computing resources.

As other examples of services that a platform implementing environment 500 may provide, server computing device 515 may be configured to perform various scaling according to various target computing resources that may be available in data center 550. can hold a family of neural network architectures. For example, server computing device 515 may maintain different families for deploying neural networks on different types of TPUs and/or GPUs housed in data center 550 and otherwise used for processing. can hold various families.

Devices 512, 515 and data center 550 can communicate directly or indirectly over network 560. For example, using network sockets, client computing device 512 can connect to services operating at data center 550 through Internet protocols. Devices 515, 512 can set up listening sockets that can accept initiated connections to send and receive information. Network 560 itself may include a variety of configurations and protocols, including the Internet, the World Wide Web, intranets, virtual private networks, wide area networks, local networks, and private networks using communication protocols proprietary to one or more companies. obtain. Network 560 can support a variety of short-range and long-range connections. Short-range and long-range connections are 2.402 GHz to 2.480 GHz (commonly mapped to the Bluetooth® standard), 2.4 GHz and 5 GHz (commonly associated with Wi-Fi® communication protocols) or using different communication standards, such as the LTE standard for wireless broadband communications. Additionally or alternatively, network 560 supports wired connections between devices 512, 515 and data center 550, including wired connections over various types of Ethernet connections. can.

Although one server computing device 515, one client computing device 512, and one data center 550 are shown in FIG. 5, aspects of the present disclosure may be implemented in paradigms for sequential or parallel processing. It should be appreciated that implementations may be implemented on a variety of configurations and quantities of computing devices, such as on a distributed network of multiple devices, or on a distributed network of multiple devices. In some implementations, aspects of the present disclosure may be performed on one device connected to multiple hardware accelerators configured to process neural networks, or any combination thereof. .

Exemplary Use Cases As described herein, aspects of the present disclosure enable the generation of scaled neural network architectures from basic neural networks in a multi-objective manner. An example of a neural network task is as follows.

By way of example, the input to the neural network can be in the form of images, videos. Neural networks can be configured to extract, identify, and generate features as part of processing a given input, such as as part of a computer vision task. A neural network trained to perform this type of neural network task can be trained to generate one output class classification from a set of different possible class classifications. Additionally or alternatively, a neural network can be trained to output a score corresponding to an estimate that an object identified in an image or video is likely to belong to a particular class.

As another example, the input to a neural network may be a data file corresponding to a particular format, for example, a format obtained from an HTML file, a word processing document, or other types of data, such as metadata of an image file. This can be metadata that has already been used. The neural network task in this situation may be to classify, score, or otherwise predict properties about the received input. For example, a neural network can be trained to predict the likelihood that input it receives contains text related to a particular theme. Neural networks can also be trained to generate text predictions as part of performing a specific task, for example as part of a tool for auto-completion of text in a document while composing the document. For example, a neural network can be trained to predict the translation of text in an input document into a target language while composing a message.

Other types of input documents may be data related to characteristics of a network comprised of interconnected devices. These input documents may include records regarding activity logs and access privileges that allow various computing devices to access various sources of potentially sensitive data. Neural networks can be trained to process these and other types of documents to predict current or future security breaches to the network. For example, neural networks can be trained to predict network intrusions by malicious actors.

As another example, the input to the neural network can be audio input, including streaming audio, pre-recorded audio, and audio as part of video or other sources or media. In the context of speech, neural network tasks may include speech recognition, including separating speech from other identified speech sources and/or emphasizing characteristics of identified speech to make it easier to hear. For example, as part of a translation tool, a neural network can be trained to predict accurate translations of input speech into a target language in real time.

Additionally, in addition to data inputs that include the various types of data described herein, neural networks can be trained to process features that correspond to a given input. A feature is a value, for example a numerical value or a definite value related to a characteristic of the input. For example, in the context of an image, image features may relate to the RGB values of each pixel in the image. A neural network task in an image/video situation may be, for example, to classify the content of an image or video with respect to the presence of different people, places, or things. Neural networks can be trained to extract and select relevant features that are processed to produce an output for a given input, creating new features based on learned relationships between various characteristics of the input data. It can also be trained to produce quantities.

Aspects of the present disclosure may be implemented as a digital circuit, a computer-readable storage medium, one or more computer programs, or a combination of one or more of these. A computer-readable storage medium can be a non-transitory computer-readable storage medium, e.g., as one or more instructions executable by processor(s) and stored on a tangible storage device. .

The phrase "configured to" is used herein in various contexts relating to a computer system, hardware, or piece of computer program. When it is stated that a system is configured to perform one or more operations, this means that the system is configured with appropriate software, firmware, and/or hardware that, when operated, causes the system to perform one or more operations. is installed on the system. When hardware is said to be configured to perform one or more operations, this means that, in operation, it accepts input and, depending on the input, produces output corresponding to the one or more operations. It means that the hardware includes one or more circuits. When a computer program is said to be configured to perform one or more operations, this means that when executed by one or more computers, it causes one or more computers to perform one or more operations. means that a computer program includes one or more program instructions to cause

Although the acts illustrated in the drawings and recited in the claims may be shown in a particular order, these acts may be performed in a different order than that shown, and some acts may be It is to be understood that this may be omitted, may be performed more than once, and/or may be performed in parallel with other operations. Furthermore, separation of various system components configured to perform various operations is not to be understood as requiring separation of these components. The components, modules, programs, and engines described can be integrated into one system or can be part of multiple systems.

Unless explicitly stated otherwise, most of the other embodiments described above are not mutually exclusive. However, they may be implemented in various combinations to achieve unique benefits. These and other variations and combinations of the features described above may be utilized without departing from the subject matter of the invention as indicated by the appended claims, and therefore the above description of the embodiments is incorporated herein by reference in the accompanying patents. The subject matter indicated by the claims is to be regarded as illustrative rather than limiting. In addition, the provision of the embodiments described herein, and the provision of phrases such as "such as," "including," and the like, are intended to exemplify the subject matter of the appended claims. should not be construed as limiting. Rather, these examples merely illustrate one of many possible implementations. Furthermore, the same reference numbers in different drawings may identify the same or similar elements.

Claims (20)

A computer-implemented method for determining the architecture of a neural network, comprising:
one or more processors receiving information specifying a target computing resource;
the one or more processors receiving data specifying a basic neural network architecture;
the one or more processors identifying a plurality of scaling parameter values for scaling the base neural network in response to information specifying the target computing resource and a plurality of scaling parameters for the base neural network; and the identifying includes:
selecting multiple scaling parameter value candidates;
and determining a performance evaluation index of the basic neural network scaled according to the plurality of scaling parameter value candidates, and the performance evaluation index is determined according to a plurality of purposes including a latency purpose. determined, said method comprising:
The method further comprising: the one or more processors generating a scaled neural network architecture using the base neural network architecture scaled according to the plurality of scaling parameter values. The plurality of purposes are a plurality of second purposes,
receiving said data specifying an architecture of said base neural network;
one or more processors receiving training data corresponding to a neural network task;
2. The one or more processors comprising: performing a neural architecture search of a search space using the training data to identify an architecture of the base neural network according to a plurality of first objectives. the method of. The search space includes candidate neural network layers, each candidate neural network layer configured to perform one or more operations;
3. The method of claim 2, wherein the search space includes neural network layer candidates that include different activation functions. The basic neural network architecture includes a plurality of component candidates, each component having a plurality of neural network layers;
The search space includes a first component of a network layer candidate including a first activation function, and a second component of a network layer candidate including a second activation function different from the first activation function. 4. The method of claim 3, comprising a plurality of candidate components of the neural network layer candidates. 3. The method of claim 2, wherein the plurality of first objectives for performing the neural architecture search are the same as the plurality of second objectives for identifying the plurality of scaling parameter values. The plurality of first objectives and the plurality of second objectives include an accuracy rate objective corresponding to an accuracy rate of the output of the basic neural network when trained using the training data. Method. The performance evaluation index is configured such that the basic neural network receives input and generates an output when the basic neural network is scaled according to the plurality of scaling parameter value candidates and is deployed on the target computing resource. 2. The method of claim 1, wherein the method corresponds at least in part to a measure of latency between. 5. The latency objective corresponds to a minimum latency between the base neural network accepting input and producing an output when the base neural network is deployed on the target computing resource. The method described in 1. The information specifying the target computing resource specifies one or more hardware accelerators;
2. The method of claim 1, further comprising running the scaled neural network on the one or more hardware accelerators to perform the neural network task. the target computing resource is a first target computing resource, the plurality of scaling parameter values are a plurality of first scaling parameter values,
The method includes:
the one or more processors receiving information specifying a second target computing resource different from the first target computing resource;
identifying a plurality of second scaling parameter values for scaling the base neural network in response to information specifying the second target computing resource, the plurality of second scaling parameter values comprising: 10. The method of claim 9, wherein the plurality of first scaling parameter values are different. The plurality of scaling parameter values are a plurality of first scaling parameter values,
The method further includes generating a scaled neural network architecture from the base neural network architecture scaled using a plurality of second scaling parameter values, the second scaling parameter values being 2. The method of claim 1, wherein the method is generated in response to one scaling parameter value and one or more composite coefficients that uniformly change the value of each of the first scaling parameter values. The basic neural network is a convolutional neural network, and the plurality of scaling parameters include one or more of a depth of the basic neural network, a width of the basic neural network, and a resolution of input to the basic neural network. , the method of claim 1. A system,
one or more processors;
one or more storage devices storing instructions coupled to the one or more processors, the instructions, when executed by the one or more processors, causing the one or more processors to performing an operation to determine the architecture of the network, the operation comprising:
an operation of accepting information specifying a target computing resource;
an act of receiving data specifying the architecture of the basic neural network;
an act of identifying a plurality of scaling parameter values for scaling the basic neural network in response to information specifying the target computing resource and a plurality of scaling parameters of the basic neural network; teeth,
selecting multiple scaling parameter value candidates;
and determining a performance evaluation index of the basic neural network scaled according to the plurality of scaling parameter value candidates, and the performance evaluation index is determined according to a plurality of purposes including a latency purpose. determined, said operation further comprising:
A system comprising an act of generating a scaled neural network architecture using the base neural network architecture scaled according to the plurality of scaling parameter values. The plurality of purposes are a plurality of second purposes,
The act of receiving the data specifying the architecture of the basic neural network comprises:
receiving training data corresponding to a neural network task;
14. The system of claim 13, comprising performing a neural architecture search of a search space using the training data to identify an architecture of the base neural network according to a plurality of first objectives. The search space includes candidate neural network layers, each candidate neural network layer configured to perform one or more operations;
15. The system of claim 14, wherein the search space includes neural network layer candidates that include different activation functions. 15. The system of claim 14, wherein the plurality of first objectives for performing the neural architecture search are the same as the plurality of second objectives for identifying the plurality of scaling parameter values. 15. The plurality of first objectives and the plurality of second objectives include an accuracy rate objective corresponding to the accuracy rate of the output of the basic neural network when trained using the training data. system. The performance evaluation index is configured such that the basic neural network receives input and generates an output when the basic neural network is scaled according to the plurality of scaling parameter value candidates and is deployed on the target computing resource. 14. The system of claim 13, wherein the system corresponds at least in part to a metric of latency between. 5. The latency objective corresponds to a minimum latency between the base neural network accepting input and producing an output when the base neural network is deployed on the target computing resource. The system described in 13. one or more non-transitory computer-readable storage media storing instructions, the instructions, when executed by the one or more processors, causing the one or more processors to implement a neural network architecture; An operation for determining is performed, and the operation is:
an operation of accepting information specifying a target computing resource;
an act of the one or more processors receiving data specifying a basic neural network architecture;
an act of identifying a plurality of scaling parameter values for scaling the basic neural network in response to information specifying the target computing resource and a plurality of scaling parameters of the basic neural network; teeth,
selecting multiple scaling parameter value candidates;
and determining a performance evaluation index of the basic neural network scaled according to the plurality of scaling parameter value candidates, and the performance evaluation index is determined according to a plurality of purposes including a latency purpose. determined, said operation further comprising:
A computer-readable storage medium comprising an act of generating a scaled neural network architecture using the base neural network architecture scaled according to the plurality of scaling parameter values.

Images