Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
  • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means

Definitions

• ANNs Artificial Neural Networks
• the human brain contains 10-20 billion neurons connected through synapses. Electrical and chemical messages are passed from neurons to neurons based on input information and their resistance to passing information.
• a neuron can be represented by a node performing a simple operation of addition coupled with a saturation function.
• a synapse can be represented by a connection between two nodes. Each of the connections can be associated with an operation of multiplication by a constant.
• the ANNs are particularly useful for solving problems that cannot be easily solved by classical computer programs. [0003] While forms of the ANNs may vary, they all have the same basic elements similar to the human brain. A typical ANN can be organized into layers, and each of the layers may include many neurons sharing similar functionality. The inputs of a layer may come from a previous layer, multiple previous layers, any other layers, or even the layer itself.
• Major architectures of ANNs include Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), and Long Term Short Memory (LTSM) network, but other architectures of ANN can be developed for specific applications.
• ANNs can then be computed in parallel on many different computing elements similar to neurons of the brain.
• a single ANN may have hundreds of layers. Each of the layers can involve millions of connections. Thus, a single ANN may potentially require billions of simple operations like multiplications and additions. [0004] Because of the larger number of operations and their parallel nature, ANNs can result in a very heavy load for processing units (e.g., CPU), even ones running at high rates.
• processing units e.g., CPU
• GPUs graphics processing units
• GPUs graphics processing units
• GPUs appear to be more efficient in the computations of ANNs than the CPUs.
• GPUs are not well suited to the computations of ANNs because the GPUs have been specifically designed to compute graphical images.
• the GPUs may provide a certain level of parallelism in computations.
• the GPUs are constraining the computations in long pipes implying latency and lack of reactivity.
• very large GPUs can be used, which may involve excessive power consumption, which is a typical issue of GPUs. Since the GPUs may require more power consumption for the computations of ANNs, the deployment of GPUs can be difficult.
• CPUs provide a very generic engine that can execute very few sequences of instructions with a minimum effort in terms of programming, but lack the power of computing for ANN.
• the GPUs are slightly more parallel and require a larger effort of programming than CPUs, which can be hidden behind libraries with some performance costs but are not very suitable for ANNs.
• Field Programmable Gate Arrays FPGAs are professional components that can be programmed at the hardware level after they are manufactured. The FPGAs can be configured to perform computations in parallel. Therefore, FPGAs can be well suited to compute ANNs.
• One of the challenges of FPGAs is the programming, which requires a much larger effort than programming CPUs and GPUs.
• Adaption of FPGAs to perform ANN computations can be more challenging than for CPUs and GPUs.
• Most attempts in programming FPGAs to compute ANNs have been focusing on a specific ANN or a subset of ANNs requiring modification of the ANN structure to fit into a specific limited accelerator or providing a basic functionality without solving the problem of computing ANN on FPGAs globally.
• the computation scale is typically not considered for existing FPGA solutions, with much of the research being limited to a single or few computation engines, which could be replicated.
• the existing FPGA solutions do not solve the problem of massive data movement required at a large scale for the actual ANN involved in real industrial applications.
• the inputs to be computed with an ANN are typically provided by an artificial intelligence (AI) framework.
• AI artificial intelligence
• a system for increasing quality of results of computation of an ANN by using complex rounding rules for parameters in the ANN may include one or more processing units.
• the processing units can receive a plurality of first parameters for one or more neurons of ANN.
• the first parameters can be of a first data type.
• the processing units can change the first parameters to second parameters of a second data type to obtain a plurality of the second parameters according to a rule in which a distance between at least one first parameter and corresponding second parameter is greater than a distance between the at least one first parameter and a value of the second data type, the value being closest to the at least one first parameter.
• Changing the first parameters to second parameters of a second data type may require scaling the range of the first data type to a range of the second data type prior to the changing.
• Changing the first parameters according to the rule can be different from rounding the first parameters to the nearest values of the second data type.
• the plurality of the first parameters may include parameters ⁇ and B such that ⁇ ⁇ B and the parameter A may correspond to a parameter ⁇ of the plurality of the second parameters and the parameter B corresponds to a parameter B of the plurality of the second parameters, such that ⁇ > B.
• the plurality of the first parameters may include a parameter ⁇ , such that: ⁇ L is a first number of the second data obtained by rounding the parameter A down, A u is a second number of the second data obtained by rounding the parameter A up, and the parameter A corresponds to parameter ⁇ , and ⁇ is located outside an interval [ ⁇ L ; ⁇ H ].
• the second data type can be formed by a subset of values of the first data type. After the first parameters are changed to the second parameters, a vector distance between a vector of the first parameters and a vector of the second parameters is a minimum of vector distances between the vector of the first parameters and vectors of values of the second data type.
• the vector distance can be determined by an average of the absolute values of the differences between the first parameters and the second parameters. Alternatively, the vector distance can be determined by a formula using the first parameters and the second parameters and returning an overall quality of the rounding.
• the changing the first parameters can be performed by iterations.
• the iterations may include the following steps: selecting a parameter from the plurality of the second parameters, modifying the selected parameter to a different value of the second data type, computing a vector distance between a vector of the second parameters and a vector of the first parameters, and keeping the different value for the selected parameter if the vector distance is less than the vector distance at a preceding iteration.
• the iterations can be terminated when the vector distance has not decreased substantially during a predetermined number of the iterations.
• the parameter can be selected randomly.
• a precision of the second type is less than a precision of the first data type.
• the first data type can be a floating-point data type and the second data type can be a fixed-point data type.
• the plurality of the first parameters includes one or more of the following: weights to the input values, activation function parameters, offsets to products of sums of the weights and the input values, and static inputs to the one or more neurons.
• the plurality of the first parameters can be associated with a feature map.
• the plurality of the first parameters can be associated with neurons selected from a proper subset of neurons of the ANN.
• the first parameters can be obtained by one or more of the following: training the ANN, retraining the ANN, and pruning the ANN.
• FIG. 1 is a block diagram showing an example system for quantization data in ANN computations, according to some example embodiments.
• FIG. 2 shows an ANN, neuron, and transfer function, according to an example embodiment.
• FIG. 3A is a flow chart showing training and inference of an ANN performed with the same data type, according to some example embodiments.
• FIG. 3B is a flow chart showing training and inference of an ANN using different data types, according to some example embodiments.
• FIG. 3A is a flow chart showing training and inference of an ANN using different data types, according to some example embodiments.
• FIG. 4 is a schematic diagram showing a regular rounding in ANN quantization.
• FIG. 5 is a schematic diagram showing a process of determination of a quantization error due to rounding in ANN quantization, according to some example embodiments of the disclosure.
• FIG. 6 is a schematic diagram showing rounding in ANN quantization according to some example embodiments of the disclosure.
• FIG. 7 is a schematic diagram showing a process of determination of a quantization error due to rounding in ANN quantization, according to some other example embodiments of the disclosure.
• FIG. 8 is a flow chart showing steps of a method for rounding parameters of ANN, according to some example embodiments of the disclosure.
• FIG. 9 is a flow chart showing steps of a method for rounding parameters in ANN quantization, according to some example embodiments of the disclosure.
• FIG. 10 shows a computing system that can be used to implement embodiments of the disclosed technology.
• the terms “or” and “and” shall mean “and/or” unless stated otherwise or clearly intended otherwise by the context of their use.
• the term “a” shall mean “one or more” unless stated otherwise or where the use of “one or more” is clearly inappropriate.
• the terms “comprise,” “comprising,” “include,” and “including” are interchangeable and not intended to be limiting.
• the term “including” shall be interpreted to mean “including, but not limited to.”
• the present disclosure is directed to improving quality of results of computations of ANNs when using lower precision of computations. Typically, ANNs are trained using a high precision data type.
• computations of ANNs can be performed using a lower precision data type than the data type used for the training the ANNs.
• a conversion of data from the high precision to low precision can pe performed by rounding the values, such as weights and biases, to close numbers of the lower precision type.
• weights of the ANN can be converted from real numbers such as 32-bit floating numbers, to integers, such as 8-bit integers. This approach allows reducing a number of bit operations and memory required for performing computations of ANN.
• a weight presented by a real number W is converted to integer I, which is the closest to the real number W.
• the precision of the second data type can be lower than the precision of the first data type.
• the ANN parameters ⁇ p 1 ,p 2 , .., p N c ⁇ an be involved in computations of a set of neurons ⁇ 1 , ⁇ 2 , .., ⁇ M ⁇ of the ANN.
• the set ⁇ 1 , ⁇ 2 , .., ⁇ M ⁇ may include a single neuron, neurons belonging to one or more feature maps, neurons belonging to a single layer of the ANN, or neurons belonging to different layers of the ANN. However, the set ⁇ 1 , ⁇ 2 , .., ⁇ M ⁇ is smaller than a full set of all neurons in the ANN.
• the ⁇ p 1 ,p 2 , .., p N m ⁇ ay include weights, biases, or combinations of weights and biases required for calculations of the neurons of the set ⁇ 1 , ⁇ 2 , .., ⁇ M ⁇ [0034]
• a scheme of rounding ANN parameters ⁇ p 1 ,p 2 , .., p N t ⁇ o parameters ⁇ q 1 ,q 2 , ..,q N e ⁇ liminates the need for preserving an order between ANN parameters ⁇ p 1 ,p 2 , .., p N.
• At least one parameter p i from the ANN parameters ⁇ p 1 ,p 2 , .., p N c ⁇ an be mapped to a value ⁇ ⁇ of the second data type that is not closest to the parameter ⁇ ⁇ .
• the method for rounding can be based on the minimization of error D between the ANN parameters ⁇ p 1 ,p 2 , .., p N a ⁇ nd the parameters ⁇ q 1 ,q 2 , ..,q N .
• the error D is an average of differences or any other metric based on the distances between p i and q i .
• the values of the parameters ⁇ q 1 ,q 2 , ..,q N ⁇ corresponding to the minimum of error D can be found in an iterative process.
• a parameter of ⁇ ⁇ can be selected and a value for the parameter of q i can be modified, iteratively, by either selecting the next closest value of the second data type or by selecting a random value.
• the distance between the parameters ⁇ p 1 ,p 2 , .., p N ⁇ and the parameters ⁇ q 1 ,q 2 , ..,q N ⁇ s recalculated. If the error D is less than a threshold, then iterations are terminated.
• Parameters ⁇ p 1 ,p 2 , .., p N ⁇ can include activation functions, biases, weights, and inputs. Parameters ⁇ p 1 ,p 2 , .., p N ⁇ can belong to several layers in a row. [0040] Technical effects of certain embodiments of the present disclosure can include increasing accuracy of fixed-point ANN computations. Further technical effects of certain embodiments of the present disclosure can allow decreasing saturations of neurons in fixed-point ANN computations.
• FIG. 1 is a block diagram showing an example system 100 for increasing quality of results computation of an ANN by using complex rounding rules for parameters in ANN, according to some example embodiments.
• the system 100 can be part of a computing system, such as a personal computer, a server, a cloud-based computing resource, and the like.
• the system 100 may include one or more processing unit(s) 110 and a memory 120.
• the memory 120 may include computer-readable instructions for execution by the processing unit(s) 110.
• the processing unit(s) 110 may include a programmable processor, such as a microcontroller, CPU, and so forth.
• the processing unit(s) 110 may include an application-specific integrated circuit(s) or programmable logic array(s), such as an FPGA(s), designed to implement the functions performed by the system 100.
• the system 100 may be installed on a remote server or may be provided as a cloud service residing in a cloud storage.
• the processing unit(s) 110 may be configured to receive a structure and parameters of an ANN. The parameters of the ANN can be presented in a first data type.
• the processing unit(s) 110 may be configured to receive a plurality of first parameters for one or more neurons of ANN, the first parameters being of a first data type.
• the processing unit(s) 110 may change the first parameters to second parameters of a second data type to obtain a plurality of the second parameters.
• the change can be performed according to a rule in which a distance between at least one first parameter and corresponding second parameter is greater than a distance between the at least one first parameter and a value of the second data type, the value being closest to the at least one first parameter.
• a vector distance between a vector of the first parameters and a vector of the second parameters can be a minimum of vector distances between the vector of the first parameters and vectors of values of the second data type.
• the details of the rules for modification of the first parameters to second parameters are described below with reference to FIG.6.
• computation of a neuron of the ANN using values of the second data type may require fewer operations of the processing unit(s) 110 than the computation of the same neuron of the ANN using values of the first data type.
• the input datasets presented using the second data type may require less memory to be stored than the same input datasets presented using the first data type.
• FIG. 2 shows ANN 210, neuron 220, and transfer function 230, according to some example embodiments.
• the ANN 210 may include one or more input layers 240, one or more hidden layers 250, and one or more output layers 260.
• Each of the input layers 240, hidden layers 250, and output layers 260 may include one or more (artificial) neurons 220. The number of neurons can be different for different layers.
• Each of neurons 220 may be represented by a calculation of a mathematical function [0048] wherein V[i] are input values to a neuron, W[i] are weights assigned to the input values at the neuron, bias is an offset to a weighted sum of the input values, and F(X) is a transfer function applied to the weighted sum.
• the transfer function 230 F(X) is selected to have a limit of zero as X approaches zero.
• the transfer function 230 F(X) can be in the form of a sigmoid.
• the result of the calculation of a neuron propagates as an input value of further neurons in the ANN.
• the further neurons can belong to either the next layer, previous layer, or the same layer.
• FIG. 3A is a flow chart showing a workflow 300A for training 310 and inference 325 of an ANN, according to some example embodiments.
• the training 310 (also known as learning) is a process of teaching ANN 305 to output a proper result based on a given set of training data 315.
• the process of training may include determining weights 320 of neurons of the ANN 305 based on training data 315.
• the training data 315 may include samples. Each sample may be represented as a pair of input values and expected output.
• the training data 315 may include hundreds to millions of samples. While training 310 is required to be performed only once, it may require a significant amount of computations and may take a considerable time.
• the ANNs can be configured to solve different tasks including, for example, image recognition, speech recognition, handwriting recognition, machine translation, social network filtering, video games, medical diagnosis, and so forth.
• the inference 325 is a process of computation of an ANN.
• the inference 325 uses the trained ANN weights 320 and new data 330 including new sets of input values. For each new set of input values, the computation of the ANN provides a new output that answer the problem that the ANN is supposed to solve.
• an ANN can be trained to recognize various animals in images.
• the ANN can be trained using millions of images of animals. Submitting a new image to the ANN would provide the information concerning animals in the new image (this process being known as image tagging). While the inference for each image takes fewer computations than training, the number of inferences can be large because new images can be received from billions of sources.
• the inference 325 includes multiple computations of sum of the following products: wherein the V[i] are new input values to neurons and W[i] are weights associated with the input values to the neurons of the ANN.
• both training 310 and inference 325 in FIG. 3A are performed using computations based on the same type of data (for example, real values in floating-point format). Performing inference for a large number of input datasets of new data 330 using floating-point calculations can be time consuming and may require significant computing resources for computations of an ANN.
• the inference of an ANN be performed using integer- based or fixed-point calculations in order to reduce computation time and computing resources required to perform ANN computations.
• FIG. 3B is a flow chart showing a workflow 300B of training 310 and inference 345 of an ANN using different data types for training and inference, according to some example embodiments.
• the training 310 can be performed using training data 315.
• the training data 315 can be of a first data type (for example, real values in the floating-point format).
• the process of training may include determining weights 320 of neurons of the ANN 305.
• the weights 320 can be also of the first data type.
• the weights 320 and other parameters of ANN can be quantized in quantization 335.
• the weights 320 can be mapped to a set including a pre-determined number of values of a second data type.
• the second data type may include integers.
• the inference 345 can be further performed using the quantized values for the weights 320.
• each input dataset in new data 330 can be also quantized (that is, mapped to the values of the second data type) in quantization 340 using the same quantization workflow as in the quantization 335. Since the weights 320 and the input sets of new data 330 are quantized and converted to the second data type, the inference 345 can be performed using hardware configured to perform computations using only the second data type. The computations using the second data type may require less time and memory resources than the same computations using the first data type. However, the result of the inference 345 performed using the second data type can be less accurate than the result of inference 325 performed using the first data type used in the training of ANN. It should be noted that the quantization differs from a simple data mapping because the quantization of a value of the first data type may result in a different value of the second data type.
• FIG. 4 is a schematic diagram showing a regular rounding of parameters in ANN quantization.
• the weights ⁇ W 1 , W 2 , ... , W j , ... ⁇ are of a first data type.
• the weights [W 1 , W 2 , W j , ... ⁇ are rounded to the weights ⁇ W 1 , W 2 , ..., W j , ... ⁇ of a second data type.
• the weights ⁇ W 1 , W 2 , ..., W j , ... ⁇ can be scaled to match the range of weights ⁇ W 1 , W 2 , ..., W j , ... ⁇ in the first data type to the range of the second data type.
• the second data type can be a subset of values of the first data type.
• the first data type can be presented by a floatingpoint format and the second data can be presented by a fix-point format.
• the second data type can have a lower precision than the first data type. According to the regular rounding, weights ⁇ W 1 , W 2 , ... , W j , ... ⁇ are mapped to closest values of the second data type.
• Another conventional rounding method includes always rounding the values of the first type to the values of the second type down. Yet another conventional rounding method includes always rounding the values of the first type to the values of the second type up.
• FIG. 5 is a schematic diagram showing a process 500 for determining a quantization error 530 due to rounding of parameters in ANN quantization, according to some example embodiments of the disclosure.
• the quantization error 530 can be determined using a set of neurons 220.
• the set of neurons 220 may include a single neuron of an ANN.
• the set of neurons may comprise neurons belonging to a layer of ANN or two or more layers of ANN. However, a number of neurons in the set of neurons 220 is less than the number of all neurons of the ANN.
• the neurons can be associated with parameters including weights ⁇ W 1 , W 2 , ... , W j , ... ⁇ for the input values for neurons, biases, and parameters of activation function used for computing outputs of neurons.
• the parameters can be of the first data type (shown in FIG. 4) and can be obtained in one of the following: training the ANN (shown in FIG. 3A), retraining the ANN, and pruning the ANN.
• the parameters of the ANN can be quantized (that is, converted from the first data type to a second data type).
• a regular rounding shown in FIG. 4 is used for the conversion of the parameters of the ANN from the first data type to the second data type.
• the weights ⁇ W 1 , W 2 , ... , W j , ... ⁇ of the first data type are converted to the ⁇ W 1 , W 2 , ... , W j , ... ⁇ of the second data type.
• the ⁇ W 1 , W 2 , ... , W j , ... ⁇ can be associated with a kernel that is used to produce a feature map by sliding a window 520 in the sets of input values 510.
• the sets of input values 510 may include at least two different sets of input values that represent two different samples to be processed by the ANN. For example, if the ANN is trained to recognize an image, the different sets of input values 510 may represent different images.
• the set of outputs 540 of the neurons 220 can be obtained by using the sets of input values 510 and original (unquantized) weights W 2 , ... , W j , ... ⁇ .
• the set of the outputs 560 of the neurons 220 can be obtained by using the same sets of input values 510 and the quantized weights ⁇ W 1 W 2 , ... , W j , ⁇ , ... ⁇ .
• a distance between a vector O formed by outputs 540 and a vector O formed by the outputs 560 can represent the quantization error 530.
• the distance can be found by one or more metrics, such as an average of absolute values of differences between the outputs 540 and the outputs 560, a sum of squares of the differences between the outputs 540 and the outputs 560, a sum of squares of relative differences between the outputs 540 and the outputs 560, and so forth.
• metrics such as an average of absolute values of differences between the outputs 540 and the outputs 560, a sum of squares of the differences between the outputs 540 and the outputs 560, a sum of squares of relative differences between the outputs 540 and the outputs 560, and so forth.
• the quantization error 530 depends on a method for rounding the weights ⁇ W 1 , W 2 , ..., W j , ... ⁇ to the weights ⁇ W 1, W 2 , W j , ... ⁇ and rounding other parameters of the neurons 220.
• the regular rounding shown in FIG. 4 may not result in the lowest value of the quantization error 530.
• retraining, or fine training of the ANN can be performed to reduce the error 530 resulting from the quantization.
• retraining requires a substantial amount of computing and input data for the ANN. The retraining may not result in better results of computations of the ANN if the inference system for the ANN is not known precisely.
• FIG. 6 is a schematic diagram showing results of computations of an ANN by using complex rounding rules for ANN parameters, according to some example embodiments of the disclosure.
• the weights ⁇ W 1 , W 2 , ..., W j , ... ⁇ are of a first data type.
• the weights ⁇ W 1 , W 2 , ..., W j , ... ⁇ are of a second data type and correspond to the weights ⁇ W 1 , W 2 , ..., W j , ... ⁇ .
• the weights W 2 , ... , W j , ... ⁇ can be scaled so the range of weights ⁇ W 1 , W 2 , ..., W j , ...
• the weights W 2 , ... , W j , ... ⁇ may produce a lower error 530 than the weights obtained by regular rounding in quantization of the weights ⁇ W 1 , W 2 , ..., W j , ... ⁇ .
• the scheme of rounding the ⁇ W 1 , W 2 , ..., W j , ... ⁇ to the weights ⁇ W 1 , W 2 , ..., W j , ... ⁇ may result in one of the following permissions for modifications of weights ⁇ W 1 , W 2 , ..., W j , ... ⁇
• FIG. 7 is a schematic diagram showing a process 800 of determination of a rounding error due to rounding in ANN quantization, according to some other example embodiments of the disclosure.
• the process 800 does not involve loading sets of input values to neurons or computing outputs of the neurons.
• the process 800 requires only loading weights associated with input values to the neurons.
• FIG. 8 is a flow chart showing steps of a method for rounding parameters in ANN quantization, according to some example embodiments of the disclosure.
• the method 900 may be performed by the system 100 described above with reference to FIG. 1.
• the method 900 may commence, in block 905, with loading weights associated with input values to one or more neurons of an ANN.
• the weights can include weights of the ANN found in training, retraining, and, optionally, pruning of the ANN.
• the weights can be of a first data type.
• the weights can be rounded to weights of the second data type.
• a regular rounding scheme can be used in the block 910. The regular rounding scheme is described in FIG. 4.
• the initial rounding of weights in block 910 can be performed based on a rule according to which: 1) a number of the weights higher than an average of the weights is the same as a number of the weights higher than an average of the weights a number of the weights values lower than the average of the weights is the same as a number of the weights lower than an average of the weights.
• a number of the weights higher than an average of the weights is the same as a number of the weights higher than an average of the weights
• a number of the weights values lower than the average of the weights is the same as a number of the weights lower than an average of the weights
• each weight W i prior to rounding weights to weights each weight W i can be scaled so the range of weights in the first data type matches the range of the second data type.
• the method 900 may start iterations with selecting a weight from the weights
• the weight can be selected randomly.
• the weight W k can be selected according to a predetermined order or a rule.
• the method 900 may include modifying to the selected weight W k to another value of the second data type.
• the modification may result in rounding of the weight W k to weight W k as described in rounding scheme of the FIG. 6.
• the method 900 may determine whether the error is lower than an error calculated before in the preceding iteration. If the error is lower, then the method 900 proceeds, in block 945, with keeping the modified weight W k and proceeding, in block 915, with selecting another weight W k . If the error is not lower, the method 900 may proceed to block 940.
• the method 900 may reject the modification of the weight W k and proceed to the decision block 945.
• the method 900 may estimate a rate of change of the error during a pre-determined number of last iterations. If the rate is higher than a rate threshold, the method 900 may proceed to proceed to block 915 with selecting another weight W k . If the rate is less than the rate threshold, the method 900 may terminate the iterations and proceed, in block 905, with loading next weights for next neurons.
• FIG. 9 is a flow chart illustrating a method 1100 for rounding parameters in ANN quantization, in accordance with some example embodiments.
• the operations can be combined, performed in parallel, or performed in a different order.
• the method 1100 may also include additional or fewer operations than those illustrated.
• the method 1100 may be performed by the system 100 described above with reference to in FIG. 1.
• the method 1100 may commence with receiving a plurality of first parameters for one or more neurons of ANN.
• the first parameters are of a first data type.
• the plurality of the first parameters includes one or more of the following: weights to the input values, activation function parameters, offsets (biases) to products of sums of the weights and the input values, and static inputs to the one or more neurons.
• the plurality of the first parameters can be associated with neurons selected from a proper subset of neurons of the ANN.
• the plurality of the first parameters can be associated with a feature map.
• the first parameters can be preliminarily obtained by one or more of the following: training the ANN, retraining the ANN, and pruning the ANN.
• the method 1100 may include changing the first parameters to second parameters of a second data type to obtain a plurality of the second parameters according to a rule.
• the rule may allow a distance between at least one first parameter and corresponding second parameter to be greater than a distance between this first parameter and a value of the second data type closest to the at least one first parameter.
• changing the first parameters is different from rounding the first parameters to the nearest values of the second data type.
• the second data type can be formed by a subset of values of the first data type.
• a precision of the second type is less than a precision of the first data type.
• the first data type can be a floating-point data type and the second data type can be a fixed-point data type.
• a vector distance between a vector of the first parameters and a vector of the second parameters is a minimum of vector distances between the vector of the first parameters and vectors of values of the second data type.
• the vector distance can be determined by an average of the absolute values of the differences between the first parameters and the second parameters.
• the second parameters can be determined by performing iterations.
• the iterations may include the following steps: selecting a parameter from the plurality of the second parameters, modifying the selected parameter to a different value of the second data type, computing a vector distance between a vector of the second parameters and a vector of the first parameters, and keeping the different value for the selected parameter if the vector distance is less than the vector distance at a preceding iteration.
• the iterations can be terminated when the vector distance has not decreased substantially during a predetermined number of the iterations.
• the plurality of the first parameters includes parameters A and B such that A ⁇ B and the parameter A corresponds to a parameter A of the plurality of the second parameters and the parameter B corresponds to a parameter B of the plurality of the second parameters, such that A > B.
• the plurality of the first parameters includes a parameter A, such that: is a first number of the second data obtained by rounding the parameter A down, is a second number of the second data obtained by rounding the parameter A up, and the parameter A corresponds to parameter A , and A is located outside an interval
• FIG. 10 illustrates an example computing system 1200 that may be used to implement embodiments described herein.
• the example computing system 1200 of FIG. 10 may include one or more processors 1210 and memory 1220.
• Memory 1220 may store, in part, instructions and data for execution by the one or more processors 1210.
• Memory 1220 can store the executable code when the exemplary computing system 1200 is in operation.
• the processor 1210 may include internal accelerators like a graphical processing unit, a FPGA, or similar accelerators that may be suitable for use with embodiments described herein.
• the memory 1220 may include internal accelerators like a GPU, a FPGA, or similar accelerators that may be suitable for use with embodiments described herein.
• the 10 may further include a mass storage 1230, portable storage 1240, one or more output devices 1250, one or more input devices 1260, a network interface 1270, and one or more peripheral devices 1280.
• the components shown in FIG.10 are depicted as being connected via a single bus 1290. The components may be connected through one or more data transport means.
• the one or more processors 1210 and memory 1220 may be connected via a local microprocessor bus, and the mass storage 1230, one or more peripheral devices 1280, portable storage 1240, and network interface 1270 may be connected via one or more input/output buses.
• Mass storage 1230 which may be implemented with a magnetic disk drive, an optical disk drive or a solid state drive, is a non-volatile storage device for storing data and instructions for use by a magnetic disk, an optical disk drive or SSD, which in turn may be used by one or more processors 1210.
• Mass storage 1230 can store the system software for implementing embodiments described herein for purposes of loading that software into memory 1220.
• the mass storage 1230 may also include internal accelerators like a GPU, a FPGA, or similar accelerators that may be suitable for use with embodiments described herein.
• Portable storage 1240 may operate in conjunction with a portable non-volatile storage medium, such as a compact disk (CD) or digital video disc (DVD), to input and output data and code to and from the computing system 1200 of FIG. 10.
• the system software for implementing embodiments described herein may be stored on such a portable medium and input to the computing system 1200 via the portable storage 1240.
• One or more input devices 1260 provide a portion of a user interface.
• the one or more input devices 1260 may include an alphanumeric keypad, such as a keyboard, for inputting alphanumeric and other information, or a pointing device, such as a mouse, a trackball, a stylus, or cursor direction keys.
• Network interface 1270 can be utilized to communicate with external devices, external computing devices, servers, and networked systems via one or more communications networks such as one or more wired, wireless, or optical networks including, for example, the Internet, intranet, LAN, WAN, cellular phone networks (e.g., Global System for Mobile communications network, packet switching communications network, circuit switching communications network), Bluetooth radio, and an IEEE 802.11-based radio frequency network, among others.
• communications networks such as one or more wired, wireless, or optical networks including, for example, the Internet, intranet, LAN, WAN, cellular phone networks (e.g., Global System for Mobile communications network, packet switching communications network, circuit switching communications network), Bluetooth radio, and an IEEE 802.11-based radio frequency network, among others.
• Network interface 970 may be a network interface card, such as an Ethernet card, optical transceiver, radio frequency transceiver, or any other type of device that can send and receive information. Other examples of such network interfaces may include Bluetooth®, 3G, 4G, and WiFi® radios in mobile computing devices as well as a USB.
• One or more peripheral devices 1280 may include any type of computer support device to add additional functionality to the computing system.
• the one or more peripheral devices 1280 may include a modem or a router.
• the example computing system 1200 of FIG. 10 may also include one or more accelerator devices 1285.
• the accelerator devices 1285 may include PCIe-form-factor boards or storage-form-factor boards, or any electronic board equipped with a specific electronic component like a GPU, a Neural Processing Unit, a Multi-CPU component, a FPGA component, or similar accelerators electronic or photonic components, that may be suitable for use with embodiments described herein.
• the components contained in the exemplary computing system 1200 of FIG. 10 are those typically found in computing systems that may be suitable for use with embodiments described herein and are intended to represent a broad category of such computer components that are well known in the art.
• the exemplary computing system 1200 of FIG. 10 can be a personal computer, handheld computing device, telephone, mobile computing device, workstation, server, minicomputer, mainframe computer, or any other computing device.
• the computer can also include different bus configurations, networked platforms, multi-processor platforms, and so forth.
• Various operating systems (OS) can be used including UNIX, Linux, Windows, Macintosh OS, Palm OS, and other suitable operating systems.
• OS operating systems
• Some of the above-described functions may be composed of instructions that are stored on storage media (e.g., computer-readable medium). The instructions may be retrieved and executed by the processor. Some examples of storage media are memory devices, tapes, disks, and the like. The instructions are operational when executed by the processor to direct the processor to operate in accord with the example embodiments. Those skilled in the art are familiar with instructions, processor(s), and storage media.
• Non-volatile media include, for example, optical or magnetic disks, such as a fixed disk.
• Volatile media include dynamic memory, such as RAM.
• Transmission media include coaxial cables, copper wire, and fiber optics, among others, including the wires that include one embodiment of a bus.
• Transmission media can also take the form of acoustic or light waves, such as those generated during radio frequency and infrared data communications.
• Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, a hard disk, magnetic tape, any other magnetic medium, SSD, a CD-read-only memory (ROM) disk, DVD, any other optical medium, any other physical medium with patterns of marks or holes, a RAM, a PROM, an EPROM, an EEPROM, a FLASHEPROM, any other memory chip or cartridge, a carrier wave, or any other medium from which a computer can read.
• Various forms of computer-readable media may be involved in carrying one or more sequences of one or more instructions to a CPU for execution.
• a bus carries the data to system RAM, from which a CPU retrieves and executes the instructions.
• the instructions received by system RAM can optionally be stored on a fixed disk either before or after execution by a CPU.
• the instructions or data may not be used by the CPU but be accessed in writing or reading from the other devices without having the CPU directing them.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Health & Medical Sciences (AREA)
• Life Sciences & Earth Sciences (AREA)
• Biomedical Technology (AREA)
• Biophysics (AREA)
• Theoretical Computer Science (AREA)
• Evolutionary Computation (AREA)
• Computational Linguistics (AREA)
• Data Mining & Analysis (AREA)
• Artificial Intelligence (AREA)
• General Health & Medical Sciences (AREA)
• Molecular Biology (AREA)
• Computing Systems (AREA)
• General Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Mathematical Physics (AREA)
• Software Systems (AREA)
• Neurology (AREA)
• Complex Calculations (AREA)

Applications Claiming Priority (1)

Publications (1)

Family

ID=71094631

Family Applications (1)

Country Status (2)

• 2020
  • 2020-05-22 EP EP20733010.1A patent/EP4154191A1/de active Pending
  • 2020-05-22 WO PCT/IB2020/054903 patent/WO2021234437A1/en unknown

Also Published As

Similar Documents

Legal Events