JP6973651B2 - Arithmetic optimizers, methods and programs - Google Patents

Description

The present invention relates to, for example, an arithmetic optimization device, an arithmetic optimization method, and an arithmetic optimization program for optimizing an arithmetic using a discrimination model such as a neural network.

Inferences about given data may be made by using a model. Such a model is called a discriminant model. For example, image data may be given, and the image data and the discriminant model may infer what the image data represents (the object shown in the image).

A neural network is known as an example of a discrimination model. A neural network is a model in which a plurality of layers are connected, and each layer is composed of one or more units (neurons). When the inference process is performed using a neural network, the inference result regarding the input data can be obtained by inputting the input data to the input layer and performing the calculation in the forward direction from the input layer side to the output layer side.

Neural networks are learned by deep learning.

A tool for performing an operation using a neural network is described in, for example, Non-Patent Document 1.

Some existing tools for performing operations using neural networks can uniformly change the calculation accuracy of all layers of the neural network. For example, there are tools that can set operations on all layers of a neural network to floating-point operations or integer operations. Further, such a setting is made by a user (human).

"NVIDIA TensorRT", [online], NVIDIA Corporation, [Search on July 3, 2018], Internet <URL: https://developer.nvidia.com/tensorrt>

As mentioned above, some existing tools for performing operations using neural networks can uniformly change the calculation accuracy of all layers of the neural network. However, with such a tool, the calculation accuracy of each layer cannot be set individually. Therefore, it has been difficult to optimize the calculation using the neural network.

For example, when the calculation of all layers of the neural network is set to the floating-point calculation, the calculation can be performed with high accuracy, but the efficiency related to power consumption and the like is lowered. On the contrary, when the calculation of all layers of the neural network is set to the integer calculation, the efficiency regarding power consumption and the like is improved, but the calculation accuracy is lowered.

In addition, with existing tools, the user had to decide whether to use floating-point arithmetic or integer arithmetic for the calculation accuracy of the entire neural network.

Therefore, the present invention provides a calculation optimization device, a calculation optimization method, and a calculation optimization program that can automatically determine the calculation accuracy in each layer of the discrimination model so that the calculation using the discrimination model can be optimized. The purpose is to do.

The calculation optimization device according to the present invention is a calculation using a discrimination model in which a plurality of layers composed of one or more units are connected, and which is a first calculation circuit that performs a calculation with the first calculation accuracy. A predetermined explanatory variable for each application pattern, which is information that defines to which layer the second arithmetic circuit, which is applied to the layer and performs the arithmetic with the second arithmetic accuracy higher than the first arithmetic accuracy, is applied. Explanatory variable to acquire the value Determine the value acquisition means, the objective function calculation means that calculates the value of the objective function represented by the predetermined explanatory variable for each application pattern, and the application pattern that minimizes the value of the objective function. It is characterized by providing means for determining an application pattern.

Further, the arithmetic optimization method according to the present invention is an arithmetic using a discriminant model in which a plurality of layers composed of one or more units are combined, and the first arithmetic circuit performs the arithmetic with the first arithmetic accuracy. Is information on which layer the second arithmetic circuit is applied to, and the second arithmetic circuit that performs the arithmetic with the second arithmetic accuracy higher than the first arithmetic accuracy is applied to which layer. A predetermined explanation is given for each application pattern. It is characterized in that the value of a variable is acquired, the value of an objective function represented by a predetermined explanatory variable is calculated for each application pattern, and the application pattern in which the value of the objective function is minimized is determined.

Further, the calculation optimization program according to the present invention is a first calculation using a discrimination model in which a plurality of layers composed of one or more units are connected to a computer, and the calculation is performed with the first calculation accuracy. For each application pattern, which is information that defines to which layer the second arithmetic circuit is applied and which layer the second arithmetic circuit is applied to, which performs arithmetic with a second arithmetic accuracy higher than that of the first arithmetic accuracy. The explanatory variable value acquisition process for acquiring the value of a predetermined explanatory variable, the objective function calculation process for calculating the value of the objective function represented by the predetermined explanatory variable for each application pattern, and the value of the objective function are minimized. It is characterized in that an application pattern determination process for determining an application pattern is executed.

According to the present invention, the calculation accuracy in each layer of the discrimination model can be automatically determined so that the calculation using the discrimination model can be optimized.

It is a schematic diagram which shows the inference processing using a neural network. It is explanatory drawing which shows the example of the input / output of the said unit, and the coupling with other units when paying attention to one unit. It is a schematic diagram which shows the example of the processing apparatus which executes the operation of a part of the layers of a neural network with low accuracy, and executes the operation of the remaining layers with high accuracy. It is a schematic block diagram which shows an example of a low precision arithmetic circuit. It is a block diagram which shows the configuration example of MAC. It is a flowchart which shows the example of the processing progress of the inference processing using a neural network. It is a block diagram which shows the structural example of the arithmetic optimization apparatus of this invention. It is a schematic diagram which shows the example of the application pattern. It is a block diagram which shows the structural example of the arithmetic optimization apparatus which includes the processing apparatus. It is a block diagram which shows the structural example of the arithmetic optimization apparatus provided with the design information storage part. It is a flowchart which shows the example of the processing progress of the arithmetic optimization apparatus of embodiment of this invention. It is a schematic block diagram which shows the structural example of the computer which concerns on embodiment of this invention, or the modification thereof. It is a block diagram which shows the outline of the arithmetic optimization apparatus of this invention.

The calculation optimization device of the present invention determines the calculation accuracy of each layer in a discrimination model in which a plurality of layers each composed of one or more units are combined. An example of such a discriminant model is a neural network. In the following description, the case where the discriminant model is a neural network will be described as an example. However, the discriminant model is not limited to the neural network.

Further, in the following description, as a process using a neural network, a process of inferring the content indicated by the given input data will be described as an example. For example, a process in which image data is given and an object represented by the image data (an object reflected in the image) is inferred by the image data and a neural network will be described as an example.

However, the processing using the neural network is not limited to the above inference processing, and may include, for example, parameter updating processing for each layer of the neural network. In the embodiment described later, the case where the calculation optimization device of the present invention determines the calculation accuracy of each layer of the neural network in the case of performing inference processing will be described as an example, but other processing such as the above parameter update processing will be described. Is also applicable to the present invention.

FIG. 1 is a schematic diagram showing inference processing using a neural network. In FIG. 1, the unit 51 corresponding to a neuron in a neural network is represented by an ellipse. There are one or more units in each layer. Further, the line segment 52 (the line connecting the units in the figure) represents the connection between the units. Further, the arrow 53 (thick arrow pointing to the right in the figure) schematically represents the inference process. Note that FIG. 1 shows an example of a feedforward type neural network in which the input to each unit 51 is the output of the unit in the previous layer, but the input to each unit 51 is not limited to this. For example, when holding time-series information, it is possible to include the output of the unit of the previous layer at the previous time in the input to each unit 51, as in the case of a recurrent type neural network. Even in such a case, the direction of the inference processing is considered to be the direction (forward direction) from the input layer to the output layer. Inference processing performed in a predetermined order from the input layer in this way is also called "forward propagation". In the following description, the input layer is referred to as a 0th layer, and the output layer is referred to as an nth layer.

FIG. 2 is an explanatory diagram showing an example of input / output of the unit 51 and connection with another unit when focusing on one unit 51. FIG. 2A shows an example of input / output of one unit 51, and FIG. 2B shows an example of coupling between the units 51 arranged in two layers. As shown in FIG. 2A, when there are four inputs (x _{1 to x 4} ) and one output (z) for one unit 51, the operation of the unit 51 is, for example, an equation. It is expressed as (1A). Here, f () represents an activation function.

z = f (u) ... (1A)
However, u = a + w ₁ x ₁ + w ₂ x ₂ + w ₃ x ₃ + w ₄ x ₄ ... (1B)

In the formula (1B), a represents an intercept, and w _{1 to w 4} represent parameters such as weights corresponding to each input (x _{1 to x 4).}

On the other hand, as shown in FIG. 2B, when each unit 51 is bonded between the layers arranged in the two layers, when focusing on the subsequent layer, the inside of the layer (the latter layer of the two layers). the output of each unit 51 for input to each unit _(x 1 ~x _4, respectively) of _(z 1 to z _4), for example, be expressed as follows. Note that i is an identifier of a unit in the same layer (i = 1 to 3 in this example).

_{z i = f (u i)} ··· (2A)
However, u _i = a + wi _{, 1} x ₁ + wi _{, 2} x ₂ + wi _{, 3} x ₃ + wi _{, 4} x ₄
... (2B)

In the following, the equation (2B) may be simplified and described as _{u i} = Σwi _{i, k} * x _k. The intercept a was omitted. It is also possible to regard the intercept a as a coefficient (one of the parameters) of the constant term having a value of 1. Here, k represents an identifier of an input to each unit 51 in the layer. More specifically, k can also be said to represent the identifier of another unit that makes the input. At this time when the input to the unit 51 in the layer is only the output of each unit of the preceding layer, the aforementioned simplified ^{_{formula, u i (L) = Σw}} i, k (L) * x k ( It is also possible to write ^L-1). In addition, L represents an identifier of a layer. In these equations, wi _{and k} correspond to the parameters of each unit i in the layer (L layer). More specifically, this parameter corresponds to the weight of the connection between each unit i and the other unit k (connection between units). In the following, the unit is not particularly distinguished, and the function (activation function) that determines the output value of the unit may be simplified and described as z = Σw * x.

In the above example, for each unit 51 of a certain layer, the operation of obtaining the output z from the input x corresponds to the inference processing in that layer.

Before explaining the embodiment of the present invention, an example of a processing device that executes the calculation of a part of the layers of the neural network with low accuracy and the calculation of the remaining layers with high accuracy will be described. FIG. 3 is a schematic diagram showing an example of the above processing apparatus. The processing device 18 includes, for example, a low-precision arithmetic circuit 5, a high-precision arithmetic circuit 6, a first memory 7, a second memory 8, and a third memory 9. The low-precision arithmetic circuit 5, the high-precision arithmetic circuit 6, the first memory 7, the second memory 8, and the third memory 9 are connected via, for example, a bus 10.

In the inference processing, the low-precision arithmetic circuit 5 executes the arithmetic of a part of each layer of the neural network with the first arithmetic accuracy.

The first memory 7 is a memory used when the low-precision arithmetic circuit 5 executes an arithmetic, and the low-precision arithmetic circuit 5 executes an arithmetic while appropriately accessing the first memory 7.

In the inference processing, the high-precision arithmetic circuit 6 executes the arithmetic of the remaining layers of each layer of the neural network with a second arithmetic accuracy higher than the first arithmetic accuracy.

The second memory 8 is a memory used when the high-precision calculation circuit 6 executes a calculation, and the high-precision calculation circuit 6 executes the calculation while appropriately accessing the second memory 8.

The first memory 7 and the second memory 8 may be realized by different memories or may be realized by a single memory. When the first memory 7 and the second memory 8 are realized by a single memory, the single memory is divided into an access area of the low-precision arithmetic circuit 5 and an access area of the high-precision arithmetic circuit 6. It suffices if it is done.

Further, the third memory 9 is a data transfer memory used when the low-precision arithmetic circuit 5 and the high-precision arithmetic circuit 6 exchange data. The third memory 9 may not be provided. That is, the low-precision arithmetic circuit 5 and the high-precision arithmetic circuit 6 may exchange data by communication without going through the third memory 9 (data transfer memory).

The calculation accuracy of the high-precision calculation circuit 6 (second calculation accuracy) is higher than the calculation accuracy of the low-precision calculation circuit 5 (first calculation accuracy). In addition, the scale of the range and fineness of the numerical data used for the calculation (more specifically, the scale of the range and fineness of the numerical data determined by the handling of the bit width and the decimal point in the arithmetic circuit). , Called "precision" or "calculation precision".

Hereinafter, a case where the arithmetic precision of the low-precision arithmetic circuit 5 is an 8-bit integer arithmetic and the arithmetic precision of the high-precision arithmetic circuit 6 is a 32-bit floating-point arithmetic will be described as an example. However, the calculation accuracy of the low-precision calculation circuit 5 and the calculation accuracy of the high-precision calculation circuit 6 are not limited to this example, and if the calculation accuracy of the high-precision calculation circuit 6 is higher than the calculation accuracy of the low-precision calculation circuit 5. good.

The low-precision arithmetic circuit 5 and the high-precision arithmetic circuit 6 are mounted on, for example, a GPU (Graphics Processing Unit).

FIG. 4 is a schematic configuration diagram showing an example of a low-precision arithmetic circuit 5. As illustrated in FIG. 4, the low-precision arithmetic circuit 5 may have, for example, a configuration in which a plurality of MACs (Multiplier-Accumulator) 221s are connected in parallel.

Similarly, the high-precision arithmetic circuit 6 may also have a configuration in which a plurality of MACs are connected in parallel, as illustrated in FIG. However, the calculation accuracy of the MAC provided in the high-precision calculation circuit 6 is higher than the calculation accuracy of the MAC 221 provided in the low-precision calculation circuit 5.

The MAC is an example of an arithmetic unit provided in the low-precision arithmetic circuit 5 and the high-precision arithmetic circuit 6.

FIG. 5 is a block diagram showing a configuration example of MAC221. The MAC 221 may include a multiplier 234, an adder 235, storage elements 231 to 233 holding three inputs, and a storage element 236 holding one output. The MAC221 illustrated in FIG. 5 is an arithmetic circuit that calculates one output variable z = a + w * x when it receives three variables a, w, and x. In this example, z corresponds to the output of the unit, a and w correspond to the parameters, and x corresponds to the input of the unit. The MAC221 receives the three variables w, x, and a via the storage elements 231, 232, and 233, respectively. The calculated z is sent to the outside via the storage element 236. In such a configuration, the calculation accuracy of the MAC 221 is determined by the bit width of the multiplier 234 and the adder 235 and the handling of the decimal point (floating point or fixed point, etc.). For example, in the MAC 221 provided in the low-precision arithmetic circuit 5, the arithmetic by the multiplier 234 and the adder 235 may correspond to the arithmetic precision of the low-precision arithmetic circuit 5 (for example, 8-bit integer arithmetic).

The MAC provided in the high-precision arithmetic circuit 6 can also be represented in the same manner as the configuration shown in FIG. However, in the MAC provided in the high-precision arithmetic circuit 6, the arithmetic by the multiplier 234 and the adder 235 corresponds to the arithmetic precision of the high-precision arithmetic circuit 6 (for example, 32-bit floating-point arithmetic).

The configurations of the low-precision arithmetic circuit 5 and the high-precision arithmetic circuit 6 are not limited to the configurations illustrated in FIG. The low-precision arithmetic circuit 5 and the high-precision arithmetic circuit 6 may be realized by a configuration different from the configuration shown in FIG. For example, the low-precision arithmetic circuit 5 and the high-precision arithmetic circuit 6 may be configured to include an arithmetic unit other than the MAC.

FIG. 6 is a flowchart showing an example of the processing progress of the inference processing using the neural network.

When the input data is given to the low-precision arithmetic circuit 5 (step S111), the low-precision arithmetic circuit 5 propagates forward from the first layer to the (k-1) layer of the neural network with the first arithmetic accuracy. (Step S112). That is, the low-precision calculation circuit 5 executes an inference operation for calculating the output of each unit included in each layer from the first layer to the (k-1) layer with the first calculation accuracy.

Next, the low-precision arithmetic circuit 5 saves the arithmetic result of step S112 in the third memory 9 (step S113). Specifically, the low-precision arithmetic circuit 5 stores the output from each unit of the (k-1) layer in the third memory 9.

Next, in the low-precision calculation circuit 5, the high-precision calculation circuit 6 reads the calculation result (output from each unit of the (k-1) layer) of step S112 from the third memory 9 (step S114).

In steps S113 and S114, the low-precision arithmetic circuit 5 and the high-precision arithmetic circuit 6 output data (calculation result of step S112, specifically, output from each unit of the (k-1) layer). 3 It means that the data is exchanged via the memory 9.

The low-precision arithmetic circuit 5 and the high-precision arithmetic circuit 6 may directly exchange data by communication without going through the third memory 9.

After step S114, the high-precision arithmetic circuit 6 performs forward propagation from the kth layer to the nth layer of the neural network with the second arithmetic accuracy (step S115). That is, the high-precision calculation circuit 6 executes an inference operation for calculating the output of each unit included in each layer from the kth layer to the nth layer with the second calculation accuracy.

In the processing process shown in FIG. 6, it is assumed that the input layer of the neural network is the 0th layer and the nth layer is the output layer. Further, it is assumed that the above-mentioned (k-1) layer is an intermediate layer after the input layer (0th layer) and before the output layer (nth layer). That is, k is an integer satisfying 0 <k-1 <n.

It can be said that the output of the nth layer unit obtained in step S115 represents the inference result.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

In the present embodiment, the case where the calculation optimization device of the present invention determines the calculation accuracy of each layer in the neural network will be described as an example. Further, as described above, as a process using a neural network, a process of inferring the content indicated by the given input data will be described as an example. For example, a process in which image data is given and an object represented by the image data (an object reflected in the image) is inferred by the image data and a neural network will be described as an example. However, the present invention can also be applied to other processes using a neural network.

FIG. 7 is a block diagram showing a configuration example of the calculation optimization device of the present invention. The calculation optimization device of the present invention includes a discrimination model storage unit 21, a data storage unit 22, an explanatory variable value acquisition unit 23, an objective function storage unit 24, an objective function calculation unit 25, and a calculation result storage unit 26. , The application pattern determination unit 27 is provided.

The discrimination model storage unit 21 is a storage device that stores a neural network as a discrimination model.

The data storage unit 22 is a storage device that stores data to be inferred by using a neural network (for example, image data in which an object in an image is inferred). The data storage unit 22 stores a plurality of (N) data to be inferred, and also stores correct answer data of the inference result corresponding to each data. For example, the data storage unit 22 stores N image data and correct answer data (data indicating an object actually shown in the image) corresponding to each image data.

The objective function storage unit 24 stores an objective function represented by a predetermined explanatory variable (hereinafter, simply referred to as an explanatory variable). The formula representing the objective function is predetermined. In this embodiment, at least "inference accuracy" and "processing speed" in inference processing using a neural network are used as the above explanatory variables. In the following description, in order to simplify the explanation, first, a case where "inference accuracy" and "processing speed" are used as explanatory variables will be described. The objective function may be represented by other explanatory variables in addition to "inference accuracy" and "processing speed", but this case will be described later.

Here, the "inference accuracy" is the accuracy of the operation result (in other words, the inference result) of the inference process.

The objective function storage unit 24 stores, for example, a function represented by the following equation (3) as an objective function.

Objective function = "inference accuracy" x α + "processing speed" x β ... (3)

"Inference accuracy" and "processing speed" are explanatory variables. α is a coefficient of “inference accuracy” and β is a coefficient of “processing speed”. The values of α and β are predetermined. In the present embodiment, the case where both α and β are defined as positive values will be described as an example.

The explanatory variable value acquisition unit 23 acquires the value of the explanatory variable used in the objective function stored in the objective function storage unit 24. In this example, the values of "inference accuracy" and "processing speed" in the inference processing using the neural network are acquired.

Further, the explanatory variable value acquisition unit 23 stores a plurality of types of application patterns in advance. The application pattern is an operation using a discrimination model (in this embodiment, a neural network), in which the low-precision arithmetic circuit 5 (see FIG. 3) is applied to which layer of the neural network, and the high-precision arithmetic circuit 6 (FIG. 3). (See) is information that defines which layer of the neural network to apply. In this embodiment, the low-precision arithmetic circuit 5 is applied to the first and subsequent layers, and the circuit applied to the layer is switched from the low-precision arithmetic circuit 5 to the high-precision arithmetic circuit 6 between any layers. May be acceptable. However, for the sake of simplicity, the case where the switching is performed once at the maximum will be described as an example. Further, the high-precision arithmetic circuit 6 may be applied to all the layers after the first layer.

Therefore, in the present embodiment, the low-precision arithmetic circuit 5 is applied from the first layer to the p-th layer, the high-precision arithmetic circuit 6 is applied from the p + 1 layer to the q-th layer, and the q + 1 layer to the n-th layer. The case where the low-precision arithmetic circuit 5 is applied again by (output layer) is excluded from the application pattern. However, in the present invention, such a case may be included in the application pattern.

FIG. 8 is a schematic diagram showing an example of an application pattern. Each rectangle shown in FIG. 8 represents each layer of the neural network.

The application pattern 1 shown in FIG. 8 defines that the low-precision arithmetic circuit 5 is applied to all layers from the first layer to the nth layer. In other words, the application pattern 1 defines that the low-precision arithmetic circuit 5 executes the arithmetic of all the layers from the first layer to the nth layer.

The application pattern 2 shown in FIG. 8 defines that the low-precision arithmetic circuit 5 is applied to each layer from the first layer to the n-1th layer, and the high-precision arithmetic circuit 6 is applied to the nth layer. In other words, the application pattern 2 defines that the low-precision arithmetic circuit 5 executes the arithmetic of each layer from the first layer to the n-1th layer, and the high-precision arithmetic circuit 6 executes the arithmetic of the nth layer. ing.

In the application pattern 3 shown in FIG. 8, the low-precision arithmetic circuit 5 is applied to each layer from the first layer to the n-2th layer, and the high-precision arithmetic circuit 6 is applied to the n-1th layer and the nth layer. Is defined.

The application pattern X-1 shown in FIG. 8 defines that the low-precision arithmetic circuit 5 is applied to the first layer and the high-precision arithmetic circuit 6 is applied to each layer from the second layer to the nth layer. In other words, the application pattern X-1 defines that the low-precision arithmetic circuit 5 executes the arithmetic of the first layer, and the high-precision arithmetic circuit 6 executes the arithmetic of each layer from the second layer to the nth layer. ing.

The application pattern X shown in FIG. 8 defines that the high-precision arithmetic circuit 6 is applied to all the layers from the first layer to the nth layer. In other words, the application pattern X defines that the high-precision arithmetic circuit 6 executes the arithmetic of all the layers from the first layer to the nth layer.

Various application patterns as illustrated in FIG. 8 are predetermined, and the explanatory variable value acquisition unit 23 stores each application pattern in advance. Then, the explanatory variable value acquisition unit 23 acquires the value of the explanatory variable “inference accuracy” and the value of the explanatory variable “processing speed” for each application pattern.

Different application patterns have different values for the explanatory variables (in this example, "inference accuracy" and "processing speed").

There are two modes in which the explanatory variable value acquisition unit 23 acquires the values of the explanatory variables (in this example, “inference accuracy” and “processing speed”). In the first aspect, the explanatory variable value acquisition unit 23 causes an actually existing processing device 18 (see FIG. 3) to execute inference processing, and acquires the values of "inference accuracy" and "processing speed" by actual measurement. Is. The second aspect is an aspect in which the explanatory variable value acquisition unit 23 acquires the values of "inference accuracy" and "processing speed" by simulation. That is, the first aspect is an aspect in which the value of the explanatory variable is acquired by actual measurement, and the second aspect is an embodiment in which the value of the explanatory variable is acquired by simulation.

When the explanatory variable value acquisition unit 23 acquires the value of the explanatory variable by actual measurement, the arithmetic optimization device may include a processing device 18 as shown in FIG. Since the configuration and operation of the processing device 18 have already been described with reference to FIG. 3 and the like, the description thereof will be omitted here.

Further, it is possible that the processing device 18 is still in the design stage and the processing device 18 does not actually exist yet. In that case, as shown in FIG. 10, the calculation optimization device may include a design information storage unit 19. The design information storage unit 19 is a storage device that stores the design information of the processing device 18. As an example of design information, the number of arithmetic units (for example, MAC) provided in the low-precision arithmetic circuit 5 in the processing device 18 and the arithmetic units (for example, MAC) provided in the high-precision arithmetic circuit 6 in the processing apparatus 18. The number of However, the design information is not limited to these examples. The explanatory variable value acquisition unit 23 may acquire the value of the explanatory variable by simulation based on the design information stored in the design information storage unit 19.

First, the operation when the explanatory variable value acquisition unit 23 acquires the value of the explanatory variable by actual measurement will be described. Here, as shown in FIG. 9, a case where the arithmetic optimization apparatus includes the processing apparatus 18 will be described as an example.

The operation of the explanatory variable value acquisition unit 23 to acquire the value of the "processing speed" by actual measurement will be described. The explanatory variable value acquisition unit 23 specifies an application pattern for the processing device 18. Then, the explanatory variable value acquisition unit 23 inputs the neural network stored in the discrimination model storage unit 21 and the data stored in the data storage unit 22 into the processing device 18, and infer processing is performed in the processing device 18. Is executed, and the processing speed when the processing device 18 performs inference processing on the data may be measured. As a result, the explanatory variable value acquisition unit 23 acquires the processing speed value. Further, at this time, the processing device 18 executes the inference processing by the operation according to the designated application pattern.

The processing speed is, for example, the inference processing time for one data (in other words, the reciprocal of the number of data that can be processed per second). Alternatively, the explanatory variable value acquisition unit 23 may acquire, for example, a latency or throughput value as the processing speed value. This point is the same even when the processing speed value is acquired by simulation.

The explanatory variable value acquisition unit 23 can acquire the value of the processing speed by causing the processing device 18 to execute the inference processing for one data.

The explanatory variable value acquisition unit 23 sequentially changes the designated application pattern, and acquires the processing speed value by actual measurement for each application pattern.

The operation of the explanatory variable value acquisition unit 23 to acquire the value of "inference accuracy" by actual measurement will be described. When the value of the inference accuracy is acquired by actual measurement, the explanatory variable value acquisition unit 23 may operate as follows, for example. The explanatory variable value acquisition unit 23 specifies an application pattern for the processing device 18. Then, the explanatory variable value acquisition unit 23 inputs the neural network stored in the discrimination model storage unit 21 to the processing device 18. Further, the explanatory variable value acquisition unit 23 inputs a plurality of (N) data stored in the data storage unit 22 to the processing device 18, and inputs each data to the processing device 18. Have the inference result derived. That is, the explanatory variable value acquisition unit 23 causes the processing device 18 to execute the inference processing N times. At this time, the processing device 18 executes the inference processing by the operation according to the designated application pattern. As a result, N inference results are obtained. The explanatory variable value acquisition unit 23 collates the correct answer data stored in the data storage unit 22 with each inference result, and determines the ratio of the number of inference processes for which the correct answer data is obtained to the number of inference processes N times. Calculate and then calculate the inverse of the ratio. The reciprocal of the ratio corresponds to the value of inference accuracy. The operation of the explanatory variable value acquisition unit 23 to acquire the inference accuracy value by actual measurement is not limited to the above example.

The explanatory variable value acquisition unit 23 sequentially changes the designated application pattern, and acquires the inference accuracy value by actual measurement for each application pattern.

Next, the operation when the explanatory variable value acquisition unit 23 acquires the value of the explanatory variable by simulation will be described. Here, as shown in FIG. 10, a case where the calculation optimization device includes the design information storage unit 19 will be described as an example. In this example, the number of arithmetic units (for example, MAC) provided in the low-precision arithmetic circuit 5 (see FIG. 3) in the processing device 18 and the high-precision arithmetic circuit 6 (see FIG. 3) in the processing apparatus 18 are provided. It is assumed that the number of arithmetic units (for example, MAC) to be used is stored in the design information storage unit 19 as design information.

The operation of the explanatory variable value acquisition unit 23 acquiring the value of the "processing speed" by simulation will be described. In this example, the explanatory variable value acquisition unit 23 holds, for example, a function for obtaining the value of the “processing speed” (hereinafter, referred to as a processing speed function) in advance. The processing speed function is predetermined. The processing speed function is, for example, the number of arithmetic units provided in the low-precision arithmetic circuit 5, the number of arithmetic units provided in the high-precision arithmetic circuit 6, and the memory access when the low-precision arithmetic circuit 5 accesses the first memory 7. Amount (number of memory accesses), amount of memory access when the high-precision arithmetic circuit 6 accesses the second memory 8 (number of memory accesses), and exchanged between the low-precision arithmetic circuit 5 and the high-precision arithmetic circuit 6. The amount of data (hereinafter, may be referred to as the amount of data exchanged) is used as a variable. Hereinafter, the case where the processing speed function is represented by each of the above variables will be described as an example. However, the variables used in the processing speed function are not limited to the above example.

The amount of data exchanged is represented by, for example, the product of the number of data exchanged and the number of bytes per data. The unit in this case is, for example, bytes.

The explanatory variable value acquisition unit 23 may calculate the value of the processing speed by substituting the value of each of the above variables into the processing speed function. Here, among the variables, the values defined in the design information may be used for the number of arithmetic units provided in the low-precision arithmetic circuit 5 and the number of arithmetic units provided in the high-precision arithmetic circuit 6. The amount of memory access (number of memory accesses) when the low-precision calculation circuit 5 accesses the first memory 7, the amount of memory access (number of memory accesses) when the high-precision calculation circuit 6 accesses the second memory 8, and Regarding the amount of data exchanged between the low-precision calculation circuit 5 and the high-precision calculation circuit 6, the explanatory variable value acquisition unit 23 selects an application pattern, and the processing device 18 is determined from the design information stored in the design information storage unit 19. It may be derived by simulating the operation according to the selected application pattern. The explanatory variable value acquisition unit 23 processes the number of the above arithmetic units, the amount of memory access derived based on the selected application pattern, and the amount of data exchanged between the low-precision arithmetic circuit 5 and the high-precision arithmetic circuit 6. The value of the processing speed may be calculated by substituting it into the speed function. As a result, the explanatory variable value acquisition unit 23 can acquire the processing speed value based on the simulation.

The explanatory variable value acquisition unit 23 sequentially changes the application pattern to be selected, and calculates the processing speed value based on the simulation for each application pattern.

The operation of the explanatory variable value acquisition unit 23 acquiring the value of "inference accuracy" by simulation will be described. When the value of the inference accuracy is acquired by simulation, the explanatory variable value acquisition unit 23 may operate as follows, for example. The explanatory variable value acquisition unit 23 selects an application pattern. Then, the explanatory variable value acquisition unit 23 is an application pattern selected by the operation of the processing device 18 determined from the design information for each of a plurality of (N in this example) data stored in the data storage unit 22. The inference result for the data is derived by simulating the operation according to. As a result, N inference results are obtained. The explanatory variable value acquisition unit 23 calculates the ratio of the number of inference results that match the correct answer data to the number of inference results (N), and further calculates the reciprocal of the ratio. The reciprocal of the ratio corresponds to the value of inference accuracy. The operation of the explanatory variable value acquisition unit 23 to acquire the inference accuracy value by simulation is not limited to the above example.

The explanatory variable value acquisition unit 23 sequentially changes the designated application pattern, and acquires the inference accuracy value by simulation for each application pattern.

In the present invention, the explanatory variable value acquisition unit 23 may acquire the value of the explanatory variable by actual measurement or by simulation. In any case, the explanatory variable value acquisition unit 23 acquires the values of the explanatory variables (in this example, “inference accuracy” and “processing speed”) for each application pattern.

When the processing speed is expressed by the inference processing time for one data (in other words, the reciprocal of the number of data that can be processed per second) as in the above example, it is preferable that the value indicating the processing speed is small. .. Similarly, the inverse of the ratio of the number of inference processes for which correct answer data was obtained to the number of inference processes of N times (in other words, the ratio of the number of inference results that match the correct answer data to the number of inference results (N)). Even when the inference accuracy is expressed by the inverse number of), the smaller the value indicating the inference accuracy, the more preferable.

The objective function calculation unit 25 uses the values of the explanatory variables (“inference accuracy” and “processing speed” in this example) calculated by the explanatory variable value acquisition unit 23 for each application pattern as an expression representing the objective function (in this example). , The value of the objective function is calculated by substituting into the above equation (3)). The objective function calculation unit 25 performs a process of calculating the value of the objective function for each application pattern.

The calculation result storage unit 26 is a storage device that stores the value of the objective function calculated for each application pattern. The objective function calculation unit 25 calculates the value of the objective function for each application pattern, and stores the value of the objective function for each application pattern in the calculation result storage unit 26.

As described above, in this example, it is preferable that the value indicating the processing speed is small, and similarly, it is preferable that the value indicating the inference accuracy is small. Therefore, it is preferable that the value of the objective function represented by the equation (3) is smaller. Therefore, it can be said that the application pattern in which the value of the objective function is the minimum is the most preferable application pattern (that is, the optimum application pattern).

The application pattern determination unit 27 refers to the value of the objective function for each application pattern stored in the calculation result storage unit 26, and determines the application pattern in which the value of the objective function is the minimum. As described above, the application pattern that minimizes the value of the objective function is the optimum application pattern.

Here, the application pattern is an operation using a discrimination model (in this embodiment, a neural network), in which the low-precision arithmetic circuit 5 (see FIG. 3) is applied to which layer of the neural network, and the high-precision arithmetic circuit 6 (in this embodiment) (neural network). This is information that defines to which layer of the neural network (see FIG. 3) is applied. Therefore, by determining the application pattern, it is possible to optimize the calculation using the neural network. Then, since it is determined whether the low-precision arithmetic circuit 5 or the high-precision arithmetic circuit 6 is applied to each layer of the neural network, the arithmetic accuracy of each layer of the neural network is determined by the first arithmetic accuracy (for example, for example). It is possible to determine whether to perform 8-bit integer arithmetic by the low-precision arithmetic circuit 5 or second arithmetic accuracy (for example, 32-bit floating-point arithmetic by the high-precision arithmetic circuit 6).

The explanatory variable value acquisition unit 23, the objective function calculation unit 25, and the application pattern determination unit 27 are realized by, for example, a CPU (Central Processing Unit) of a computer that operates according to a calculation optimization program. In this case, the CPU reads the calculation optimization program from the program recording medium such as the program storage device. Then, the CPU may operate as the explanatory variable value acquisition unit 23, the objective function calculation unit 25, and the application pattern determination unit 27 according to the calculation optimization program.

Next, an example of the processing process of the embodiment of the present invention will be described. FIG. 11 is a flowchart showing an example of the processing progress of the calculation optimization device according to the embodiment of the present invention. Here, the objective function storage unit 24 stores the objective function represented by the above equation (3), and the explanatory variable value acquisition unit 23 uses the value of "inference accuracy" and the "processing speed" as the values of the explanatory variables. The case of acquiring the value of "" will be described as an example. In addition, the matters already explained will be omitted as appropriate.

First, the explanatory variable value acquisition unit 23 selects one unselected application pattern from the plurality of application patterns stored in advance (step S1).

Next, the explanatory variable value acquisition unit 23 acquires the value of the explanatory variable in the operation of the inference processing under the application pattern selected in step S1 (step S2). In this example, the explanatory variable value acquisition unit 23 acquires the value of “inference accuracy” and the value of “processing speed” in the operation under the application pattern selected in step S1, respectively.

The explanatory variable value acquisition unit 23 may acquire the value of the explanatory variable by actual measurement, or may acquire the value of the explanatory variable by simulation. The operation of acquiring the "inference accuracy" value and "processing speed" value by actual measurement and the operation of acquiring the "inference accuracy" value and "processing speed" value by simulation have already been explained, so here. The explanation is omitted.

After step S2, the objective function calculation unit 25 determines the value of the explanatory variable (in this example, the value of “inference accuracy” and the value of “processing speed”) acquired in step S2 with respect to the selected application pattern. , The value of the objective function is calculated by substituting it into the equation representing the objective function (in this example, the above equation (3)) (step S3). Then, the objective function calculation unit 25 associates the application pattern selected in step S1 with the value of the objective function and stores it in the calculation result storage unit 26.

Next, the explanatory variable value acquisition unit 23 determines whether or not all the application patterns stored in advance have been selected in step S1 (step S4).

If there is an unselected application pattern (No in step S4), the arithmetic optimization device repeats the processes after step S1.

When all the application patterns are selected (Yes in step S4), the application pattern determination unit 27 refers to the value of the objective function for each application pattern stored in the calculation result storage unit 26, and aims. The application pattern that minimizes the value of the function is determined (step S5). The process ends in step S5.

As described above, by determining the application pattern, it is possible to optimize the operation using the neural network. Then, since it is determined whether the low-precision arithmetic circuit 5 or the high-precision arithmetic circuit 6 is applied to each layer of the neural network, the arithmetic accuracy of each layer of the neural network is determined by the first arithmetic accuracy (for example, for example). It is possible to determine whether to perform 8-bit integer arithmetic by the low-precision arithmetic circuit 5 or second arithmetic precision (for example, 32-bit floating-point arithmetic by the high-precision arithmetic circuit 6).

Further, in the present embodiment, since the application pattern is determined by the above steps S1 to S5, the application pattern can be automatically determined. Therefore, it is possible to automatically determine whether the calculation accuracy of each layer of the neural network is the first calculation accuracy or the second calculation accuracy.

Next, as a modification of the embodiment of the present invention, a case where the objective function is represented by other explanatory variables in addition to "inference accuracy" and "processing speed" will be described. In the description of the modification shown below, the description of the matters already described will be omitted as appropriate.

In addition to "inference accuracy" and "processing speed", the objective function may also be represented by "the amount of data exchanged between the low-precision arithmetic circuit 5 and the high-precision arithmetic circuit 6" as explanatory variables. good. Hereinafter, "the amount of data exchanged between the low-precision arithmetic circuit 5 and the high-precision arithmetic circuit 6" is simply referred to as the amount of data exchanged. As described above, the amount of data exchanged is represented by, for example, the product of the number of data exchanged and the number of bytes per piece of data.

In this example, the objective function storage unit 24 may store, for example, a function represented by the following equation (4) as the objective function.

Objective function = "inference accuracy" x α + "processing speed" x β + "data transfer amount" x γ
... (4)

γ is a coefficient of “data transfer amount” and is predetermined. In this example, the case where γ is defined as a positive value will be described as an example.

In this modification, the explanatory variable value acquisition unit 23 acquires the value of the “data transfer amount” for each application pattern in addition to the “inference accuracy” and the “processing speed”.

The operation of the explanatory variable value acquisition unit 23 to acquire the value of the “data transfer amount” by actual measurement will be described. The explanatory variable value acquisition unit 23 specifies an application pattern for the processing device 18. Then, the explanatory variable value acquisition unit 23 inputs the neural network stored in the discrimination model storage unit 21 and the data stored in the data storage unit 22 into the processing device 18, and infer processing is performed in the processing device 18. , And the amount of data exchanged when the processing device 18 performs inference processing on the data may be measured. As a result, the explanatory variable value acquisition unit 23 acquires the value of the amount of data exchanged. Further, at this time, the processing device 18 executes the inference processing by the operation according to the designated application pattern. The explanatory variable value acquisition unit 23 can acquire the value of the data transfer amount by causing the processing device 18 to execute the inference processing for one data.

The explanatory variable value acquisition unit 23 sequentially changes the designated application pattern, and acquires the value of the data transfer amount by actual measurement for each application pattern.

The operation of the explanatory variable value acquisition unit 23 acquiring the value of the “data transfer amount” by simulation will be described. The explanatory variable value acquisition unit 23 selects an application pattern, and simulates the operation of the processing device 18 determined from the design information stored in the design information storage unit 19 according to the selected application pattern. The value of the transfer amount may be derived. The explanatory variable value acquisition unit 23 can derive the value of the data transfer amount by simulating the operation of the processing device 18 for one data.

The explanatory variable value acquisition unit 23 sequentially changes the application pattern to be selected, and derives the value of the data transfer amount by simulation for each application pattern.

In this modification, the objective function calculation unit 25 substitutes the value of "inference accuracy", the value of "processing speed", and the value of "data transfer amount" into the equation (4) for each application pattern. You just have to calculate the value of the objective function.

Other points are the same as those in the above embodiment.

According to this modification, it is possible to determine whether the calculation accuracy of each layer of the neural network is the first calculation accuracy or the second calculation accuracy in consideration of the “data transfer amount”.

Further, the objective function may be expressed using "the circuit scale of the processing device 18 (hereinafter, simply referred to as the circuit scale)" as an explanatory variable in addition to the "inference accuracy" and the "processing speed".

In this example, the objective function storage unit 24 may store, for example, a function represented by the following equation (5) as the objective function.

Objective function = "inference accuracy" x α + "processing speed" x β + "circuit scale" x δ
... (5)

δ is a coefficient of “circuit scale” and is predetermined. In this example, the case where δ is defined as a positive value will be described as an example.

In the following description, the number of arithmetic units (for example, MAC) included in the low-precision arithmetic circuit 5 and the number of arithmetic units (for example, MAC) included in the high-precision arithmetic circuit 6 are included in the low-precision arithmetic circuit 5. A case where the value expressed with reference to the arithmetic unit or the arithmetic unit included in the high-precision arithmetic circuit 6 is referred to as “circuit scale” will be described as an example. In this example, it will be described assuming that the arithmetic unit included in the low-precision arithmetic circuit 5 is used as a reference. When the arithmetic unit included in the low-precision arithmetic circuit 5 is used as a reference, the value corresponding to the number of arithmetic units included in the high-precision arithmetic circuit 6 corresponds to the number of arithmetic units included in the low-precision arithmetic circuit 5. Converted to and expressed. Further, how many arithmetic units included in the high-precision arithmetic circuit 6 correspond to one arithmetic unit included in the low-precision arithmetic circuit 5 is determined by the number of arithmetic units included in the high-precision arithmetic circuit 6. It may be obtained depending on how many arithmetic units included in the low-precision arithmetic circuit 5 occupy the occupied area. Hereinafter, for the sake of simplicity, one arithmetic unit included in the high-precision arithmetic circuit 6 will be described as corresponding to J arithmetic units included in the low-precision arithmetic circuit 5.

In this modification, the explanatory variable value acquisition unit 23 acquires the value of the “circuit scale” in addition to the “inference accuracy” and the “processing speed”.

The operation of the explanatory variable value acquisition unit 23 to acquire the value of the "circuit scale" by actual measurement will be described. When the processing device 18 exists, the number of arithmetic units included in the low-precision arithmetic circuit 5 in the processing apparatus 18, the number of arithmetic units included in the high-precision arithmetic circuit 6, and the high-precision arithmetic circuit 6 Information on how many arithmetic units included in the low-precision arithmetic circuit 5 correspond to one included arithmetic unit is known information. It is assumed that the explanatory variable value acquisition unit 23 stores this known information in advance, for example. Further, for the sake of simplicity, one arithmetic unit included in the high-precision arithmetic circuit 6 will be described as corresponding to J arithmetic units included in the low-precision arithmetic circuit 5.

In this case, the explanatory variable value acquisition unit 23 may calculate the value of the “circuit scale” by the calculation of the following equation (6).

Circuit scale = "Number of arithmetic units included in low-precision arithmetic circuit 5" +
"Number of arithmetic units included in high-precision arithmetic circuit 6" x J
... (6)

In the above example, since the value of the circuit scale does not depend on the application pattern, the explanatory variable value acquisition unit 23 may calculate the value of the circuit scale as a value common to each application pattern.

The operation in which the explanatory variable value acquisition unit 23 acquires the value of the “circuit scale” by simulation will be described. In this case, the design information stored in the design information storage unit 19 (see FIG. 10) includes the design value of the number of arithmetic units included in the low-precision arithmetic circuit 5 and the design of the number of arithmetic units included in the high-precision arithmetic circuit 6. It suffices to include the value and the design value of how many arithmetic units included in the low-precision arithmetic circuit 5 correspond to one arithmetic unit included in the high-precision arithmetic circuit 6. Also in this example, one arithmetic unit included in the high-precision arithmetic circuit 6 will be described as corresponding to J arithmetic units included in the low-precision arithmetic circuit 5.

In this case, the explanatory variable value acquisition unit 23 may calculate the value of the "circuit scale" by the calculation of the following equation (7).

Circuit scale = "Design value of the number of arithmetic units included in the low-precision arithmetic circuit 5" +
"Design value of the number of arithmetic units included in the high-precision arithmetic circuit 6" x J
... (7)

In the above example, since the value of the circuit scale does not depend on the application pattern, the explanatory variable value acquisition unit 23 may calculate the value of the circuit scale as a value common to each application pattern.

Further, when the value of "circuit scale" is acquired by simulation, the explanatory variable value acquisition unit 23 uses a value different from the value represented by the number of arithmetic units based on the arithmetic units included in the low-precision arithmetic circuit 5. You may ask. For example, the explanatory variable value acquisition unit 23 may calculate the value of the circuit scale by a function for obtaining the value of the “circuit scale” (hereinafter, referred to as a circuit scale function). In this case, the explanatory variable value acquisition unit 23 holds the circuit scale function in advance. Further, the circuit scale function is predetermined. The circuit scale function is, for example, the number of arithmetic units provided in the low-precision arithmetic circuit 5, the number of arithmetic instruments provided in the high-precision arithmetic circuit 6, and the first memory 7 accessed by the low-precision arithmetic circuit 5 (see FIG. 3). Memory size, memory size of the second memory 8 (see FIG. 3) accessed by the high-precision arithmetic circuit 6, and data transfer amount (data exchanged between the low-precision arithmetic circuit 5 and the high-precision arithmetic circuit 6). Amount) is a variable. Hereinafter, the case where the circuit scale function is represented by each of the above variables will be described as an example. However, the variables used in the circuit scale function are not limited to the above example. The memory size of the first memory 7 and the memory size of the second memory 8 may be stored in the design information storage unit 19 as design information.

The explanatory variable value acquisition unit 23 may calculate the value of the circuit scale by substituting the value of each of the above variables into the circuit scale function. Here, among the variables, the number of arithmetic units provided in the low-precision arithmetic circuit 5, the number of arithmetic units provided in the high-precision arithmetic circuit 6, the memory size of the first memory 7, and the memory size of the second memory 8. May use the value specified in the design information. Regarding the amount of data exchanged, the explanatory variable value acquisition unit 23 selects an application pattern, and the operation of the processing device 18 determined from the design information stored in the design information storage unit 19 is simulated according to the selected application pattern. By doing so, it may be derived. If the explanatory variable value acquisition unit 23 calculates the value of the circuit scale by substituting the number of the above arithmetic units, the memory size, and the data transfer amount derived based on the selected application pattern into the circuit scale function. good. Further, in this case, the explanatory variable value acquisition unit 23 sequentially changes the application pattern to be selected, and calculates the value of the circuit scale based on the simulation for each application pattern.

In this modification, the objective function calculation unit 25 assigns the value of "inference accuracy", the value of "processing speed", and the value of "circuit scale" to the equation (5), thereby performing the purpose for each application pattern. Just calculate the value of the function.

Other points are the same as those in the above embodiment.

According to this modification, it is possible to determine whether the calculation accuracy of each layer of the neural network is the first calculation accuracy or the second calculation accuracy in consideration of the “circuit scale”.

Further, the objective function may be expressed with "power consumption of the processing device 18 (hereinafter, simply referred to as power consumption)" as an explanatory variable in addition to "inference accuracy" and "processing speed".

In this example, the objective function storage unit 24 may store, for example, a function represented by the following equation (8) as the objective function.

Objective function = "inference accuracy" x α + "processing speed" x β + "power consumption" x ε
... (8)

ε is a coefficient of "power consumption" and is predetermined. In this example, the case where ε is defined as a positive value will be described as an example.

In this modification, the explanatory variable value acquisition unit 23 acquires the value of "power consumption" in addition to the "inference accuracy" and "processing speed" for each application pattern.

The operation of the explanatory variable value acquisition unit 23 to acquire the value of "power consumption" by actual measurement will be described. The explanatory variable value acquisition unit 23 specifies an application pattern for the processing device 18. Then, the explanatory variable value acquisition unit 23 inputs the neural network stored in the discrimination model storage unit 21 and the data stored in the data storage unit 22 into the processing device 18, and infer processing is performed in the processing device 18. Is executed, and the power consumption when the processing device 18 performs inference processing on the data may be measured. As a result, the explanatory variable value acquisition unit 23 acquires the value of the power consumption. Further, at this time, the processing device 18 executes the inference processing by the operation according to the designated application pattern. The explanatory variable value acquisition unit 23 can acquire the value of the power consumption by causing the processing device 18 to execute the inference processing for one data.

The explanatory variable value acquisition unit 23 sequentially changes the designated application pattern, and acquires the value of the power consumption by actual measurement for each application pattern.

The operation of the explanatory variable value acquisition unit 23 acquiring the value of "power consumption" by simulation will be described. When deriving the value of "power consumption" by simulation, the data necessary for deriving the power consumption value defined in the design stage is included in the design information stored in the design information storage unit 19. The explanatory variable value acquisition unit 23 selects an application pattern and consumes it by simulating the operation of the processing device 18 determined from the design information stored in the design information storage unit 19 according to the selected application pattern. The value of power may be derived. The explanatory variable value acquisition unit 23 can derive the value of the power consumption by simulating the operation of the processing device 18 for one data.

The explanatory variable value acquisition unit 23 sequentially changes the application pattern to be selected, and derives the power consumption value by simulation for each application pattern.

In this modification, the objective function calculation unit 25 substitutes the value of "inference accuracy", the value of "processing speed", and the value of "power consumption" into the equation (8), thereby performing the purpose for each application pattern. Just calculate the value of the function.

Other points are the same as those in the above embodiment.

According to this modification, it is possible to determine whether the calculation accuracy of each layer of the neural network is the first calculation accuracy or the second calculation accuracy in consideration of "power consumption".

In each of the above variants, the objective function is represented in addition to "inference accuracy" and "processing speed", with any of "data transfer amount", "circuit scale", and "power consumption" as explanatory variables. I explained the case. The objective function is represented by any one or more explanatory variables of "data transfer amount", "circuit scale" and "power consumption" in addition to "inference accuracy" and "processing speed". May be good.

The objective function may be expressed with "inference accuracy", "processing speed", "data transfer amount", "circuit scale", and "power consumption" as explanatory variables. In this case, the objective function storage unit 24 may store, for example, a function represented by the following equation (9) as the objective function.

Objective function = "inference accuracy" x α + "processing speed" x β + "data transfer amount" x γ
+ "Circuit scale" x δ + "Power consumption" x ε
... (9)

In this case, the explanatory variable value acquisition unit 23 determines each explanatory variable (“inference accuracy”, “processing speed”, “data transfer amount”, “circuit scale”, and “power consumption”) by actual measurement or simulation. The value may be acquired for each application pattern.

Further, the objective function calculation unit 25 may calculate the value of the objective function for each application pattern by substituting the value of each explanatory variable acquired by the explanatory variable value acquisition unit 23 into the equation (9).

In this case, whether the calculation accuracy of each layer of the neural network is the first calculation accuracy or the second calculation accuracy in consideration of "data transfer amount", "circuit scale", and "power consumption". Can be decided.

It should be noted that the term “data transfer amount” × γ may not be included in the equation (9). In this case, the explanatory variable value acquisition unit 23 does not have to acquire the value of the “data transfer amount”.

Further, the term “circuit scale” × δ may not be included in the equation (9). In this case, the explanatory variable value acquisition unit 23 does not have to acquire the value of the “circuit scale”.

Further, in the equation (9), the term “power consumption” × ε may not be provided. In this case, the explanatory variable value acquisition unit 23 does not have to acquire the value of "power consumption".

FIG. 12 is a schematic block diagram showing a configuration example of a computer according to an embodiment of the present invention or a modification thereof. The computer 1000 includes a CPU 1001, a main storage device 1002, an auxiliary storage device 1003, and an interface 1004.

The arithmetic optimization device of the present invention is mounted on the computer 1000. The operation of the calculation optimization device is stored in the auxiliary storage device 1003 in the form of a calculation optimization program. The CPU 1001 reads the calculation optimization program from the auxiliary storage device 1003, deploys it to the main storage device 1002, and executes the processing described in the above embodiment and its modification according to the calculation optimization program.

Auxiliary storage 1003 is an example of a non-temporary tangible medium. Other examples of non-temporary tangible media include magnetic disks, magneto-optical disks, CD-ROMs (Compact Disk Read Only Memory), DVD-ROMs (Digital Versatile Disk Read Only Memory), which are connected via interface 1004. Examples include semiconductor memory. When the program is distributed to the computer 1000 by the communication line, the distributed computer 1000 may expand the program to the main storage device 1002 and execute the above processing.

Further, the program may be for realizing a part of the above-mentioned processing. Further, the program may be a difference program that realizes the above-mentioned processing in combination with another program already stored in the auxiliary storage device 1003.

Further, a part or all of each component may be realized by a general-purpose or dedicated circuitry, a processor, or a combination thereof. These may be composed of a single chip or may be composed of a plurality of chips connected via a bus. A part or all of each component may be realized by the combination of the circuit or the like and the program described above.

When a part or all of each component is realized by a plurality of information processing devices and circuits, the plurality of information processing devices and circuits may be centrally arranged or distributed. For example, the information processing device, the circuit, and the like may be realized as a form in which each is connected via a communication network, such as a client-and-server system and a cloud computing system.

Next, the outline of the present invention will be described. FIG. 13 is a block diagram showing an outline of the calculation optimization device of the present invention. The arithmetic optimization apparatus of the present invention includes explanatory variable value acquisition means 73, objective function calculation means 75, and application pattern determination means 77.

The explanatory variable value acquisition means 73 (for example, the explanatory variable value acquisition unit 23) is an operation using a discrimination model (for example, a neural network) in which a plurality of layers each composed of one or more units are connected. A second calculation in which the first calculation circuit (for example, the low-precision calculation circuit 5) that performs the calculation with the calculation accuracy of 1 is applied to which layer, and the calculation is performed with the second calculation accuracy higher than the first calculation accuracy. The value of a predetermined explanatory variable is acquired for each application pattern, which is information that defines to which layer the circuit (for example, the high-precision arithmetic circuit 6) is applied.

The objective function calculation means 75 (for example, the objective function calculation unit 25) calculates the value of the objective function represented by a predetermined explanatory variable for each application pattern.

The application pattern determination means 77 (for example, the application pattern determination unit 27) determines the application pattern that minimizes the value of the objective function.

With such a configuration, the calculation accuracy in each layer of the discrimination model can be automatically determined so that the calculation using the discrimination model can be optimized.

The objective function may at least represent the processing speed of the operation using the discriminant model and the accuracy of the operation result as predetermined explanatory variables.

The objective function may represent the amount of data exchanged between the first arithmetic circuit and the second arithmetic circuit as a predetermined explanatory variable.

The objective function may be expressed as a predetermined explanatory variable of the circuit scale of the circuit that performs the calculation using the discrimination model.

The objective function may express the power consumption in the calculation using the discriminant model as a predetermined explanatory variable.

The explanatory variable value acquisition means 73 may be configured to acquire the value of a predetermined explanatory variable by actual measurement.

The explanatory variable value acquisition means 73 may be configured to acquire the value of a predetermined explanatory variable by simulation.

Although the invention of the present application has been described above with reference to the embodiments, the invention of the present application is not limited to the above-described embodiment. Various changes that can be understood by those skilled in the art can be made within the scope of the present invention in terms of the configuration and details of the present invention.

Possibility of industrial use

The present invention is suitably applied to, for example, an operation optimization device that optimizes an operation using a discrimination model such as a neural network.

5 Low-precision arithmetic circuit 6 High-precision arithmetic circuit 7 1st memory 8 2nd memory 9 3rd memory 18 Processing device 19 Design information storage unit 21 Discrimination model storage unit 22 Data storage unit 23 Explanation variable value acquisition unit 24 Objective function storage unit 25 Arbitrary-precision calculation unit 26 Calculation result storage unit 27 Applicable pattern determination unit

Claims (9)

In the calculation using the discrimination model in which a plurality of layers each composed of one or more units are combined, the first calculation circuit that performs the calculation with the first calculation accuracy is applied to which layer, and the first Explanatory variable value acquisition to acquire the value of a predetermined explanatory variable for each application pattern, which is information that defines to which layer the second arithmetic circuit that performs arithmetic with a second arithmetic accuracy higher than the arithmetic accuracy is applied. Means and
An objective function calculation means for calculating the value of the objective function represented by the predetermined explanatory variable for each application pattern,
An arithmetic optimization device including an application pattern determining means for determining an application pattern that minimizes the value of an objective function. The calculation optimization device according to claim 1, wherein the objective function is at least the processing speed of the calculation using the discrimination model and the accuracy of the calculation result as predetermined explanatory variables. The arithmetic optimization device according to claim 2, wherein the objective function represents the amount of data exchanged between the first arithmetic circuit and the second arithmetic circuit as a predetermined explanatory variable. The operation optimization device according to claim 2 or 3, wherein the objective function represents the circuit scale of a circuit that performs an operation using a discrimination model as a predetermined explanatory variable. The calculation optimization device according to any one of claims 2 to 4, wherein the objective function represents the power consumption in the calculation using the discrimination model as a predetermined explanatory variable. The arithmetic optimization device according to any one of claims 1 to 5, wherein the explanatory variable value acquisition means acquires the value of a predetermined explanatory variable by actual measurement. The arithmetic optimization device according to any one of claims 1 to 5, wherein the explanatory variable value acquisition means acquires the value of a predetermined explanatory variable by simulation. In the calculation using a discrimination model in which a plurality of layers each composed of one or more units are combined, the first calculation circuit that performs the calculation with the first calculation accuracy is applied to which layer, and the first The value of a predetermined explanatory variable is acquired for each application pattern, which is information that defines to which layer the second arithmetic circuit that performs the arithmetic with the second arithmetic accuracy higher than the arithmetic accuracy is applied.
The value of the objective function represented by the predetermined explanatory variable is calculated for each application pattern, and the value is calculated.
An arithmetic optimization method characterized by determining the application pattern that minimizes the value of the objective function. On the computer
In the calculation using the discrimination model in which a plurality of layers each composed of one or more units are combined, the first calculation circuit that performs the calculation with the first calculation accuracy is applied to which layer, and the first Explanatory variable value acquisition to acquire the value of a predetermined explanatory variable for each application pattern, which is information that defines to which layer the second arithmetic circuit that performs arithmetic with a second arithmetic accuracy higher than the arithmetic accuracy is applied. process,
Objective function calculation processing that calculates the value of the objective function represented by the predetermined explanatory variable for each application pattern, and
An arithmetic optimization program for executing the application pattern determination process that determines the application pattern that minimizes the value of the objective function.

Images