JP7120308B2 - DATA PROCESSING DEVICE, DATA PROCESSING CIRCUIT AND DATA PROCESSING METHOD - Google Patents

Description

The present invention relates to a data processing device, a data processing circuit, and a data processing method that perform data processing including calculations with two types of calculation accuracy.

With the spread of machine learning, further ingenuity is required to cope with ever-changing situations.

For that reason, it is necessary to incorporate various raw data obtained in the actual environment as learning data into learning. In learning (machine learning) using learning data, for example, parameters of arithmetic expressions and discriminants used in predetermined learning devices are adjusted based on the relationship between input and output indicated by learning data. . A learner is, for example, a discriminant model or the like that discriminates about one or more labels when data is input.

Regarding the relationship between computational resources and computational accuracy in machine learning, for example, Non-Patent Document 1 describes an example of a computational circuit for learning and a learning method for executing deep learning of a neural network efficiently, particularly with low power consumption. Have been described.

In addition, in non-patent document 2, in deep learning in CNN (Convolutional Neural Network), a plurality of convolution layers are divided into a layer whose weight is fixed and a layer whose weight is updated (enhanced function layer), and the learning range An example of a learning method for shortening the learning time by restricting is described.

As an example of circuit configuration for learning calculation in machine learning, Non-Patent Document 3 describes an optimization example of accelerator design based on FPGA (Field-Programmable Gate Array).

Y.H.Chen, et.al., "Eyeriss: an Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks", in IEEE Journal of Slid-State Circuits, vol.52, no.1, Jan. 2017, pp.127-138 . Wei. Liu, et.al., "SSD: Single shot MultiBox Detector", arXiv:1512.02325v5, Dec. 2016. Chen Zhang, et.al., "Optimizing FPGA-based Accelerator Design for Deep convolutional Neural Networks", In ACM FPGA 2015, pp.160-170.

Most of the machine learning using training data is performed in a cloud environment where large-scale high-precision arithmetic circuits can be constructed in order to support general-purpose learning algorithms.

However, depending on the site, there are various restrictions on data movement, such as network bandwidth restrictions and privacy protection. desired. For that purpose, a learning method that can obtain a sufficient recognition rate with less computer resources and lower power consumption is desired.

According to the learning method described in Non-Patent Document 1, compared to NVIDIA's TK1 (Jetson Kit), which performs learning using a 32-bit floating-point arithmetic circuit, by using a 16-bit fixed-point arithmetic circuit, It is said that learning can be realized with low power consumption. However, this method is intended to reduce power consumption in exchange for a decrease in calculation accuracy by reducing the bit width in the calculation circuit that performs all learning calculations (all calculations for adjusting parameters). However, no consideration is given to the adverse effects caused by the lowering of the arithmetic accuracy of the arithmetic circuit itself. For example, no consideration is given to the possibility that sufficient computational accuracy for executing the learning computation may not be ensured.

For example, in an arithmetic circuit that performs deep learning, multi-layered calculations are performed using a structure in which multiple units are connected in layers. Processing (for example, forward propagation processing) and calculation for updating parameters (for example, weights) used in the calculation (so-called parameter update processing (for example, back propagation processing)). It can be said that parameter update processing, in particular, among these processes corresponds to the actual learning calculation part in machine learning. Therefore, the calculation accuracy of the parameter updating process greatly affects the recognition rate during operation, and the higher the accuracy, the better. On the other hand, it is often the case that the calculation accuracy of inference processing does not need to be so high.

Therefore, among the calculations included in the learning process, for example, if only calculations that require high accuracy are performed with high accuracy, and calculations that do not require high accuracy are performed with low accuracy, sufficient power consumption can be achieved while reducing power consumption. Accurate learning becomes possible. Therefore, a device that performs processing that requires a mixture of computations that require high accuracy and computation that does not require high accuracy has arithmetic circuits with two types of computational accuracies. Consider executing the circuit while switching according to the accuracy required for the operation. In that case, data exchange between cores (arithmetic circuits) with different arithmetic accuracies is an essential requirement in the device. In order to further improve the efficiency of learning processing including data exchange between cores with different calculation accuracies, it is important to increase the speed of data exchange between cores.

It should be noted that the learning method described in Non-Patent Document 2 is merely an attempt to shorten the learning time by limiting the learning range. In particular, no consideration is given to the efficiency of data exchange between cores having different accuracies. In addition, the method described in Non-Patent Document 3 is nothing more than an attempt to reduce the circuit scale and calculation time by optimizing the circuit configuration of the circuit that performs all learning calculations. No consideration is given to improving the efficiency of learning processing including data exchange, especially the efficiency of data exchange between cores having different accuracies.

SUMMARY OF THE INVENTION In view of the above problems, the present invention provides a data processing device, a data processing circuit, and a data processing method capable of further improving the efficiency of processing in which operations that require high accuracy and operations that do not require high accuracy are mixed. intended to

A data processing apparatus according to the present invention comprises low-accuracy arithmetic processing means for performing predetermined arithmetic operations with a first precision, high-precision arithmetic processing means for performing predetermined arithmetic operations with a second precision higher than the first precision, and high-precision arithmetic processing means. a first data conversion means provided at an end point on the side of the high-precision arithmetic processing means of a communication path for exchanging data between the precision arithmetic processing means and the low-precision arithmetic processing means; The means is that the data transferred between the high-precision arithmetic processing means of the connection destination is data that can be handled by the high-precision arithmetic processing means, and the amount of data passing through the communication path is data of the first precision. and the accuracy of the data passing through the communication channel is equal to or less than the first accuracy, performing a predetermined conversion on the data passing between the communication channel and the high-precision arithmetic processing means, and communicating a second data conversion means provided at an end point of the low-precision arithmetic processing means side of the path; The data to be transmitted is data that can be handled by the arithmetic processing means of the connection destination, the amount of data passing through the communication channel is smaller than the amount of data when data of the first accuracy is used, and the accuracy of the data passing through the communication channel is It is characterized by performing a predetermined conversion on data passing between the communication path and the arithmetic processing means of the connection destination so as to be lower than the first accuracy .

A data processing circuit according to the present invention comprises a low-precision arithmetic circuit that performs predetermined arithmetic with a first precision, a high-precision arithmetic circuit that performs predetermined arithmetic with a second precision higher than the first precision, and a high-precision arithmetic circuit. It is provided at the end point of the high-precision arithmetic circuit side of the communication path for exchanging data between the circuit and the low-precision arithmetic circuit, and is predetermined for the data passing between the communication path and the high-precision arithmetic circuit. and a first data conversion circuit for performing the conversion, data transferred between the first data conversion circuit and the high-precision arithmetic circuit to which the high-precision arithmetic circuit is connected is data handled by the high-precision arithmetic circuit, and the communication path further comprising a second data conversion circuit provided at an end point on the low-precision arithmetic circuit side of the second data conversion circuit for performing predetermined conversion on data passing between the communication path and the low-precision arithmetic circuit; The data transferred between the conversion circuit and the connected low-precision arithmetic circuit is the data handled by the low-precision arithmetic circuit, and the amount of data passing through the communication channel is the amount of data when the first precision data is used. and the accuracy of data passing through the communication path is lower than the first accuracy .

In the data processing method according to the present invention, a low-precision arithmetic processing means for performing a predetermined arithmetic operation with a first accuracy and a high-precision arithmetic processing means for performing a predetermined arithmetic operation with a second accuracy higher than the first accuracy are provided. The first data conversion means provided at the end point of the high-precision arithmetic processing means side of the communication path for exchanging data with the high-precision arithmetic processing means of the connection destination performs high-precision arithmetic operation on the data transferred between It is data that can be handled by the processing means, and the amount of data passing through the communication channel is equal to or less than the amount of data when data with the first accuracy is used, and the accuracy of the data passing through the communication channel is equal to or less than the first accuracy. Then, the data passing between the communication channel and the high-precision arithmetic processing means is subjected to a predetermined conversion, and the first data conversion means transfers the data to and from the high-precision arithmetic processing means of the connection destination. is data that can be handled by the high-precision arithmetic processing means, the amount of data passing through the communication channel is less than the amount of data when using data of the first precision, and the accuracy of the data passing through the communication channel is the first precision Data passing between the communication path and the high-precision arithmetic processing means is subjected to a predetermined conversion so that the data passing between the communication path and the high-precision arithmetic processing means is further reduced, and the second data conversion means provided at the end point of the communication path on the side of the low-precision arithmetic processing means is , the data transferred between the low-precision arithmetic processing means of the connection destination is data that can be handled by the low-precision arithmetic processing means, and the amount of data passing through the communication path is data in the case of using data of the first precision A predetermined conversion is performed on data passing between the communication channel and the low-precision arithmetic processing means so that the accuracy of the data passing through the communication channel is less than the amount and the accuracy of the data passing through the communication channel is lower than the first accuracy. .

According to the present invention, it is possible to further improve the efficiency of processing in which operations that require high accuracy and operations that do not require high accuracy are mixed.

FIG. 2 is an explanatory diagram showing an outline of a learning method as an example of the data processing method of the present invention; FIG. 4 is an explanatory diagram showing an example of input/output of a certain unit and connection with other units; 1 is a block diagram showing a configuration example of a learning device according to a first embodiment; FIG. 3 is a configuration diagram showing an example of the hardware configuration of a learning processing unit 106; FIG. FIG. 3 is an explanatory diagram showing an example of a combination of arithmetic accuracy in a low-precision arithmetic circuit 11 and arithmetic accuracy in a high-precision arithmetic circuit 12; 2 is a schematic block diagram showing a configuration example of a computer related to the learning device 100; FIG. 1 is a schematic configuration diagram showing an example of an arithmetic circuit; FIG. FIG. 4 is a schematic configuration diagram showing another example of an arithmetic circuit; FIG. 4 is a schematic configuration diagram showing another example of an arithmetic circuit; FIG. 4 is a schematic configuration diagram showing another example of an arithmetic circuit; 4 is a flow chart showing an example of the operation of the learning device 100 of the first embodiment; 4 is a flowchart showing a more specific example of operation of the learning device 100; 7 is a flow chart showing another example of a more specific operation of the learning device 100. FIG. 7 is a flow chart showing another example of a more specific operation of the learning device 100. FIG. It is an explanatory view showing an example of composition of a data processor of a 2nd embodiment. It is an explanatory view showing an example of composition of a data processor of a 2nd embodiment. 1 is a block diagram showing an outline of a data processing device of the present invention; FIG. 1 is a configuration diagram showing the configuration of a data processing circuit of the present invention; FIG. 4 is a configuration diagram showing another configuration of the data processing circuit of the present invention; FIG.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings. Hereinafter, the present invention will be described using learning processing in deep learning as an example of processing in which calculations that require high accuracy and calculations that do not require high accuracy are mixed. The data processing method is not limited to the learning process, learning device and learning method.

First, an outline of learning processing as an example of data processing of the present invention will be described. FIG. 1(a) is an explanatory diagram showing a general learning method in a neural network including one or more intermediate layers between an input layer and an output layer and an example of a circuit configuration therefor; ) is an explanatory diagram showing an example of a learning method and a circuit configuration therefor as an example of the data processing method of the present invention.

In the example shown in FIG. 1(a), a large-scale learning circuit 90 is used to learn the entire neural network, which is a predetermined discriminant model, in order to deal with general-purpose learning algorithms.

In FIG. 1, the balloon attached to the circuit schematically shows the direction and range of processing in the learning process of the neural network. In the balloon, reference numerals 51 (circles in the figure) represent units corresponding to neurons in the neural network. Reference numeral 52 (a line connecting units in the drawing) represents a connection between units. Reference numeral 53 (rightward thick arrow in the figure) represents the inference process and its range. Reference numeral 54 (thick leftward arrow in the figure) represents parameter update processing and its range. Note that FIG. 1 shows an example of a feedforward neural network in which the input to each unit is the output of the unit in the preceding layer, but the input to each unit is not limited to this. For example, when time-series information is held, the input to each unit can include the output of the previous layer unit at the previous time, like a recurrent neural network. Even in such a case, the direction of inference processing is considered to be the direction (forward direction) from the input layer to the output layer. Such inference processing performed in a predetermined order from the input layer is also called “forward propagation”. On the other hand, the direction of parameter update processing is not particularly limited. It may be in the direction (reverse direction) from the output layer to the input layer as in the parameter update processing in the figure. The direction of the parameter updating process in the figure is an example of the error backpropagation method, but the parameter updating process is not limited to the error backpropagation method. For example, the parameter update process may be STDP (Spike Timing Dependent Plasticity) or the like.

Examples of model learning methods in deep learning, not limited to neural networks, include the following learning methods. First, after inputting learning data to the input layer, inference processing is performed to calculate the output of each unit in the forward direction in each layer up to the output layer (forward propagation: see arrow 53 in the figure). Then, based on the error calculated from the output (final output) from the output layer and the relationship between the input and output indicated by the learning data, etc., from the output layer to the first layer so as to minimize the error , and performs parameter update processing for updating parameters for calculating the output of each unit in the layer (back propagation: see arrow 54 in the figure).

As shown in Fig. 1(a), when the entire model is targeted for learning, in the parameter update process, the output of each unit in each layer in all layers (1st to nth layers) after the input layer is Update the parameters for the calculation (eg, the weight of the unit connection that connects each unit in a layer with units in other layers). By repeating such parameter update processing a plurality of times while changing the learning data, for example, a trained model having a high recognition rate can be generated. FIG. 1A shows a large-scale learning circuit 90 that performs the above-described inference processing and parameter updating processing with high accuracy as an implementation example of an arithmetic circuit that performs such learning. However, the higher the calculation accuracy of the inference processing and the parameter update processing, and the wider the calculation range of the processing, the larger the number of expansion terms of the error function and the larger the circuit, resulting in a significant increase in power consumption.

On the other hand, in the present invention, as shown in FIG. 1B, only part of the model is subject to learning. Note that learning here refers to parameter update processing, which is more actual learning processing, as in the above case. If only part of the model is to be learned, forward propagation is performed in the same manner as above. Then, based on the error calculated from the output (final output) from the output layer and the relationship between the input and output indicated by the learning data, a specified unit (for example, the nth layer which is the output layer to the k-th layer), a parameter updating process is performed to update the parameters for calculating the output of the unit (for example, the weight for coupling with other units, etc.).

In FIG. 1B, as an implementation example of the arithmetic circuit 10 that performs such learning, a high-accuracy arithmetic circuit 12 that performs parameter update processing of some units specified with high arithmetic accuracy and a high-accuracy arithmetic circuit 12 An example of combination with a low-accuracy arithmetic circuit 11 that performs inference processing of at least a designated unit with an arithmetic accuracy lower than that is shown. In addition to such arithmetic circuits having two different arithmetic accuracies, the high-precision arithmetic circuit 12 is caused, for example, to perform parameter update processing for some units that require high-precision arithmetic, and low-precision arithmetic The arithmetic circuit 11 is caused to perform other processing that does not require high-precision arithmetic. In this way, in the learning calculation for one piece of learning data, at least part of the inference processing is performed with low calculation accuracy, and at least part of the parameter update processing is performed with high calculation accuracy, and high calculation accuracy is performed. By optimizing the range of parameter update processing performed in , sufficient computational accuracy is ensured while improving the efficiency of computer resources (low power consumption, etc.).

Although FIG. 1(b) shows an example in which some layers on the output side are used as the range for updating parameters (actual learning range), the range for updating parameters is not limited to the layers on the output side. , odd-numbered layers and even-numbered layers among the first to n-th layers can also be specified individually. Also, FIG. 1B shows an example in which the range of the parameter update process itself is limited, but the range of the parameter update process itself is not limited, and the range of the parameter update process performed with high calculation accuracy is limited. good too. In other words, it is possible to perform parameter update processing with high calculation accuracy only for some units among all units, and to perform parameter update processing with low calculation accuracy for other units. It is also possible to divide the target of parameter update processing into three types: units that are executed by high-precision calculations, units that are executed by low-precision calculations, and units that are not executed (in which case the parameters are fixed). is.

Another example of how to divide the processing targeted for high-precision calculations and low-precision calculations is to perform inference processing for all units using low-precision calculations, and perform parameter update processing for all units using high-precision calculations. It is also possible to Further, for example, it is possible to perform the inference processing of all units by low-precision calculations and perform the parameter update processing of some units by high-precision calculations. In this case, the remaining units excluded from high-precision calculation may be subjected to parameter update processing using low-precision calculation, or may be excluded from parameter update processing. Also, for example, it is possible to perform inference processing and parameter update processing with low-precision calculations for some units, and perform inference processing and parameter update processing with high-precision calculations for some remaining units.

In other words, in the learning method as an example of the data processing method of the present invention, the learning device includes a low-accuracy arithmetic circuit having relatively low arithmetic accuracy and a high-accuracy arithmetic circuit having relatively high arithmetic accuracy. , the low-precision arithmetic circuit is caused to perform the inference processing of at least some of the units, and the high-precision arithmetic circuit is made to perform the parameter update processing of at least some of the units. After that, the inference processing of the remaining part of the units may be performed by the low-precision arithmetic circuit or by the high-precision arithmetic circuit. Further, the parameter update processing of the remaining part of the units may be performed by a low-precision arithmetic circuit, or the processing itself may be omitted. Which units are targeted for high-precision inference processing or low-precision inference processing, and which units are targeted for high-precision parameter update processing or low-precision parameter update processing There is no particular limitation as to whether or not to be processed.

The above is an example of using two arithmetic circuits with different arithmetic accuracies, but basically the same applies to the case of using two or more arithmetic circuits with different arithmetic accuracies, for example. That is, if the configuration is such that the parameter update processing of some units is performed in an arithmetic circuit having a higher arithmetic accuracy than the arithmetic circuit that performs the inference processing of some units, other There is no particular limitation as to which arithmetic circuit specifically performs the inference processing and parameter update processing of some units or whether the processing itself is not performed.

FIG. 2 is an explanatory diagram showing an example of input/output of a unit and connection with other units when focusing attention on one unit. FIG. 2A shows an example of input/output of one unit, and FIG. 2B shows an example of connection between units arranged in two layers. As shown in FIG. 2(a), when there are four inputs (x ₁ to x ₄ ) and one output (z) for one unit, the operation of the unit is expressed by equation (1A). is represented as where f() represents the activation function.

z=f(u) (1A)
However, u= _a +w1x1 _{+ w2x2 + w3x3} + _{w4x4 ( 1B} )

In equation (1B), a is an intercept, and w ₁ to w ₄ are parameters such as weights corresponding to each input (x ₁ to x ₄ ).

On the other hand, as shown in FIG. 2(b), when each unit is connected between layers arranged in two layers, focusing on the latter layer, the input to each unit in the layer (each x ₁ to x ₄ ) of each unit (z ₁ to z ₄ ) are expressed as follows, for example. Note that i is an identifier of a unit in the same layer (i=1 to 3 in this example).

z _i =f(u _i ) (2A)
However, u _i =a+wi _,1 x ₁ +wi _,2 x ₂ +wi _,3 x ₃ +wi _,4 x ₄ (2B)

Equation (2B) may be simplified below as z _i =Σw _i,k *x _k . In addition, the intercept a is omitted. It is also possible to regard the intercept a as a constant term coefficient (one of the parameters) with a value of 1. Here, k represents the input to each unit in the layer, more specifically, the identifier of another unit that performs the input. At this time, when the input to each unit in the layer is only the output of each unit in the previous layer, the above simplified expression is u _i ^(L) =Σw _i,k ^(L) * z _k ^{( L-1)} can also be written. Note that L represents a layer identifier. In these expressions, wi _,k is the parameter of each unit i in the layer (L-th layer), more specifically, the weight of the connection (inter-unit connection) between each unit i and another unit k. Equivalent to. In the following description, a function (activation function) that determines the output value of a unit may be simply expressed as z=Σw*x without distinguishing between units.

In the above example, the calculation for obtaining the output z from the input x for a certain unit corresponds to the inference processing in that unit. At this time, the parameter w is fixed. On the other hand, the calculation for obtaining the parameter w for a certain unit corresponds to parameter update processing for that unit.

Embodiment 1.
FIG. 3 is a block diagram showing a configuration example of the learning device of the first embodiment. The learning device 100 shown in FIG. 3 includes a pre-learning model storage unit 101 , a learning data storage unit 102 , a learning processing unit 106 , and a post-learning model storage unit 107 .

The pre-learning model storage unit 101 stores information on the pre-learning model. The information of the model before learning may include the initial values of the parameters.

The learning data storage unit 102 stores learning data that is used for model learning. Note that the format of the learning data is not particularly limited.

The learning processing unit 106 uses the learning data stored in the learning data storage unit 102 to learn the model stored in the pre-learning model storage unit 101 .

The learning processing unit 106 of this embodiment includes at least a high-efficiency inference processing unit 103a, a high-precision parameter update processing unit 104b, and a control unit 105. FIG. Note that the learning processing unit 106 may further include a high-accuracy inference processing unit 103b and a high-efficiency parameter update processing unit 104a, as shown in FIG.

The high-efficiency inference processing unit 103a performs inference processing on the specified layer or unit with the first calculation accuracy.

The high-precision parameter update processing unit 104b performs parameter update processing for a designated layer, unit, or parameter with a second calculation precision higher than the first calculation precision.

The control unit 105 controls each processing unit (in this example, the high efficiency inference processing unit 103a, the high accuracy inference processing unit 103b, the high efficiency parameter update processing unit 104a and the high accuracy parameter update processing unit 104b) that performs the learning process. to perform the required learning process. More specifically, the control unit 105 reads a pre-learning model and learning data, and controls the switching of the calculation accuracy of the learning process by instructing each processing unit that performs the learning process to perform the calculation. Instructions for calculation include designation of a unit to be calculated and input of parameters required for calculation.

The learned model storage unit 107 stores information on the model after learning. The information of the model after learning may include the updated parameter values of each unit.

4 is a configuration diagram showing an example of the hardware configuration of the learning processing unit 106. As shown in FIG. As shown in FIG. 4, the learning processing unit 106 is implemented by an arithmetic processing unit or the like in which a low-precision arithmetic circuit 11, a high-precision arithmetic circuit 12, a memory 13, and a control device 14 are connected via a bus 15. may be implemented. It should be noted that the high-precision arithmetic circuit 12 may be any circuit that can perform arithmetic operations with higher precision than the low-precision arithmetic circuit 11 .

In that case, the highly efficient inference processor 103a and the highly efficient parameter update processor 104a may be realized by the low-precision arithmetic circuit 11, for example. Also, the high-precision inference processing unit 103b and the high-precision parameter update processing unit 104b may be realized by the high-precision arithmetic circuit 12, for example. Also, the control unit 105 may be realized by the control device 14, for example.

In this example, the low-precision arithmetic circuit 11 and the high-precision arithmetic circuit 12 are connected via a bus 15, respectively, and can exchange data, such as notifying each other of arithmetic results, via the bus 15. FIG. A memory 13 may be further connected to the bus 15, in which case the low-precision arithmetic circuit 11 and the high-precision arithmetic circuit 12 can exchange data via the memory 13, respectively. In that case, the memory 13 is treated as part of the communication path. Note that the memory 13 may be mounted on the same chip as the low-precision arithmetic circuit 11 and the high-precision arithmetic circuit 12 as an on-chip memory. That is, the low-precision arithmetic circuit 11, the high-precision arithmetic circuit 12 and the memory 13 may be internally connected within the chip. Also, the memory 13 may not be mounted on the same chip as the low-precision arithmetic circuit 11 or the high-precision arithmetic circuit 12 as an off-chip memory. That is, it may be externally connected via an external memory interface.

In the present embodiment, the scale of the range and fineness of numeric data actually used in calculations by the processing unit that performs learning processing (especially inference processing and parameter update processing) (more specifically, the processing unit A measure of the breadth and fineness of the range of numerical data determined by the bit width and the handling of decimal points in an arithmetic circuit that implements is called "precision" or "calculation accuracy". Examples of combinations of low arithmetic accuracy, which is the arithmetic accuracy in the low-accuracy arithmetic circuit 11, and high arithmetic accuracy, which is the arithmetic accuracy in the high-accuracy arithmetic circuit 12, include combinations such as those shown in FIG. FIG. 5 is an explanatory diagram showing an example of a combination of low arithmetic precision, which is the arithmetic precision in the low-precision arithmetic circuit 11, and high arithmetic precision, which is the arithmetic precision in the high-precision arithmetic circuit 12. As shown in FIG.

Note that the combination of the arithmetic accuracy in the low-accuracy arithmetic circuit 11 and the arithmetic accuracy in the high-accuracy arithmetic circuit 12 is not limited to that shown in FIG. For example, the arithmetic precision (low arithmetic precision) in the low-precision arithmetic circuit 11 is either fixed decimal point {1, 2, 8, 16} bits or integer {1, 2, 8, 16} bits. and the arithmetic precision (high arithmetic precision) in the high-precision arithmetic circuit 12 is either fixed point {2, 8, 16, 32} bits, floating point {9, 16, 32} bits, or power of 2 It may be any of {8, 16, 24, 32} bits of floating point. However, high precision arithmetic means higher precision than low arithmetic precision (for example, the range of numeric data is wider, the range of numeric data is finer, and the number of significant digits that can be represented is greater). do.

FIG. 6 is a schematic block diagram showing a configuration example of a computer related to the learning device 100. As shown in FIG. Computer 1000 includes processor 1008 , main memory device 1002 , auxiliary memory device 1003 , interface 1004 , display device 1005 and input device 1006 . Also, the processor 1008 may include various arithmetic/processing devices such as the CPU 1001 and the GPU 1007 .

The learning device 100 may be implemented in a computer 1000 as shown in FIG. 6, for example. In that case, the operation of learning device 100 (especially control unit 105) may be stored in auxiliary storage device 1003 in the form of a program. CPU 1001 reads a program from auxiliary storage device 1003, develops it in main storage device 1002, and executes predetermined processing in study device 100 according to the program. Note that the CPU 1001 is an example of an information processing apparatus that operates according to a program, and the computer 1000 includes, in addition to a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an MCU (Memory Control Unit), and a GPU (Graphics Unit). Processing Unit).

FIG. 6 shows an example in which the computer 1000 further includes a GPU 1007 implementing the low-precision arithmetic circuit 11 and the high-precision arithmetic circuit 12 in addition to the CPU 1001. However, the low-precision arithmetic circuit 11 and the high-precision arithmetic If the circuit 12 is implemented by another processor or arithmetic device (MAC (multiplier-accumulator), multiplier tree, ALU (Arthmetic Logic Unit) array, etc. described later), it is not limited to this example, and the other processor or It suffices if it has an arithmetic unit. Also, the low-precision arithmetic circuit 11 and the high-precision arithmetic circuit 12 may be mounted on different chips, and the specific chip configuration is not particularly limited.

Auxiliary storage device 1003 is an example of a non-transitory tangible medium. Other examples of non-transitory tangible media include magnetic disks, magneto-optical disks, CD-ROMs, DVD-ROMs, semiconductor memories, etc. connected via interface 1004 . Also, when this program is distributed to computer 1000 via a communication line, computer 1000 receiving the distribution may develop the program in main storage device 1002 and execute predetermined processing in learning device 100 .

Further, the program may be for realizing a part of predetermined processing in learning device 100 . Furthermore, the program may be a difference program that realizes a predetermined process in learning device 100 in combination with another program already stored in auxiliary storage device 1003 .

Interface 1004 transmits and receives information to and from other devices. The display device 1005 also presents information to the user. Also, the input device 1006 receives input of information from the user.

Also, some elements of the computer 1000 can be omitted depending on the processing content of the learning device 100 . For example, display device 1005 may be omitted if computer 1000 does not present information to a user. For example, input device 1006 may be omitted if computer 1000 does not accept information input from a user.

Also, some or all of the above components are implemented by general-purpose or dedicated circuits (Circuitry), processors, etc., or combinations thereof. These may be composed of a single chip, or may be composed of multiple chips connected via a bus. Also, part or all of the components described above may be realized by a combination of the circuits and the like described above and a program.

When part or all of the above components are realized by a plurality of information processing devices, circuits, etc., the plurality of information processing devices, circuits, etc. may be centrally arranged or distributed. good. For example, the information processing device, circuits, and the like may be implemented as a client-and-server system, a cloud computing system, or the like, each of which is connected via a communication network.

[Circuit configuration]
Next, some configurations of inference circuits, which are implementation examples of at least the high-efficiency inference processing unit 103a, will be exemplified. The highly efficient inference processing unit 103a, for example, for each unit in the designated layer or the designated unit, upon receiving an input to the unit, performs inference processing for calculating the output of the unit with a predetermined low arithmetic accuracy. , may output the calculation result. At that time, the high-efficiency inference processing unit 103a accepts the input values used when calculating the output of the unit and the values of other variables (parameters such as weights and intercepts) as inputs, and performs the above processing. good. An operation performed in inference processing may be referred to as an inference operation hereinafter.

Hereinafter, a circuit for performing an inference operation will be referred to as an "inference circuit". It is called a high-efficiency inference circuit. In this way, the calculation precision of the inference circuit is made as low as possible, at least lower than the calculation precision of the parameter update calculation performed by the high-precision parameter update processing unit 104b (for example, the bit width is changed from 32 bits to 16 bits, floating point Power consumption can be reduced by using fixed decimal point calculations, etc.). In order to distinguish from the high-efficiency inference circuit, a circuit for performing an inference operation with the same accuracy as the parameter update operation performed by the high-precision parameter update processing unit 104b may be called a "high-precision inference circuit". be. The high-precision inference processor (not shown) described above may be realized by such a high-precision inference circuit.

The configuration of the inference circuit shown below can be implemented regardless of whether the inference operation is performed with high accuracy or low accuracy. That is, the difference between the high-efficiency inference processing unit 103a and the high-precision inference processing unit 103b may be only the precision of variables, adders, and multipliers used for calculation in the arithmetic circuit implementing the operation of the processing unit.

The simplest example of an inference circuit is a configuration including one multiplier-adder (MAC) 221 combining a multiplier and an adder (see arithmetic circuit 22a in FIG. 7A). Reference numeral 21 denotes a bus.

The MAC 221 may include a multiplier, an adder, a storage element holding three inputs, and a storage element holding one output (see FIG. 7(b)). The MAC 221 shown in FIG. 7B is an example of an arithmetic circuit that receives three variables a, w, and x and calculates one output variable z=a+w*x. In this example, z corresponds to the output of the unit, a and w to parameters (fixed in inference processing), and x to the input of the unit. In such a configuration, the arithmetic precision of the circuit is determined by the bit width of the multiplier and adder included in the circuit and the handling of the decimal point (floating point or fixed point, etc.). For example, when the high-efficiency inference processing unit 103a is realized by the arithmetic circuit 22a, each variable (a, w, x, z) in the MAC 221 included in the circuit, and the calculation by the adder and the multiplier are performed with low calculation accuracy (first calculation accuracy). At this time, all variables, additions and multiplications in the circuit need not have the same precision (the same applies hereinafter). For example, the precision used in either addition or multiplication for each variable should be lower than the precision used in either addition or multiplication for each variable of the arithmetic circuit that implements the high-precision parameter update processing unit 104b.

8 to 10 are schematic configuration diagrams showing other examples of arithmetic circuits (inference circuits) for inference operations. The inference circuit may have a configuration in which a plurality of MACs 221 are connected in parallel (a so-called GPU configuration), for example, like an arithmetic circuit 22b shown in FIG. Even in such a configuration, the arithmetic precision of the circuit is determined by the bit width of the multiplier and adder included in the circuit and the handling of the decimal point (floating point or fixed point, etc.).

Also, the inference circuit may have a configuration in which a plurality of multiplication-addition trees 223 are connected in parallel via a memory layer 222, like an arithmetic circuit 22c shown in FIG. The multiply-add tree 223 shown in FIG. 9 is a circuit in which four multipliers, two adders, and one adder are connected in a tree configuration. An example of the arithmetic circuit 22c shown in FIG. 9 is also disclosed in Non-Patent Document 3. Even in such a configuration, the arithmetic precision of the circuit is determined by the bit width of the multiplier and adder included in the circuit and the handling of the decimal point (floating point or fixed point, etc.).

Also, the inference circuit may have a configuration (systolic array configuration) in which a plurality of ALUs 224 are connected in an array via a memory layer 222, like an arithmetic circuit 22d shown in FIG. 10, for example. An example of the arithmetic circuit 22d shown in FIG. 10 is also disclosed in Non-Patent Document 1. Even in such a configuration, the arithmetic precision of the circuit is determined by the bit width of the multiplier and adder included in the circuit and the handling of the decimal point (floating point or fixed point, etc.).

For example, when the high-efficiency inference processing unit 103a is realized by the arithmetic circuit 22b, the arithmetic circuit 22c, or the arithmetic circuit 22d shown in FIGS. It suffices if the calculation by the unit corresponds to the low calculation accuracy (first calculation accuracy).

On the other hand, for example, when the high-precision inference processing unit 103b is realized by the arithmetic circuit 22a, the arithmetic circuit 22b, the arithmetic circuit 22c, or the arithmetic circuit 22d, each variable used for the arithmetic operation in the circuit, the arithmetic operation by the adder or the multiplier is It suffices if it supports high calculation accuracy (second calculation accuracy).

Next, some configurations of parameter update circuits, which are implementation examples of at least the high-precision parameter update processing unit 104b, are illustrated. For example, for each parameter in each unit of the designated layer, each parameter in the designated unit, or the designated parameter, the high-precision parameter update processing unit 104b updates an objective function such as an error function including the parameter as an adjustment parameter. A parameter update process for solving the optimization problem and updating the adjustment parameter may be performed with a predetermined high calculation accuracy, and the updated value may be output. At that time, the high-accuracy parameter update processing unit 104b may receive the values of variables used when solving the optimization problem (which may include the values of the parameters before updating) as parameters and perform the above processing. Hereinafter, the computation performed in the parameter update process may be referred to as parameter update computation.

Hereinafter, a circuit for performing parameter update calculations will be referred to as a "parameter update circuit", and in particular, a circuit for performing learning calculations with higher calculation accuracy than the calculation accuracy of the inference calculations performed by the high-efficiency inference processing unit 103a. It is called a "high-precision parameter update circuit". In order to distinguish from the high-precision parameter update circuit, a circuit for performing parameter update calculation with the same calculation accuracy as the calculation precision of the inference calculation performed by the high-efficiency inference processing unit 103a is called a "high-efficiency parameter update circuit". There is The high-efficiency parameter update processing section (not shown) described above may be realized by such a high-efficiency parameter update circuit.

The configuration of the parameter update circuit described below can be realized regardless of whether the parameter update operation is performed with high precision or with low precision. That is, even if the difference between the high-efficiency parameter update processing unit 104a and the high-precision parameter update processing unit 104b is only the precision of each variable, adder or multiplier used for calculation in the arithmetic circuit implementing the operation of the processing unit, good.

The simplest example of the parameter updating circuit is a configuration including one multiplier-adder (MAC) 221 combining a multiplier and an adder, as in the inference circuit (arithmetic circuit 22a in FIG. 7A, FIG. See MAC221, etc. in 7(b)). The parameter update circuit can also be implemented by, for example, the arithmetic circuit 22b, the arithmetic circuit 22c, and the arithmetic circuit 22d shown in FIGS. That is, the arithmetic circuits shown in FIGS. 7 to 10 are also examples of arithmetic circuits for parameter update arithmetic.

For example, when the high-precision parameter update processing unit 104b is realized by the arithmetic circuit 22a, the arithmetic circuit 22b, the arithmetic circuit 22c, or the arithmetic circuit 22d, each variable used in the arithmetic operation in the circuit, the arithmetic operation by the adder and the multiplier are high. It suffices if it corresponds to the calculation accuracy (second calculation accuracy). At this time, it is not necessary that all variables, additions and multiplications have the same precision, and the precision of any one of the variables, additions and multiplications used in the parameter update operation in the relevant circuit realizes the highly efficient inference processing unit 103a. It is sufficient if the precision of each variable, addition or multiplication used for inference calculation in the arithmetic circuit to be used is higher than the precision of either one.

On the other hand, for example, when the high-efficiency parameter update processing unit 104a is realized by the arithmetic circuit 22a, the arithmetic circuit 22b, the arithmetic circuit 22c, or the arithmetic circuit 22d, each variable used in the arithmetic operation in the circuit, the arithmetic operation by the adder and the multiplier corresponds to the low calculation accuracy (first calculation accuracy).

[motion]
Next, the operation of the learning device 100 of this embodiment will be described. FIG. 11 is a flow chart showing an example of the operation of the learning device 100 of this embodiment. The operation shown in FIG. 11 is executed under the control of the control unit 105, for example.

In the example shown in FIG. 11, first, the control unit 105 reads a pre-learning model from the pre-learning model storage unit 101 and reads learning data from the learning data storage unit 102 (step S11).

Next, the control unit 105 controls the high-efficiency inference processing unit 103a and the high-precision inference processing unit 103b as necessary to sequentially perform inference processing on each unit included in all layers from the first layer to the n-th layer. (Step S12: forward propagation). At this time, the control unit 105 causes the highly efficient inference processing unit 103a to perform the inference processing of at least some units. The control unit 105 may cause the high-efficiency inference processing unit 103a to perform inference processing for all units, or may cause the high-efficiency inference processing unit 103a to perform inference processing for some units. In forward propagation, when the high-efficiency inference processing unit 103a performs inference processing for some units, the control unit 105 may cause the high-precision inference processing unit 103b to perform inference processing for the remaining units.

The high-efficiency inference processing unit 103a and the high-precision inference processing unit 103b carry out inference processing of designated layers or units in accordance with instructions from the control unit 105. FIG.

Next, the control unit 105 controls the high-efficiency parameter update processing unit 104a and the high-accuracy parameter update processing unit 104b as necessary to calculate the output of each layer unit. A parameter update process is performed (step S13: parameter update process). At this time, the control unit 105 causes the high-precision parameter update processing unit 104b to perform parameter update processing for at least some of the parameters. Note that the control unit 105 may cause the high-precision parameter update processing unit 104b to perform parameter update processing for all parameters, or may cause the high-precision parameter update processing unit 104b to perform parameter update processing for some parameters. good too. In the parameter update processing, when the high-accuracy parameter update processing unit 104b is caused to perform only the parameter update processing for some parameters, the control unit 105 instructs the high-efficiency parameter update processing unit 104a to perform the parameter update processing for all the remaining parameters. Alternatively, the high-efficiency parameter update processing unit 104a may be caused to perform the parameter update processing for part of the remaining parameters. In the latter case, the parameter update process itself is omitted for some parameters.

The high-efficiency parameter update processing unit 104a and the high-precision parameter update processing unit 104b perform parameter update processing for designated parameters in accordance with instructions from the control unit 105. FIG.

Finally, the control unit 105 stores the learned model including the parameters updated in step S13 in the post-learning model storage unit 107 (step S14).

As another variation of the above operation, for example, when a plurality of learning data are held, the operations of steps S11 to S14 may be repeated by the number of learning data. In this case, a trained model as a learning result for the previous learning data is used as a pre-learning model for learning for the next learning data.

Further, for example, when a plurality of pieces of learning data are held, it is possible to repeat the operations of steps S12 and S13 by the number of pieces of learning data.

Furthermore, regardless of the number of pieces of learning data, it is also possible to repeat the repeating operations of steps S11 to S14 or the repeating operations of steps S12 to S14 a plurality of times using the same learning data (epoch process).

Further, in forward propagation in step S12, for example, the range in which inference processing is performed with low calculation accuracy (low-precision inference range) is not only predetermined, but also can be specified by the user, or can be specified for each learning data or epoch. It is also possible to change each iteration of the process.

Further, in the parameter update process of step S13, for example, the range in which the parameter update process is performed with high calculation accuracy (high-precision parameter update range) may be limited only to the fully connected layers. In addition, for example, the high-precision parameter update range, the range in which parameter update processing is performed with low calculation precision (low-precision parameter update range), and the range in which parameter update processing is not performed are not only determined in advance, but also can be specified by the user. Alternatively, it can be changed each time processing is performed (for each learning data or each repetition of epoch processing).

12 and 13 are flow charts showing a more specific operation example of the learning device 100 of this embodiment. The operation examples shown in FIGS. 12 and 13 are examples in which the operation of each step is illustrated by focusing on the hardware that constitutes the learning apparatus 100. FIG. It should be noted that the hardware configuration was the configuration shown in FIG.

In the example shown in FIG. 12, first, the low-precision arithmetic circuit 11 as the high-efficiency inference processing unit 103a reads the learning data/pre-learning model from the memory 13 in response to an instruction from the control device 14 as the control unit 105. (Step S111).

Next, the low-precision arithmetic circuit 11 performs part of the forward propagation (in this example, an inference operation for calculating the output of each unit included in each layer from the first layer to the (k-1)-th layer) with a low arithmetic precision. (step S112). Then, the low-precision arithmetic circuit 11 saves the arithmetic result of step S112 (output from each unit of the k-1th layer in this example) in the memory 13 (step S113).

In this example, the pre-learning model is assumed to be a multi-layered neural network of n+1 layers from the 0th layer to the nth layer, where the input layer is the 0th layer and the output layer is the nth layer. The (k−1)-th layer is an intermediate layer that is located after the input layer (0th layer) and before the output layer (nth layer). That is, k is an integer that satisfies 0<k-1<n.

Next, the high-precision arithmetic circuit 12 as the high-precision inference processing unit 103b reads out the arithmetic result (output from each unit of the k−1th layer) saved in step S113 in accordance with the instruction of the control device 14 ( step S211).

Then, the high-accuracy arithmetic circuit 12 carries out the continuation of the forward propagation (in this example, the inference operation for calculating the output of each unit included in each layer from the k-th layer to the n-th layer) with high arithmetic accuracy ( step S212).

Next, the high-precision arithmetic circuit 12 as the high-precision parameter update processing unit 104b, according to the instruction of the control device 14, each layer included in some layers (each layer from the k-th layer to the n-th layer in this example) A parameter update operation for updating the parameters of the unit (such as weights of connections with other units) is performed with high accuracy (step S212). Then, the high-precision arithmetic circuit 12 saves the calculation result of step S212 (in this example, updated parameters in each unit included in each of the k-th to n-th layers) in the memory 13 (step S213).

It should be noted that the updated parameters stored as the calculation results in step S213 correspond to the learned model described above.

In the example shown in FIG. 12, first, the low-precision arithmetic circuit 11 performs inference processing on some layers as the high-efficiency inference processing unit 103a, and then the high-precision arithmetic circuit 12 performs the high-precision parameter update processing unit 104b. , is an operation example in which inference processing and parameter update processing are performed for the remaining layers.

In the example shown in FIG. 13, first, the low-precision arithmetic circuit 11 as the high-efficiency inference processing unit 103a stores the learning data/pre-learning model in the memory 13 in response to an instruction from the control device 14 as the control unit 105. (step S121).

Next, the low-accuracy arithmetic circuit 11 performs forward propagation (in this example, an inference operation for calculating the output of each unit included in each layer from the first layer to the n-th layer) with low arithmetic accuracy (step S122). . Then, the low-precision arithmetic circuit 11 saves the arithmetic result of step S122 (in this example, the output from the unit of the n-th layer which is the output layer) in the memory 13 (step S123).

Also in this example, the pre-learning model is assumed to be a multi-layered neural network of n+1 layers from the 0th layer to the nth layer, where the input layer is the 0th layer and the output layer is the nth layer.

Next, the high-precision arithmetic circuit 12 as the high-precision inference processing unit 103b reads out the arithmetic result (output from the unit of the n-th layer which is the output layer) saved in step S123 in accordance with the instruction of the control device 14. (Step S221).

Next, the high-precision arithmetic circuit 12, in accordance with instructions from the control device 14, sets parameters (comparisons with other units) in each unit included in some layers (each layer from the k-th layer to the n-th layer in this example). A parameter update calculation for updating connection weights, etc., is performed with high calculation accuracy (step S222). Then, the high-precision arithmetic circuit 12 saves the calculation result of step S222 (in this example, updated parameters in each unit included in each of the k-th to n-th layers) in the memory 13 (step S223).

It should be noted that the updated parameters stored as the calculation results in step S223 correspond to the learned model described above.

In the example shown in FIG. 13, the low-precision arithmetic circuit 11 performs inference processing for all layers as the high-efficiency inference processing unit 103a, and the high-precision arithmetic circuit 12 performs the high-precision parameter update processing unit 104b. It is an operation example of performing parameter update processing for some layers.

After step S213 in FIG. 12 and step S223 in FIG. 13, the low-accuracy arithmetic circuit 11 can also perform the operations shown in FIG. 14 as the high-efficiency parameter update processing unit 104a.

That is, the low-precision arithmetic circuit 11, as the high-efficiency parameter update processing unit 104a, reads the updated parameters in each unit included in each of the k-th to n-th layers stored in the memory 13 (step S231). .

Next, the low-precision arithmetic circuit 11 calculates the parameters (weight of connection with other units, etc.) in each unit included in the remaining layers (each layer from the 1st layer to the (k-1)th layer in this example). A parameter update calculation for updating is performed with low calculation accuracy (step S232). Then, the low-precision arithmetic circuit 11 saves the arithmetic result of step S232 (in this example, updated parameters in each unit included in each layer from the first layer to the (k−1)th layer) in the memory 13 ( step S233).

In the case of this example, the updated parameter saved as the calculation result in step S213 or step S223 and the updated parameter saved as the calculation result in step S233 correspond to the learned model described above.

Note that the operations shown in FIGS. 12 to 14 are examples of learning processing for one piece of learning data. Therefore, when a plurality of pieces of learning data are stored, it is possible to repeat the above operation and each operation step included in the above operation as many times as the number of pieces of learning data. Moreover, regardless of the number of learning data, it is also possible to repeat the above operation or each operation step included in the above operation a plurality of times using the same learning data (epoch processing). In addition, the k-th layer to the n-th layer, which are the high-precision parameter update range in the above operation, may be fully connected layers, or k may be specified by the user or changed for each process. .

As described above, according to the present embodiment, the arithmetic processing of the learning algorithm is divided into the inference processing and the parameter update processing, at least part of the inference processing is operated with low arithmetic accuracy, and at least one of the parameter update processing is performed. By calculating the part with high calculation accuracy, the calculation part that requires high calculation accuracy can be optimized, so that power consumption can be reduced and learning with sufficient accuracy becomes possible.

Embodiment 2.
Next, a second embodiment of the invention will be described. FIG. 15 is a block diagram showing a configuration example of the main part of the data processing device of the second embodiment. A data processing device 300 shown in FIG. 15 includes a low-precision arithmetic processing unit 31, a high-precision arithmetic processing unit 32, a communication path 33, and a data conversion unit .

The low-precision arithmetic processing unit 31 is a processing unit that performs predetermined arithmetic operations with relatively low arithmetic accuracy. Here, the relatively low calculation accuracy may be any calculation accuracy lower than the calculation accuracy of the calculation performed by the high-precision calculation processing unit 32 .

The high-accuracy arithmetic processing unit 32 is a processing unit that performs predetermined arithmetic operations with relatively high arithmetic accuracy. Here, the relatively high calculation accuracy may be any calculation accuracy higher than the calculation accuracy of the calculation performed by the low-accuracy calculation processing unit 31 .

The low-accuracy arithmetic processing unit 31 may be, for example, the high-efficiency inference processing unit 103a or the high-efficiency parameter update processing unit 104a. Further, the high-precision arithmetic processing unit 32 may be, for example, the high-precision inference processing unit 103b or the high-precision parameter update processing unit 104b. Also in the present embodiment, the scale of the breadth and fineness of the value range of numerical data used for the calculation actually performed in the data processing performed by the low-precision arithmetic processing unit 31 and the high-precision arithmetic processing unit 32 (more specifically, A measure of the breadth and fineness of the value range of numerical data determined by the bit width and the handling of decimal points in an arithmetic circuit that implements the processing unit is called "accuracy" or "calculation precision."

In this example, the low-precision arithmetic processing section 31 and the high-precision arithmetic processing section 32 are connected via a communication path 33 and a data conversion section 34 . Note that the data converter 34 is provided between the high-precision arithmetic processor 32 and the communication path 33 .

The communication path 33 may be implemented by, for example, a bus. Note that the communication path 33 may be realized by an interconnection circuit (Inter-connect) provided inside the chip. Moreover, the communication path 33 may include not only a bus and a connection circuit, but also a memory (external memory, buffer, etc.) connected to the bus and the connection circuit.

The data conversion unit 34 performs predetermined conversion processing on data exchanged between the low-precision arithmetic processing unit 31 and the high-precision arithmetic processing unit 32 . At this time, the data conversion by the data conversion unit 34 is performed, for example, in data communication (data exchange) performed on the communication path 33, for data communication with an arithmetic accuracy that reduces the data amount (communication amount per piece of data). It is done so that

For example, the data conversion unit 34 converts each data passing through the communication path 33 into data with the smaller amount of data, out of the calculation accuracy of the low-precision calculation processing unit 31 and the calculation accuracy of the high-precision calculation processing unit 32. Transmit and receive data is converted as follows. If the amount of data is the same, the transmission/reception data is converted so as to obtain data with lower calculation accuracy. In the configuration of FIG. 15, the data conversion unit 34 may convert data so that each piece of data passing through the communication path 33 becomes data with the calculation precision of the low-precision calculation processing unit 31. FIG. By adjusting to the lower calculation accuracy, data conversion on the low-precision calculation processing unit 31 side can be made unnecessary while minimizing deterioration in calculation accuracy due to data exchange.

Here, the data conversion includes type conversion to match the data type with a data type with lower arithmetic precision in the processing unit that is the communication end point, data compression (in particular, numeric string compression, digit number reduction, etc.). numerical data compression) and synthesis of two or more transformed data.

The data conversion unit 34 receives, for example, data transmitted from the low-accuracy arithmetic processing unit 31 to the high-accuracy arithmetic processing unit 32 via the communication path 33, and converts the received data (low-accuracy data) to high-accuracy The data is converted into data of calculation precision (high calculation precision) handled by the precision calculation processing unit 32 and transferred to the high precision calculation processing unit 32 . Further, the data conversion unit 34, for example, receives data transmitted from the high-precision arithmetic processing unit 32 toward the low-precision arithmetic processing unit 31, and converts the received data (high-precision arithmetic data) to the low-precision arithmetic processing unit 31 is converted into data of calculation accuracy (low calculation accuracy) handled by 31 and sent to the communication path 33 .

For example, the arithmetic precision of the low-precision arithmetic processing unit 31 (here, the data type of numerical values used for arithmetic) is 16-bit integer (INT16), and the arithmetic precision of the high-precision arithmetic processing unit 32 is 32-bit floating point (FP32 ), the data conversion unit 34 may perform data conversion so that the data passing through the communication path 33 becomes integer 16-bit data. Further, for example, when the arithmetic precision of the low-precision arithmetic processing unit 31 is 16-bit integer (INT16) and the arithmetic precision of the high-precision arithmetic processing unit 32 is 16-bit floating point (FP16), the data conversion unit 34 Data conversion may be performed so that the data passing through the communication path 33 becomes integer 16-bit data.

Data communication through the communication path 33 is one-way communication (for example, only transmission from the low-accuracy arithmetic processing unit 31 to the high-accuracy arithmetic processing unit 32, and transmission from the high-accuracy arithmetic processing unit 32 to the low-accuracy arithmetic processing unit 31). ) may be used. In that case, the data conversion unit 34 only needs to perform data conversion corresponding to communication that is actually performed.

In this embodiment, the data conversion unit 34 may be realized by a dedicated data conversion circuit that performs data conversion designed in accordance with the configuration and operation of the data processing device 300 . By implementing the data conversion unit 34 in a dedicated circuit, it is possible to omit processing for generalization such as reading of setting values and states and branching according to them, and further efficiency improvement can be achieved. In addition, by dedicating data conversion, it is possible to easily implement processing such as collectively converting multiple data, performing data conversion of multiple data in parallel, and collectively sending the results. , further efficiency can be achieved. Here, the parallel processing of data conversion and the compilation of the results are one example of data conversion in which the conversion of each data and the synthesis of data after conversion are combined. The data conversion unit 34 may realize such data conversion by SIMD (Single Instruction Multiple Data) calculation, for example.

FIG. 16 is a block diagram showing another configuration example of the data processing device of the second embodiment. In the example shown in FIG. 15, the data conversion section 34 is provided only on the high-precision arithmetic processing section 32 side, but the data conversion section can also be provided on the low-precision arithmetic processing section 31 side. The data processor 300 shown in FIG. 16 differs from the configuration shown in FIG. 15 in that a data converter 35 is further provided between the low-precision arithmetic processor 31 and the communication path 33 . That is, in this example, the low-precision arithmetic processing section 31 and the high-precision arithmetic processing section 32 are connected via the data conversion section 35 , the communication path 33 and the data conversion section 34 .

The data conversion unit 35 performs predetermined conversion processing on data exchanged between the low-precision arithmetic processing unit 31 and the high-precision arithmetic processing unit 32 .

In this example, the data conversion by the data conversion unit 34 and the data conversion unit 35 is performed with the calculation accuracy handled by the low-precision calculation processing unit 31 in the communication (data exchange) performed on the communication path 33. The data communication is performed so that the data communication has an arithmetic precision that is even smaller than the data amount of the .

For example, the data conversion unit 34 and the data conversion unit 35 each data passing through the communication path 33, the calculation accuracy (hereinafter referred to as ultra-low calculation accuracy) less than the amount of data communication performed with the calculation accuracy of the low-precision calculation processing unit 31 Transmit and receive data is converted so that it becomes the data of

In this example, the data conversion unit 34 receives, for example, data transmitted from the low-precision arithmetic processing unit 31 toward the high-precision arithmetic processing unit 32 via the data conversion unit 35 and the communication path 33, and The data (ultra-low calculation accuracy data after conversion by the data conversion unit 35 ) is converted into calculation accuracy (high calculation accuracy) data handled by the high-accuracy calculation processing unit 32 and passed to the high-accuracy calculation processing unit 32 . Further, the data conversion unit 34, for example, receives data transmitted from the high-accuracy arithmetic processing unit 32 to the low-accuracy arithmetic processing unit 31, and converts the received data (high-accuracy data) to ultra-low arithmetic accuracy. It is converted into data and sent to the communication path 33 .

Further, the data conversion unit 35 receives, for example, data transmitted from the high-precision arithmetic processing unit 32 toward the low-precision arithmetic processing unit 31 via the data conversion unit 34 and the communication path 33, and converts the received data ( The ultra-low-accuracy data converted by the data converter 34 ) is converted into arithmetic-accuracy (low-accuracy) data handled by the low-accuracy arithmetic processing unit 31 , and passed to the low-accuracy arithmetic processing unit 31 . Further, the data conversion unit 35, for example, receives data transmitted from the high-accuracy arithmetic processing unit 32 to the low-accuracy arithmetic processing unit 31, and converts the received data (high-accuracy data) to ultra-low arithmetic accuracy. It is converted into data and sent to the communication path 33 .

For example, the arithmetic precision of the low-precision arithmetic processing unit 31 (here, the data type of numerical values used for arithmetic) is 16-bit integer (INT16), and the arithmetic precision of the high-precision arithmetic processing unit 32 is 32-bit floating point (FP32 ), the data conversion unit 34 and the data conversion unit 35 convert data so that the data passing through the communication path 33 becomes a 12-bit integer (INT12) or an 8-bit integer (INT8), which has a smaller data amount than INT16. Can be compressed. The data conversion unit 34 and the data conversion unit 35 perform numerical data compression (for example, reduction of lower bits, etc.) that lowers only the accuracy so that the data does not lose its meaning as numerical data when compressing data.

Note that the data conversion unit 34 and the data conversion unit 35 can also perform data compression using the feature of the activation function in deep learning. For example, using a step function, which is one of the activation functions, data can be compressed to 1 bit. Also, if ReLU (ramp function) is used, the number of sign bits of data can be reduced.

Further, when performing data conversion for reducing the number of bits, the data conversion unit 34 and the data conversion unit 35 collect a plurality of pieces of data with an odd number of bits, or divide the collected data into a plurality of pieces of data. Processing (packing/unpacking processing) for disassembling into data may be performed. Such packing/unpacking processing can also be made more efficient by specializing it.

By reducing the amount of data passing through the communication path 33 in this way, the speed of data exchange between cores (arithmetic circuits) with different arithmetic accuracies can be increased. Furthermore, when data exchange between cores is performed via memory, the amount of memory used for data exchange can be reduced, so power consumption associated with memory use can also be reduced.

Note that when there are two or more combinations of cores with different arithmetic accuracies that perform data exchange, for each combination, one or both end points of the communication path for inter-core communication in the combination are connected to the data conversion unit 34 or the data conversion unit. A portion 35 may be provided.

Next, an outline of the present invention will be described. FIG. 17 is a block diagram showing the outline of the data processing device of the present invention. A data processing device 500 shown in FIG.

The low-precision arithmetic processing means 501 (for example, the low-precision arithmetic processing section 31) performs a predetermined arithmetic operation with a first precision.

The high-precision arithmetic processing means 502 (for example, the high-precision arithmetic processing section 32) performs a predetermined arithmetic operation with a second precision higher than the first precision.

The first data conversion means 504 (for example, the data conversion unit 34) is a high-precision arithmetic processing means of the communication path 503 for exchanging data between the high-precision arithmetic processing means 502 and the low-precision arithmetic processing means 501. It is provided at the end point on the 502 side.

The first data conversion means 504 ensures that the data transferred to and from the high-precision arithmetic processing means 502 of the connection destination is data that can be handled by the high-precision arithmetic processing means 502, and that the amount of data passing through the communication path 503 is Pass between the communication path 503 and the high-precision arithmetic processing means 502 so that the amount of data is less than the amount of data when using data with an accuracy of 1, and the accuracy of the data passing through the communication path 503 is less than or equal to the first accuracy. Predetermined conversion is performed on the data.

With such a configuration, it is possible to improve efficiency even in a process in which calculations requiring high accuracy and calculations not requiring high accuracy are mixed.

FIG. 18 is a configuration diagram showing a configuration example of the data processing circuit of the present invention. A data processing circuit 600 shown in FIG. 18 includes a low-precision arithmetic circuit 601 , a high-precision arithmetic circuit 602 , and a first data conversion circuit 604 .

The low-precision arithmetic circuit 601 (for example, the low-precision arithmetic processing unit 31 and the low-precision arithmetic circuit 11) performs a predetermined arithmetic operation with a first precision.

The high-precision arithmetic circuit 602 (for example, the high-precision arithmetic processing unit 32 and the high-precision arithmetic circuit 12) performs a predetermined arithmetic operation with a second precision higher than the first precision.

The first data conversion circuit 604 (for example, the data conversion unit 34) is provided on the high-precision arithmetic circuit 602 side of the communication path 603 for exchanging data between the high-precision arithmetic circuit 602 and the low-precision arithmetic circuit 601. It is provided at the end point and performs predetermined conversion on data passing between the communication path 603 and the high-precision arithmetic circuit 602 .

In such a data processing circuit 600 , the data transferred between the first data conversion circuit 604 and the high-precision arithmetic circuit 602 connected thereto is the data handled by the high-precision arithmetic circuit 602 , and passes through the communication path 603 . The amount of data is equal to or less than the amount of data when data with the first accuracy is used, and the accuracy of data passing through the communication path is configured to be equal to or less than the first accuracy.

With such a configuration as well, it is possible to improve the efficiency of processing in which operations that require high accuracy and operations that do not require high accuracy are mixed.

As shown in FIG. 19, the data processing circuit 600 is further provided at the end point of the communication path 603 on the low-precision arithmetic circuit 601 side. may include a second data conversion circuit 605 that performs a predetermined conversion.

In the data processing circuit 600 as described above, the data transferred between the second data conversion circuit 605 and the low-precision arithmetic circuit 601 connected thereto is the data handled by the low-precision arithmetic circuit 601, and the communication path The amount of data passing through 603 may be smaller than the amount of data when data with the first accuracy is used, and the accuracy of the data passing through the communication path 603 may be lower than the first accuracy.

According to such a configuration, it is possible to further improve the efficiency of processing in which calculations that require high accuracy and calculations that do not require high accuracy are mixed.

It should be noted that the above embodiment can also be described as the following additional remarks.

(Appendix 1) Low-precision arithmetic processing means for performing predetermined arithmetic operations with a first precision, high-precision arithmetic processing means for performing predetermined arithmetic operations with a second precision higher than the first precision, and high-precision arithmetic processing means and a first data conversion means provided at the end point of the high-precision arithmetic processing means side of the communication path for exchanging data between the low-precision arithmetic processing means and the first data conversion means, the connection The data transferred to and from the high-precision arithmetic processing means is data that can be handled by the high-precision arithmetic processing means, and the amount of data passing through the communication path is equal to or less than the amount of data when using data of the first precision. and performing a predetermined conversion on the data passing between the communication channel and the high-precision arithmetic processing means so that the accuracy of the data passing through the communication channel is equal to or lower than the first accuracy. Device.

(Appendix 2) The first data conversion means accepts data passed from the low-precision arithmetic processing means to the high-precision arithmetic processing means from the communication path as first precision data, and converts the received first precision data to The first data conversion means receives the data passed from the high-precision arithmetic processing means to the low-precision arithmetic processing means as data that can be handled by the high-precision arithmetic processing means. , the data processing device according to appendix 1, which converts the received data into data of precision that can be handled by the low-precision arithmetic processing means.

(Appendix 3) Further comprising a second data conversion means provided at an end point of the communication path on the side of the low-precision arithmetic processing means, the first data conversion means and the second data conversion means are connected to the arithmetic processing means of the connection destination. The data passed between the 3. The data processing device according to appendix 1 or 2, wherein the data passing between the communication path and the arithmetic processing means of the connection destination is subjected to a predetermined conversion so that the accuracy of the passing data is lower than the first accuracy. .

(Appendix 4) The first data conversion means and the second data conversion means convert the data passed from the connection destination arithmetic processing means to the counterpart arithmetic processing means through the communication path into a predetermined accuracy lower than the first accuracy. receiving as third precision data, converting the received third precision data into precision data that can be handled by the arithmetic processing means of the connection destination, and the first data conversion means and the second data conversion means Supplementary note 3 that accepts the data passed from the arithmetic processing means of the other to the arithmetic processing means of the other party as data that can be handled by the arithmetic processing means of the connection destination, and converts the received data into data of accuracy that can be handled by the arithmetic processing means of the other party The data processing device according to .

(Appendix 5) At least one of the first data conversion means and the second data conversion means, when sending the converted data to the communication channel, collectively sends a plurality of converted data, at least one of the data conversion means and the second data conversion means receives a plurality of data after conversion from the communication channel, decomposes the received data after conversion, and decomposes the data after decomposing 4. The data processing device according to appendix 3 or appendix 4, wherein each data is converted into data of accuracy that can be handled by the arithmetic processing means of the connection destination.

(Appendix 6) The data processing apparatus according to any one of Appendices 3 to 5, wherein conversions performed by the first data conversion means and the second data conversion means are predetermined and fixed.

(Appendix 7) The data processing device is a learning device that learns a predetermined discriminant model composed of two or more units connected in layers, and when learning data is input, each unit of the discriminant model and a parameter update process for updating at least part of the parameters used in calculating the output of each unit based on the result of the inference process. , the learning means includes, as the low-precision arithmetic processing means, high-efficiency inference means for performing a specified arithmetic operation among the arithmetic operations performed in the inference processing with a first arithmetic precision, and the high-accuracy arithmetic processing means. and high-precision parameter update means for performing a specified operation among the operations performed in the parameter update process with a second operation accuracy higher than the first operation accuracy. A data processing device according to any one of the above.

(Appendix 8A) A low-precision arithmetic circuit that performs a predetermined operation with a first precision, a high-precision arithmetic circuit that performs a predetermined arithmetic operation with a second precision that is higher than the first precision, a high-precision arithmetic circuit, and a low-precision It is provided at the end point of the high-precision arithmetic circuit side of the communication path for exchanging data with the arithmetic circuit, and performs a predetermined conversion on the data passing between the communication path and the high-precision arithmetic circuit. data transferred between the first data conversion circuit and the connected high-precision arithmetic circuit is data handled by the high-precision arithmetic circuit, and the amount of data passing through the communication path is less than or equal to the amount of data when using data with a first precision, and the precision of data passing through the communication path is less than or equal to the first precision.

(Appendix 8B) Further, a second data conversion circuit is provided at an end point of the communication path on the low-precision arithmetic circuit side and performs predetermined conversion on data passing between the communication path and the low-precision arithmetic circuit. The data transferred between the second data conversion circuit and the connected low-precision arithmetic circuit is the data handled by the low-precision arithmetic circuit, and the amount of data passing through the communication path is the data of the first precision. 8B. The data processing circuit of clause 8B, wherein the amount of data is less than the amount of data used and the accuracy of the data passing through the channel is less than the first accuracy.

(Appendix 9) Transferring data between low-precision arithmetic processing means that performs predetermined arithmetic operations with a first precision and high-precision arithmetic processing means that performs predetermined arithmetic operations with a second precision that is higher than the first precision The first data conversion means provided at the end point of the high-precision arithmetic processing means side of the communication path for performing the above, the data transferred between the high-precision arithmetic processing means of the connection destination can be handled by the high-precision arithmetic processing means In addition to being data, the amount of data passing through the communication channel is equal to or less than the amount of data when data with the first accuracy is used, and the accuracy of the data passing through the communication channel is equal to or less than the first accuracy. and a high-precision arithmetic processing means, wherein a predetermined conversion is performed on data passing between the high-precision arithmetic processing means.

(Supplementary Note 10) The data transferred between the first data conversion means and the high-precision arithmetic processing means of the connection destination is data that can be handled by the high-precision arithmetic processing means, and the amount of data passing through the communication path is For data passing between the communication channel and the high-precision arithmetic processing means so that the amount of data is less than when data with a precision of 1 is used, and the precision of the data passing through the communication channel is lower than the first precision the second data conversion means provided at the end point of the communication path on the low-precision arithmetic processing means side performs the low-precision arithmetic processing on the data transferred between the low-precision arithmetic processing means of the connection destination In addition to being data that can be handled by means, the amount of data passing through the communication channel is less than the amount of data when using data with the first accuracy, and the accuracy of the data passing through the communication channel is lower than the first accuracy 10. The data processing method according to appendix 9, wherein the data passing between the communication channel and the low-precision arithmetic processing means is subjected to a predetermined conversion.

Although the present invention has been described with reference to the embodiments and examples, the present invention is not limited to the above embodiments and examples. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

The present invention is not limited to deep learning, and is suitable for devices that perform processing that requires high accuracy and processing that does not require high accuracy, and is suitable when it is desired to perform the processing while suppressing power consumption. Applicable.

10 arithmetic circuit 11 low-precision arithmetic circuit 12 high-precision arithmetic circuit 13 memory 14 control device 15 bus 51 unit 52 inter-unit connection 53 inference processing 54 parameter update processing 100 learning device 101 pre-learning model storage unit 102 learning data storage unit 103a high Efficient inference processing unit 103b High-precision inference processing unit 104a High-efficiency parameter update processing unit 104b High-precision parameter update processing unit 105 Control unit 106 Learning processing unit 107 Post-learning model storage unit 1000 Computer 1001 CPU
1002 Main Storage Device 1003 Auxiliary Storage Device 1004 Interface 1005 Display Device 1006 Input Device 1007 GPU
1008 processor 21 bus 22a, 22b, 22c, 22d arithmetic circuit 221 MAC
222 memory layer 223 multiply-add tree 224 ALU
300 data processing device 31 low-precision arithmetic processing unit 32 high-precision arithmetic processing unit 33 communication path 34, 35 data conversion unit 500 data processing device 501 low-precision arithmetic processing means 502 high-precision arithmetic processing means 503 communication path 504 first data conversion means 600 data processing circuit 601 low precision arithmetic circuit 602 high precision arithmetic circuit 603 communication channel 604 first data conversion circuit 605 second data conversion circuit 90 large scale learning circuit

Claims (7)

low-precision arithmetic processing means for performing a predetermined arithmetic operation with a first precision;
high-precision arithmetic processing means for performing a predetermined arithmetic operation with a second precision higher than the first precision;
A first data conversion means provided at an end point on the high-precision arithmetic processing means side of a communication path for exchanging data between the high-precision arithmetic processing means and the low-precision arithmetic processing means,
In the first data conversion means, the data transferred between the high-precision arithmetic processing means of the connection destination is data that can be handled by the high-precision arithmetic processing means, and the amount of data passing through the communication path is between the communication path and the high-precision arithmetic processing means so that the amount of data is equal to or less than the amount of data when data with a first accuracy is used, and the accuracy of data passing through the communication path is equal to or less than the first accuracy. performs a predetermined transformation on the data passing through
Further comprising a second data conversion means provided at an end point of the communication path on the low-precision arithmetic processing means side,
In the first data conversion means and the second data conversion means, the data transferred between the arithmetic processing means of the connection destination is data that can be handled by the arithmetic processing means of the connection destination, and the communication path is The communication path and the Performs a predetermined conversion on the data passing between it and the arithmetic processing means of the connection destination
A data processing device characterized by: The first data conversion means and the second data conversion means convert data passed from the communication path from the counterpart arithmetic processing means to the connection destination arithmetic processing means to a predetermined precision lower than the first accuracy. and converting the received data of the third precision into data of precision that can be handled by the arithmetic processing means of the connection destination,
The first data conversion means and the second data conversion means accept data transferred from the arithmetic processing means of the connection destination to the arithmetic processing means of the other party as data that can be handled by the arithmetic processing means of the connection destination. , converting the received data into data of the third accuracy . At least one of the first data conversion means and the second data conversion means collectively sends a plurality of converted data when sending the converted data to the communication path,
At least one of the first data conversion means and the second data conversion means receives from the communication path the plurality of converted data that have been put together, and decomposes the received plurality of converted data. 3. The data processing apparatus according to claim 1 , further comprising converting each piece of decomposed data into data with a precision that can be handled by the arithmetic processing means of the connection destination. 4. The data processing apparatus according to any one of claims 1 to 3 , wherein conversions performed by said first data conversion means and said second data conversion means are predetermined and fixed. The data processing device is a learning device that learns a predetermined discriminant model composed of two or more units connected in layers,
When learning data is input, inference processing for calculating the output of each unit of the discriminant model in a predetermined order; A learning means for performing parameter update processing for partially updating,
The learning means is
high-efficiency inference means for performing, as the low-precision arithmetic processing means, a specified arithmetic operation among the arithmetic operations performed in the inference processing with a first arithmetic precision;
high-precision parameter update means for performing a specified calculation among the calculations performed in the parameter update process with a second calculation precision higher than the first calculation precision, as the high-precision calculation processing means. 5. A data processing apparatus according to any one of claims 1 to 4 . a low-precision arithmetic circuit that performs a predetermined arithmetic operation with a first precision;
a high-precision arithmetic circuit that performs a predetermined arithmetic operation with a second precision higher than the first precision;
provided at an end point on the high-precision arithmetic circuit side of a communication path for exchanging data between the high-precision arithmetic circuit and the low-precision arithmetic circuit, and connecting between the communication path and the high-precision arithmetic circuit a first data conversion circuit that performs a predetermined conversion on passing data,
Data transferred between the first data conversion circuit and the high-precision arithmetic circuit to which it is connected is data handled by the high-precision arithmetic circuit,
a second data conversion circuit provided at an end point of the communication path on the low-precision arithmetic circuit side and performing predetermined conversion on data passing between the communication path and the low-precision arithmetic circuit; prepared,
Data transferred between the second data conversion circuit and the low-precision arithmetic circuit to which it is connected is data handled by the low-precision arithmetic circuit,
The amount of data passing through the communication channel is less than the amount of data when using the data with the first accuracy, and the accuracy of the data passing through the communication channel is lower than the first accuracy.
A data processing circuit characterized by: To exchange data between low-precision arithmetic processing means for performing predetermined arithmetic operations with a first precision and high-precision arithmetic processing means for performing predetermined arithmetic operations with a second precision higher than the first precision. A first data conversion means provided at an end point of the communication path on the side of the high-precision arithmetic processing means,
The data transferred between the high-precision arithmetic processing means of the connection destination is data that can be handled by the high-precision arithmetic processing means, and the amount of data passing through the communication path uses the data of the first precision. data passing between the communication channel and the high-precision arithmetic processing means so that the amount of data passing through the communication channel is less than or equal to the first precision, and the accuracy of the data passing through the communication channel is less than or equal to the first precision and
In the first data conversion means, the data transferred between the high-precision arithmetic processing means of the connection destination is data that can be handled by the high-precision arithmetic processing means, and the amount of data passing through the communication path is between the communication path and the high-precision arithmetic processing means so that the amount of data is less than the amount of data when data with the first accuracy is used, and the accuracy of the data passing through the communication path is lower than the first accuracy. performs a predetermined transformation on the data passing through
The second data conversion means provided at the end point of the communication path on the low-precision arithmetic processing means side can handle the data transferred between the low-accuracy arithmetic processing means to which it is connected and the low-accuracy arithmetic processing means. data, the amount of data passing through the communication path is less than the amount of data when using data with the first accuracy, and the accuracy of the data passing through the communication path is lower than the first accuracy. and performing a predetermined conversion on the data passing between the communication path and the low-precision arithmetic processing means
A data processing method characterized by:

Images