JP4947983B2 - Arithmetic processing system - Google Patents

Description

  The present invention relates to an LSI chip and an arithmetic processing system.

  Conventionally, there is known an arithmetic processing system that executes hierarchical arithmetic processing by connecting a plurality of substrates on which arithmetic circuits are mounted (Patent Document 1). Such an arithmetic processing system will be described with reference to FIG. FIG. 23 is a diagram schematically illustrating a configuration example of an arithmetic processing system formed by connecting a plurality of substrates on which arithmetic circuits are mounted.

  In FIG. 23, a plurality of neuron mimic elements 20 with a learning function corresponding to an arithmetic circuit are connected in a hierarchical network. Furthermore, in the arithmetic processing system provided with the control means which controls these neuron mimic elements 20, the neuron mimic element 20 is divided and mounted on the substrates 55a and 55b provided for each of the layers A2 and A3. Yes. With such a configuration, an arithmetic processing system as a neurocomputer system is constructed.

  There is also known an LSI (large-scale integration) chip that performs a convolution operation on a two-dimensionally distributed data such as image data with a kernel having a two-dimensionally distributed weight. Patent Document 1). The processing flow of such an LSI chip will be described with reference to FIG. FIG. 24 is a diagram schematically illustrating a data flow in an LSI chip that performs a convolution operation on two-dimensional data.

  Image data and kernel data are stored in SRAM (Image SRAM and Kernel SRAM), respectively. The image data is input to the PWM product-sum operation circuit (PWM-MAC) through the shift register (SR) and the digital / PWM converter (D / P). Kernel data is input to PWM-MAC via SR, D / P, and PWM / analog converter (P / A). However, SR converts serial data output from the SRAM into parallel data. D / P and P / A convert the digital signal into a PWM signal and the PWM signal into an analog voltage, respectively. The PWM-MAC performs convolution operation processing in parallel. Note that PWM is an abbreviation for pulse-width modulation.

The output of the PWM-MAC is accumulated by a digital accumulator (ACC) via a PWM / digital converter (P / D). The accumulated result is nonlinearly converted by a look-up table (LUT) through a multiplexer (MUX) and stored again in the image SRAM. By repeating the above processing, features are detected from the image data.
JP-A-5-233582 K. Korekado et al., “An Image Filtering Processor for Face / Object Recognition Using Merged / Mixed Analog-Digital Architecture”, in 2005 Symposium on VLSI Circuits, Digest of Technical papers, pp. 220-223, Kyoto, Japan, June 2005.

  However, in the arithmetic processing system disclosed in Patent Document 1, since it is necessary to mount wiring for all connections between neuron mimic elements on different substrates, when the number of connected elements increases, the wiring is It becomes difficult to implement everything. In particular, when performing a convolution operation with a kernel having a two-dimensional weight distribution on two-dimensionally distributed data such as image data, it is necessary to connect many elements in a complicated manner. It is difficult to implement all connections between mimic elements.

  In the LSI chip disclosed in Non-Patent Document 1, it is difficult to perform arithmetic processing on two-dimensional data having a size that exceeds the memory size of Image SRAM included in the LSI chip.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique capable of efficiently executing arithmetic processing on two-dimensional data exceeding the memory size of a memory circuit block built in an LSI chip.

The arithmetic processing system according to the present invention has the following configuration. That is,
An arithmetic processing system that connects a plurality of LSI chips and performs a convolution operation between two-dimensional target data and a two-dimensional kernel,
Each of the LSI chips is
A target data memory for holding the target data;
A register that holds the target data required for the operation;
Kernel memory holding the kernel;
An arithmetic circuit that performs convolution arithmetic processing based on the target data held in the register and the kernel;
An input / output wiring connected to an output line from the target data memory to the register, and for inputting / outputting the target data to and from the adjacent LSI chip;
A switch circuit for switching connection with the adjacent LSI chip in the input / output wiring,
The target data is distributed and held in the target data memory provided in each of the plurality of adjacent LSI chips,
In each of the plurality of LSI chips, when the target data is output from the target data memory provided in the LSI chip to the register provided in the LSI chip, in the calculation in the arithmetic circuit of the adjacent LSI chip Necessary target data that does not exist in the target data memory provided in the adjacent LSI chip is output to the register provided in the same LSI chip as the target data memory that holds the data. At the same time, by switching the connection in the input / output wiring by the switch circuit , the output is simultaneously output to the register provided in the adjacent LSI chip via the input / output wiring.

  According to the present invention, it is possible to provide a technique capable of efficiently executing arithmetic processing on two-dimensional data exceeding the memory size of a memory circuit block built in an LSI chip.

  Embodiments according to the present invention will be described below in detail with reference to the accompanying drawings. However, the constituent elements described in this embodiment are merely examples, and are not intended to limit the scope of the present invention only to them.

<< First Embodiment >>
(Calculation processing system)
FIG. 1 is a diagram schematically showing an arithmetic processing system according to the present embodiment. As shown in FIG. 1, the arithmetic processing system is configured by arranging four LSI chips 1 to 4 in one row. Note that the number of LSI chips can be changed according to the data size to be processed, and in this embodiment, four cases are shown as an example. Each LSI chip is connected to an adjacent LSI chip by wiring 5 for inputting and outputting data. Here, in this embodiment, the data width input and output between the chips is 6 bits, but may have other data widths. In that case, the number of wirings changes according to the data width. In FIG. 1, 6-bit wiring and LSI chip pins are abbreviated as one wiring and pins. In FIG. 1, wirings other than the input and output wirings 5 described above are not shown because they are not necessary for the description of this embodiment.

(Example of calculation processing)
Next, an example of arithmetic processing realized by the arithmetic processing system will be described with reference to FIG. FIG. 2 is a diagram schematically showing processing for detecting the position of a face using a hierarchical convolutional neural network.

As shown in FIG. 2, the arithmetic processing in this embodiment inputs image data 7 to the first-stage layer 8, and repeatedly executes a convolution operation with a kernel 6 having weights distributed two-dimensionally in a hierarchical manner. To do. In hierarchies 1 to 6, the convolution operation with the kernel 6 is executed on the operation result group of the convolution operation performed on the first layer similarly to the first layer. In this case, an operation executed by one arithmetic element included in a certain hierarchy is expressed by the following equation (1), where o is the output value of the previous stage and w is the weight of the kernel.
u = Σw · o (1)
When performing the convolution operation in a hierarchical manner, the operation result calculated in each layer becomes an input value for the convolution operation in the next stage. In each layer, a plurality of different calculation result groups may be calculated as convolution calculation results. A plurality of different calculation results in this case correspond to detection results of a plurality of different features, as shown in FIG. This is achieved by changing the kernel weight distribution. In addition, as shown in FIG. 2, the convolution calculation may use a plurality of calculation result groups in the previous layer as input values.

  The calculation processing in the present embodiment assumes a case where face position detection is performed using a hierarchical convolutional neural network, which is known as a technique for detecting features from a natural image in which data is distributed two-dimensionally. However, it is not limited to this. In other words, any other algorithm may be executed as long as it performs a convolution operation between two-dimensionally distributed data and a kernel having a predetermined two-dimensionally distributed weight.

(Circuit configuration of LSI chip)
Next, the circuit configuration of the LSI chips 1 to 4 constituting the arithmetic processing system will be described with reference to FIG. FIG. 3 is a block diagram schematically showing the circuit configuration of the LSI chip. As shown in FIG. 3, the LSI chip in this embodiment has the following circuit blocks.
A memory circuit block (MEM) 10;
Kernel data memory circuit block (KMEM) 11
Registers (REG) 12,14.
Digital-PWM conversion circuit blocks (D / P) 13, 15
PWM-analog conversion circuit (P / A) 17
Arithmetic circuit block (PWM-MAC) 16
An accumulation circuit block (ACC) 19;
PWM-digital conversion circuit block (P / D) 18
A multiplexer (MUX) 20.
A look-up table (LUT) 21;
In FIG. 3, wiring is omitted.

(Process flow)
Next, the flow of processing executed by each circuit block will be described with reference to FIG. FIG. 4 is a block diagram schematically showing the flow of data between circuit blocks.

  The image data or the calculation result of the previous layer is stored in the memory circuit block (MEM) 10. Data relating to kernel weights is stored in a kernel data memory circuit block (KMEM) 11.

  Data relating to the kernel weight held in the kernel data memory circuit block (KMEM) 11 is input to the register (REG) 12 and converted into parallel data. Subsequently, the signal is converted into a PWM signal by 51 digital-PWM conversion circuit blocks (D / P) 13. The PWM signal is converted into an analog signal by a PWM-analog conversion block (P / A) 17 and further input to an arithmetic circuit block (PWM-MAC) 16.

  Further, the image data held in the memory circuit block (MEM) 10 or the data of the calculation result by the previous layer passes through the register (REG) 14 and the digital-PWM conversion circuit block (D / P) 15, and the calculation circuit block (PWM-MAC) 16 is input. At this time, the image data output from the memory circuit block (MEM) 10 or the data of the calculation result by the previous layer is also output to the adjacent LSI chip. Further, as shown in FIG. 4, the register (REG) 14 receives image data output from an adjacent LSI chip or data of a calculation result by the previous layer. A process related to data input / output with an adjacent LSI chip and a process related to data input to the register (REG) 14 will be described in detail later.

  The register (REG) 14 converts serial data output from the memory circuit block (MEM) 10 into parallel data. The digital-PWM conversion circuit blocks (D / P) 13 and 15 and the PWM-analog conversion circuit block (P / A) 17 convert the digital signal into a PWM signal and the PWM signal into an analog voltage, respectively. The arithmetic circuit block (PWM-MAC) 16 executes a convolution operation process in parallel on the input data relating to the kernel weight and the image data or the calculation result of the previous layer.

  The output result of the arithmetic circuit block 16 is output as a PWM signal, converted into a digital signal by a PWM-digital conversion circuit block (P / D) 18 and then accumulated by an accumulation circuit block (ACC) 19. The accumulated result is nonlinearly converted by a look-up table (LUT) 21 through a multiplexer (MUX) 20 and held in the memory circuit block (MEM) 10 again.

  By repeating the above processing several times for features and several times for layers, the arithmetic processing of the hierarchical convolutional neural network is realized.

(Circuit block)
Next, a detailed circuit configuration of the arithmetic circuit block (PWM-MAC) 16 among the circuit blocks constituting the LSI chip described above will be described with reference to FIG. FIG. 5 is a diagram showing a circuit configuration of the arithmetic circuit block 16.

The arithmetic circuit 25 (shaded part in the figure) in the arithmetic circuit block 16 in this embodiment has 51 switched current sources (SCS) 23 and one integral capacitor (C) 24 for each arithmetic circuit. (For convenience of space, only five SCSs 23 are shown in FIG. 5). The arithmetic circuit block 16 is composed of 80 arithmetic circuits 25 and matches the number of memory cells in one column of one memory circuit block (MEM) 10 (three arithmetic operations are shown in FIG. Only the circuit is shown). As shown in FIG. 5, the plurality of arithmetic circuits 25 in the arithmetic circuit block 16 share the PWM signal PI _i corresponding to the input image data or the calculation result of the previous layer and the analog voltage VW _j corresponding to the kernel weight data. To do. However, i = 1,..., 130 and j = 1,. The PWM signal PI _i is input from the register (REG) 14 via D / P 15, and the analog voltage VW _j is input from the register (REG) 12 via D / P 12 and P / A 17.

  In the present embodiment, a configuration in which a convolution operation is performed using a kernel having a size of 51 × 51 and an output having a size of 80 × 80 can be obtained in one LSI will be described as an example. Therefore, in order to obtain 80 × 80 output by a convolution operation using a 51 × 51 kernel, the size of two-dimensional input data in one LSI is 130 × 130 (where 80 + 51-1 = 130). The memory circuit block (MEM) 10 is composed of 80 × 80 memory cells.

In this case, the arithmetic circuit 25 operates according to the following procedure.
(1) The input PWM signal PI is input to the switched current source (SCS) 23.
(2) to convert the PWM signal to the charge of the integrating capacitor (C) 24, performs weighting addition by the analog voltage VW _j.
(3) The voltage V _k applied to both ends of the integration capacitor (C) 24 is converted to the PWM signal PO _k by comparing it with the linear ramp signal V _ref .
That is, the following equation is established.
PO ₁ = PI ₁ · VW ₁ +... + PI ₅₁ · VW ₅₁ .
...
PO ₈₀ = PI ₈₀ · VW ₁ +... + PI ₁₃₀ · VW ₅₁ .

  The charge of the integration capacitor (C) 24 is reset by inputting a high signal to RST before the calculation is started.

  Next, other circuit blocks constituting the LSI chip in this embodiment will be described.

PWM- analog conversion circuit (P / A) 17 is further converted to an analog voltage VW _j the weight data of the kernel that has been converted into PWM signals, outputs the converted analog voltage VW _j with respect to operation circuit block 16 To do. The PWM-analog conversion circuit (P / A) 17 is composed of one switched current source, an integration capacitor, and a source follower buffer, and the switched current source converts the PWM signal into the charge of the integration capacitor, The source follower buffer outputs the voltage of the integration capacitor.

  In the present embodiment, the memory circuit block (MEM) 10 and the kernel data memory circuit block (KMEM) 11 are composed of SRAM. REG12, 14, D / P13, 15, ACC19, P / D18, MUX20, and LUT21 are digital circuits and may have any circuit configuration as long as they have the functions described above. Detailed description will be omitted.

(Convolution operation)
An execution flow of the convolution operation executed in the arithmetic circuit block 16 will be described with reference to FIG. FIG. 6 is a diagram schematically showing the state of the convolution operation by the arithmetic circuit block 16.

  As described above, the maximum image size to be calculated by one LSI chip in this embodiment is 130 × 130 pixels (however, as will be described later, all 130 × 130 pixels are stored in the memory of the LSI. Not retained). The maximum pixel size on one side of the kernel is 51, which is equal to the number of switched current sources (SCS) 23 in one arithmetic circuit.

  In such a situation, 130 pixels belonging to one row of the image data or the calculation result of one feature of the preceding layer are simultaneously input to 80 arithmetic circuits 25. In other words, each of the 80 arithmetic circuits 25 acquires 51 data necessary for calculation from the REG 14 through the D / P 15 among 130 pixels belonging to one row of the two-dimensional data to be calculated. Each of the arithmetic circuits 25 executes a convolution operation of the input data and the weight data for one row of the kernel in parallel. This operation (parallel operation) is repeated 51 times for the calculation of 51 rows of the kernel, and further, the operation is repeated twice to execute it divided into positive and negative values of the kernel. Therefore, in order to determine the calculation results for the number of memory cells in one row of the memory circuit block 10, the calculation circuit block 16 executes the parallel calculation 51 × 2 times. Further, the 51 × 2 calculations are executed 80 times in order to determine the calculation results for all 80 rows.

(Parallel operation)
As described above, in order to acquire 80 × 80 data by a convolution operation based on a 51 × 51 kernel, it is necessary to input 130 × 130 two-dimensional data. However, the size of the memory circuit block (MEM) 10 is 80 × 80. For this reason, as shown in FIG. 6, among the data of 130 pixels to be calculated that are input to the calculation circuit 25, the data of the portion exceeding the pixel size of 80 × 80 (shaded portion in the figure) is the LSI that executes the calculation. It is not held in the chip. Therefore, in the configuration according to the present embodiment, LSI chips are connected in parallel, pixel data necessary for processing is acquired from adjacent LSI chips, and convolution calculation of two-dimensional data is performed using the pixel data. In the following, parallel calculation using a plurality of LSI chips will be described by taking as an example the case where four LSI chips are connected in a row and calculation is performed on an image size of 320 × 240 pixels.

  FIG. 7 is a diagram schematically showing how the convolution operation with the kernel is performed on 320 × 240 two-dimensional data using the four LSI chips 1 to 4. In FIG. 7, reference numerals 711 to 714 denote areas to be calculated by the four LSI chips 1 to 4 connected in a row. The LSI chips 1 to 4 perform convolution calculations on data at corresponding positions in the calculation areas 711 to 714 in synchronization with calculations by other LSI chips.

  Reference numerals 701 to 704 denote kernels of the LSI chips 1 to 4, that is, ranges of two-dimensional data necessary for the LSI chips 1 to 4 to perform operations. As illustrated in FIG. 7, when an image having a size larger than the size (80 × 80) of one memory circuit block 10 is an operation target, an image held by the kernel in the memory circuit block 10 of an adjacent LSI chip It overflows into the data or calculation result area. In FIG. 7, the protruding portion is indicated by diagonal lines.

  In the configuration according to the present embodiment, the LSI chips 1 to 4 are adjacently connected in order to perform an operation using the image data included in the protruding portion. Further, between the adjacent LSI chips 1 to 4, computation target data (data of a portion protruding from the kernel) that is not held in the built-in memory is mutually input and output and complemented.

  Next, in the arithmetic processing of the LSI chips 1 to 4, the manner in which data is input / output between adjacent chips when the kernel protrudes into the data area held by the adjacent LSI chips 1 to 4 is shown in FIG. Explanation will be made with reference to FIG. FIG. 8 shows the memory read order of input rows when data to be calculated is read out from the image data held in the memory circuit block 10 of each chip or the calculation result of the previous layer and input / output to / from the register of the adjacent chip. FIG. 4 is a diagram schematically showing the flow of data. FIG. 8 shows only the 80 × 80 area to be calculated in the memory circuit block 10 in each of the LSI chips 1 to 4. In addition, the calculation position of the calculation in the latter layer is displayed as a target row.

  In FIG. 8, it can be seen that the input line data (black portion in the figure) of the protruding portion is taken from the adjacent LSI chip in order to calculate the case where the kernel protrudes. For example, the LSI chip 1 fetches 811 data from the LSI chip 2 in order to perform the calculation at the position 801. The memory reading order at this time and the flow of data when data is input to the register will be described in detail below.

  In the present embodiment, since data is read one by one from the left side of one memory circuit block (input row in the figure), it is first held on the left side of the memory circuit as shown in FIG. Is transferred to the LSI chip adjacent to the left side. For example, 811 data is transferred to the LSI chip 1. In other words, the data exchange in (a) corresponds to the case where a kernel protrusion in each LSI chip occurs on the right side of the kernel. Here, the transferred data is used for the calculation of the target row in the subsequent layer. Although not shown in FIG. 8, the transferred data is also input to the register 14 of the LSI chip itself having the memory circuit block 10 holding the data.

  Thereafter, when the reading of data from the memory circuit block 10 moves to the right side, the kernel does not protrude and only the data from the memory circuit block 10 built in each LSI chip is input to the register 14. .

  Then, when the reading of data from the memory circuit further moves to the right side, this time, the protrusion of the kernel in each LSI chip occurs on the left side of the kernel. Therefore, as shown in FIG. 8B, the data held on the right side of the memory circuit is transferred to the LSI chip adjacent on the right side. For example, 812 data is transferred to the LSI chip 2. Here, the transferred data is used for the calculation of the target row in the subsequent layer. Although not shown in FIG. 8, the transferred data is also input to the register 14 of the LSI chip itself having the memory circuit block 10 holding the data.

  As described above, in the configuration according to the present embodiment, the data (130) necessary for the calculation in each LSI chip is correctly input and held in the registers (REG) 14 of the four LSI chips. The For this reason, in the configuration according to the present embodiment, it is possible to perform computation on two-dimensional data having a size larger than the size that can be stored in the memory of each LSI chip. Furthermore, since the memory reading positions of the LSI chips 1 to 4 are all common to the four LSI chips, the control during reading is easy. Further, since the kernel weight data to be calculated by the LSI chips 1 to 4 is common to the LSI chips 1 to 4, the control is easy.

(Data input / output)
Next, referring to FIG. 4 again, a method for inputting / outputting data with an adjacent LSI chip will be described. As shown in FIG. 4, data input / output between adjacent LSI chips is performed when data is input from the left LSI chip to the right LSI chip and when data is input from the right chip to the left chip. There are two types of cases. Therefore, in this embodiment, the tri-state buffer 26 is used to switch the left / right direction of data input / output. The two tri-state buffers 26 can switch the data output direction between the left side and the right side by controlling with the opposite phase control signals.

  Further, by using the tristate buffer 27 as shown in the block diagram of FIG. 9, it is possible to share wiring for inputting and outputting data from the LSI chip. FIG. 9 is a diagram exemplarily showing a configuration in which wirings for inputting and outputting data from adjacent LSI chips are shared.

  In this case, when capturing the output from the left LSI chip, the wiring is switched so that the data is output to the right LSI chip, and when capturing the output from the right LSI chip, the data is output to the left LSI chip. Switch the wiring. When this method is used, the wiring between adjacent LSI chips is only one set of 6-bit wirings 28 as shown in FIG. 10, and the number of wirings can be reduced.

  As long as data can be input / output between adjacent LSI chips as described above, other configurations such as a wiring structure and a buffer circuit may be used. Input / output wiring may be shared or distributed inside the chip, or shared or distributed outside the chip. FIG. 10 shows a case where wiring is shared inside the chip.

  As described above, the present embodiment discloses an LSI chip that performs a convolution operation between two-dimensional target data to be calculated and a two-dimensional kernel. This LSI chip includes a MEM 10 (target data memory) that holds target data, a KMEM 11 (kernel memory) that holds a kernel, a PWM-MAC 16 (arithmetic circuit), an input / output wiring that inputs and outputs the target data to and from an external LSI chip. . However, the PWM-MAC 16 performs a convolution operation process based on the target data and the kernel. The PWM-MAC 16 is the target data necessary for the calculation in the calculation circuit, and the data not existing in the MEM 10 provided in the LSI chip is provided in the external LSI chip via the input / output wiring. Input from MEM10.

  For this reason, according to the configuration according to the present embodiment, it is possible to efficiently execute arithmetic processing on two-dimensional data exceeding the memory size of the memory circuit block built in the LSI chip. When performing arithmetic processing on large-size two-dimensional data (such as image data), it is possible to perform arithmetic processing without excessively increasing the number of wires. Further, by constructing an arithmetic processing system by connecting a plurality of identical LSI chips, the area of one LSI chip can be reduced, and the yield at the time of manufacturing can be increased. Further, by changing the number of LSI chips to be connected, it is possible to execute a convolution operation on operation target data of various sizes without changing the circuit configuration of the LSI chip.

  In addition, since the arithmetic processing in the arithmetic circuit is executed in parallel, the arithmetic processing can be efficiently executed on the two-dimensional data. Further, in the configuration according to the present embodiment, LSI chips are arranged in a line, so that a convolution operation can be performed on two-dimensional data of various sizes with a small circuit scale.

  It should be noted that the numbers, image sizes, and the like shown in the description of the circuit block and its constituent elements are examples for explaining the present embodiment, and are obviously not limited to this.

<< Second Embodiment >>
In the configuration according to the first embodiment, the arithmetic circuit block 16 is configured by an analog circuit. In the present embodiment, a case where the arithmetic circuit block is configured by a digital circuit will be described. The configuration according to the present embodiment is the same as that of the first embodiment except for the change accompanying the digital circuit of the arithmetic circuit block. For this reason, in this embodiment, only a different part from 1st Embodiment is demonstrated, and since it is the same as that of 1st Embodiment other than that, description is abbreviate | omitted.

(Circuit configuration of LSI chip)
FIG. 11 is a block diagram schematically showing the circuit configuration of an LSI chip that constitutes the arithmetic processing system in the present embodiment. As shown in FIG. 11, the LSI chip in this embodiment has the following circuit blocks.
A memory circuit block (MEM) 10;
Kernel data memory circuit block (KMEM) 11
Registers (REG) 12,14.
A digital arithmetic circuit block (D-MAC) 29.
An accumulation circuit block (ACC) 19;
A multiplexer (MUX) 20.
A look-up table (LUT) 21;
In FIG. 11, wiring is omitted.

(Process flow)
Next, the flow of processing executed by each circuit block will be described with reference to FIG. FIG. 12 is a block diagram schematically showing the flow of data between circuit blocks.

  The image data or the calculation result of the previous layer is stored in the memory circuit block (MEM) 10. Data relating to kernel weights is stored in a kernel data memory circuit block (KMEM) 11.

  Data relating to the kernel weight held in the kernel data memory circuit block (KMEM) 11 is input to the register (REG) 12 to be converted into parallel data, and further input to the arithmetic circuit block (D-MAC) 29. The image data or the calculation result of the previous layer is input to the arithmetic circuit block (D-MAC) 29 through the register (REG) 14. At this time, the image data output from the memory circuit block (MEM) 10 or the data of the calculation result of the previous layer is also output to the adjacent LSI chip at the same time. Further, as shown in FIG. 12, the register (REG) 14 receives image data output from an adjacent LSI chip or data of an operation result of the previous layer. Here, the register (REG) 14 converts serial data output from the memory circuit block (MEM) 10 into parallel data.

  The arithmetic circuit block (D-MAC) 29 is composed of a digital multiplier circuit, and executes convolution arithmetic processing in parallel on the input data relating to the kernel weight and the image data or the calculation result of the previous layer. The output result of the arithmetic circuit block (D-MAC) 29 is accumulated by the accumulation circuit block (ACC) 19. The accumulated result is nonlinearly converted by a look-up table (LUT) 21 through a multiplexer (MUX) 20 and stored again in the memory circuit block (MEM) 10.

  By repeating the above processing several times for features and several times for layers, the arithmetic processing of the hierarchical convolutional neural network is realized.

(Arithmetic circuit block)
Next, a detailed circuit configuration of the arithmetic circuit block (D-MAC) 29 among the circuit blocks constituting the LSI chip described above will be described with reference to FIG. FIG. 13 is a diagram showing a circuit configuration of the arithmetic circuit block 29.

One arithmetic circuit 31 in the arithmetic circuit block (D-MAC) 29 in the present embodiment has 51 digital multiplier circuits (MUL) 30, and each multiplier circuit 30 is an accumulator circuit in the subsequent stage. It is connected to a block (ACC) 19. However, only five MULs 30 are shown in FIG. 5 for the sake of space. The arithmetic circuit block (D-MAC) 29 is composed of 80 arithmetic circuits 31 and matches the number of memory cells in one column of one memory circuit block (in FIG. Only the arithmetic circuit is described). As shown in FIG. 13, the plurality of arithmetic circuits 31 in the arithmetic circuit block 29 share a digital signal DI _i corresponding to input two-dimensional data and a digital voltage DW _j corresponding to kernel weight data. However, i = 1,..., 130 and j = 1,. The input digital signal DI _i is input from the register (REG) 14, and the digital voltage DW _j is input from the register (REG) 12.

  In this embodiment as well, a configuration in which a convolution operation is performed using a kernel with a size of 51 × 51 in one LSI and an output with a size of 80 × 80 can be obtained will be exemplarily described. For this reason, as in the first embodiment, in order to obtain 80 × 80 output by a convolution operation using a 51 × 51 kernel, the size of the two-dimensional input data in one LSI is 130 × 130. The memory circuit block (MEM) 10 is composed of 80 × 80 memory cells.

In this case, the arithmetic circuit 29 and the accumulator circuit 19 connected to the subsequent stage operate in the following procedure.
(1) The digital signals DI and DW are input to the multiplication circuit (MUL) 30.
(2) The multiplication result of DI and DW is output from the multiplication circuit (MUL) 30 and input to the accumulation circuit block (ACC) 19.
(3) The accumulation circuit block (ACC) 19 accumulates inputs from a maximum of 51 multiplication circuits (MUL) 30.

  As described above, according to the configuration of the present embodiment, the arithmetic operation performed by the arithmetic circuit in the first embodiment can be executed digitally. Note that the arithmetic processing system in the present embodiment is the same as the configuration according to the first embodiment except for the change accompanying the digital circuit of the arithmetic circuit block described above. Therefore, the method for connecting the LSI chips in one column and inputting / outputting data to be calculated is the same as in the first embodiment, and the description thereof is omitted. Further, the input / output method of data between adjacent LSI chips may share the input wiring and the output wiring, as in the first embodiment. Further, other circuit configurations may be used as long as the same function can be realized.

  In the above description, the accumulation circuit has shown an example of accumulating the multiplication results (51) of 51 arithmetic circuits in parallel, but is not limited thereto. For example, as shown in FIG. 14, 51 multiplication circuits may be connected to the accumulation circuit block by BUS, and 51 multiplication results may be serially accumulated by pipeline processing. Further, even if the configuration is other than that, the arithmetic circuit block may have any configuration as long as it is a digital circuit capable of realizing the same calculation.

<< Third Embodiment >>
In the configuration according to the first and second embodiments, LSI chips are connected in a row. In the present embodiment, a description will be given of a configuration capable of executing higher-speed arithmetic processing by arranging and connecting a plurality of LSI chips in a plane.

(Calculation processing system)
FIG. 15 is a diagram schematically showing an arithmetic processing system according to the present embodiment. As shown in FIG. 15, the arithmetic processing system is configured by arranging 12 LSI chips 01 to 12 in 3 rows and 4 columns. Note that the number of LSI chips can be changed according to the data size to be processed, and in this embodiment, 12 cases are shown. Each LSI chip is connected to an adjacent LSI chip by wiring 5 for inputting and outputting data. Here, in this embodiment, the data size input and output between the chips is 6 bits, but may have other data sizes. In that case, the number of wires changes according to the data size. In FIG. 15, wirings other than the input and output wirings described above are not shown because they are not necessary for the description of this embodiment.

  Note that the arithmetic processing realized by the arithmetic processing system in this embodiment is similar to the first embodiment in that image data is input to the first layer and a convolution operation with a kernel having a predetermined two-dimensionally distributed weight. Is repeatedly executed in a hierarchical manner. Therefore, the details of the arithmetic processing are the same as those in the first embodiment, and thus description thereof is omitted.

(Circuit configuration of LSI chip)
Next, the circuit configuration of the LSI chips 01 to 12 constituting the arithmetic processing system will be described with reference to FIG. FIG. 16 is a block diagram schematically showing the circuit configuration of the LSI chip.

As shown in FIG. 16, the LSI chip in this embodiment has the following circuit blocks.
A memory circuit block (MEM) 10;
Kernel data memory circuit block (KMEM) 11
Registers (REG) 12,14.
Digital-PWM conversion circuit blocks (D / P) 13, 15
PWM-analog conversion circuit (P / A) 17
Arithmetic circuit block (PWM-MAC) 16
An accumulation circuit block (ACC) 19;
PWM-digital conversion circuit block (P / D) 18
A multiplexer (MUX) 20.
A look-up table (LUT) 21;
A selector (SEL) 32.
The configuration of each circuit block and the flow of processing are the same as those in the first embodiment except that data is input / output with an adjacent chip via the selector 32 in this embodiment, as will be described later. Therefore, the description is omitted. The execution flow of the convolution operation by the arithmetic circuit block of the LSI chip is also the same as that in the first embodiment, and thus the description thereof is omitted.

(Parallel operation)
FIG. 17 is a diagram schematically showing a state in which a convolution operation with a kernel is performed on 320 × 240 two-dimensional data using twelve LSI chips 01 to 12. As shown in FIGS. 15 and 17, in this embodiment, four rows of LSI chips arranged in one column in the first embodiment are arranged in three rows, so that computation is performed by sequential processing in the first embodiment. The executed vertical calculation step can be executed by 1/3 of the calculation steps. Note that the LSI chips 01 to 12 perform a convolution operation on the data at the corresponding position in each operation area in synchronization with the operation by other LSI chips.

  In the present embodiment, as shown in FIG. 17, LSI chips arranged in 3 rows and 4 columns with respect to image data of 320 × 240 pixels are configured as an arithmetic processing system. For this reason, there occurs a case where the kernel overlaps the data area of the pixel held by the four LSI chips in the memory circuit. Therefore, in the configuration according to the present embodiment, operation target data (data of a portion protruding from the kernel) that is not held in the built-in memory is mutually input and output between adjacent LSI chips. In the present embodiment, the adjacent LSI chips include chip pairs arranged in an oblique direction.

Hereinafter, in the arithmetic processing of each LSI chip, the state of data input / output between adjacent chips when the kernel protrudes into the data area held by the adjacent LSI chip will be described in detail. In the present embodiment, there are the following three cases where the kernel protrudes from the memory circuit in each chip.
(1) Cases that protrude to the left or right (2) Cases that protrude to the left or right, and the lower or lower left or lower right (3) Cases that protrude to the left or right and upper, upper left, or upper right of these (1 In the case of ()), data is input / output only between the LSI chips adjacent to the left and right, and therefore the data flow between the LSI chips is the same as that of the first embodiment. Therefore, the description is omitted.

  Subsequently, cases (2) and (3) will be described with reference to FIGS. The processing when the kernel in (2) and (3) protrudes to the left and right is to input and output data only between the LSI chips adjacent to the left and right, as in the case of (1). The data flow between the LSI chips is the same as in the first embodiment. For this reason, detailed description is abbreviate | omitted in this embodiment.

  18 and 19 show the memory read order of input rows when data to be calculated is read out from the image data held in the memory circuit block of each chip or the calculation result of the previous layer and input / output to / from the register of the adjacent chip. FIG. 5 is a diagram illustrating a flow of data. However, FIG. 18 is a diagram related to the case (2) above, and FIG. 19 is a diagram related to the case (3) above. FIGS. 18 and 19 show only the 80 × 80 area to be calculated in the memory circuit block in each LSI chip. In addition, the calculation position of the calculation in the subsequent layer is displayed as a target row.

  In (2), since data is read one by one from the left side of one memory block (input row in FIG. 18), first, as shown in FIG. 18 (a), the data is held at the upper left side of the memory circuit. The transferred data is transferred to the LSI chip adjacent on the upper left side and the upper side. For example, 1811 data is transferred to the LSI chip 01 to perform the calculation at the position 1801, and for example, 1811 data is transferred to the LSI chip 02 to perform the calculation at the position 1802. Here, the transferred data is used for the calculation of the target row in the subsequent layer. That is, the data exchange in (a) corresponds to the case where the protrusion of the kernel in each LSI chip occurs on the lower right side and the lower side of the kernel. Although not shown in FIG. 18, the transferred data is also input to the register (REG) 14 of the LSI chip itself having the memory circuit block (MEM) 10 holding this data. ing.

  Next, as shown in FIG. 18B, when data reading from the memory circuit moves to the right side (input row in FIG. 18), the data held at the upper center of the memory circuit is changed. , And passed to the LSI chip adjacent on the upper side. For example, 1812 data is transferred to the LSI chip 02 in order to perform calculation at the position 1803. Here, the transferred data is used for the calculation of the target row in the subsequent layer. In other words, the data exchange in (b) corresponds to the case where the protruding of the kernel in each LSI chip occurs below the kernel. Although not shown in FIG. 18, the transferred data is also input to the register (REG) 14 of the LSI chip itself having the memory circuit block (MEM) 10 holding this data. ing.

  Then, when the reading of data from the memory circuit further moves to the right (input line in FIG. 18), the protrusion of the kernel in each LSI chip occurs on the lower left side and the lower side of the kernel. For this reason, as shown in FIG. 18C, data held on the upper right side of the memory circuit is transferred to the LSI chips adjacent on the upper right side and the upper side. For example, 1813 data is transferred to the LSI chip 01 in order to perform calculation at the position 1804, and for example, 1813 data is transferred to the LSI chip 02 in order to perform calculation at the position 1805. Here, the transferred data is used for the calculation of the target row in the subsequent layer. Although not shown in FIG. 18, the transferred data is also input to the register (REG) 14 of the LSI chip itself having the memory circuit block (MEM) 10 holding this data. ing.

  Next, the case (3) will be described with reference to FIG. In (3), since data is read one by one from the left side of one memory block (input row in FIG. 19), first, as shown in (a) of FIG. 19, the data is held at the lower left side of the memory circuit. The transferred data is transferred to the LSI chip adjacent to the lower left side and the lower side. For example, 1911 data is transferred to the LSI chip 05 to perform the calculation at the position 1901, and for example, 1911 data is transferred to the LSI chip 06 to perform the calculation at the position 1902. Here, the transferred data is used for the calculation of the target row in the subsequent layer. That is, the data exchange in (a) corresponds to the case where the protruding of the kernel in each LSI chip occurs on the upper right side and the upper side of the kernel. Although omitted in FIG. 19, the transferred data is also input to the register (REG) 14 of the LSI chip itself having the memory circuit block (MEM) 10 holding the data. ing.

  Next, as shown in FIG. 19B, when data reading from the memory circuit moves to the right (input row in the figure), the data held at the upper center of the memory circuit is It is delivered to the LSI chip adjacent to the lower side. For example, 1912 data is transferred to the LSI chip 06 in order to perform calculation at the position 1903. Here, the transferred data is used for the calculation of the target row in the subsequent layer. In other words, the data exchange in (b) corresponds to the case where the protruding of the kernel in each LSI chip occurs on the upper side of the kernel. Although omitted in FIG. 19, the transferred data is also input to the register (REG) 14 of the LSI chip itself having the memory circuit block (MEM) 10 holding the data. ing.

  Then, when the reading of data from the memory circuit further moves to the right (input line in the figure), this time, the protrusion of the kernel in each LSI chip occurs on the upper left side and the upper side of the kernel. For this reason, as shown in FIG. 19C, data held on the lower right side of the memory circuit is transferred to the LSI chips adjacent on the lower right side and the lower side. For example, 1913 data is transferred to the LSI chip 05 to perform calculation at the position of 1904, and for example, 1913 data is transferred to the LSI chip 06 to perform calculation at the position of 1905. Here, the transferred data is used for the calculation of the target row in the subsequent layer. Although omitted in FIG. 19, the transferred data is also input to the register (REG) 14 of the LSI chip itself having the memory circuit block (MEM) 10 holding the data. ing.

  The above description relates to the case where there are eight adjacent LSI chips. If there are less than eight adjacent LSI chips, the above-described data exchange may not occur. In this case, however, the adjacent LSI chip is simply not connected to the input / output wiring, and no special circuit configuration or control is required.

  In this way, 130 pieces of data to be calculated are correctly input and held in the shift registers of the 12 LSI chips. In this case, since the memory read position of each LSI chip is common to the 12 LSI chips, the control during reading is simplified. In addition, since the kernel weight data to be calculated in each LSI chip is common, the control is simplified.

  Next, FIG. 20 shows a block diagram to which a data flow with an adjacent LSI chip is added. As shown in FIG. 20, each LSI chip is connected to a maximum of eight adjacent LSI chips via a selector 32. That is, when data is input / output between adjacent chips as described above, the selector is switched to output data to an appropriate adjacent chip and receive input from the appropriate adjacent chip. As long as data can be input / output between adjacent LSI chips as described above, other configurations such as a wiring structure and a selector may be used.

  As described above, by connecting LSI chips in a matrix and operating them in parallel as in the present embodiment, it is possible to process a large size image at higher speed. Further, by changing the number of LSI chips to be connected, it is possible to execute a convolution operation on operation target data of various sizes without changing the circuit configuration of the LSI chip.

<< Fourth Embodiment >>
In the configuration according to the third embodiment, the arithmetic circuit block 16 is configured by an analog circuit. In the present embodiment, a case where the arithmetic circuit block is configured by a digital circuit will be described. The configuration according to the present embodiment is the same as that of the third embodiment except for the change accompanying the digital circuit of the arithmetic circuit block. For this reason, in this embodiment, only a different part from 3rd Embodiment is demonstrated, and since it is the same as that of 3rd Embodiment other than that, description is abbreviate | omitted.

(Circuit configuration of LSI chip)
FIG. 21 is a block diagram schematically showing a circuit configuration of an LSI chip constituting the arithmetic processing system in the present embodiment. As shown in FIG. 21, the LSI chip in this embodiment has the following circuit blocks.
A memory circuit block (MEM) 10;
Kernel data memory circuit block (KMEM) 11
Registers (REG) 12,14.
A digital arithmetic circuit block (D-MAC) 29.
A selector (SEL) 32.
An accumulation circuit block (ACC) 19;
A multiplexer (MUX) 20.
A look-up table (LUT) 21;
In FIG. 21, wiring is omitted.

(Process flow)
Next, the flow of processing executed by each circuit block will be described with reference to FIG. FIG. 22 is a block diagram schematically showing the flow of data between circuit blocks.

  The image data or the calculation result of the previous layer is stored in the memory circuit block (MEM) 10. Data relating to kernel weights is stored in a kernel data memory circuit block (KMEM) 11.

  Digital data relating to the kernel weight held in the kernel data memory circuit block (KMEM) 11 is input to the register (REG) 12 to be converted into parallel data, and further input to the arithmetic circuit block (D-MAC) 29. . The image data or the calculation result of the previous layer is input to the arithmetic circuit block (D-MAC) 29 through the register (REG) 14. At this time, the image data output from the memory circuit block (MEM) 10 or the data of the operation result of the previous layer is simultaneously output to the adjacent appropriate LSI chip via the selector (SEL) 32. Further, as shown in FIG. 22, the register (REG) 14 receives image data output from an adjacent LSI chip or data of the operation result of the previous layer. Here, the register (REG) 14 converts serial data output from the memory circuit block (MEM) 10 into parallel data.

  The arithmetic circuit block (D-MAC) 29 is composed of a digital multiplier circuit (MUL) 30, and performs convolution arithmetic processing in parallel on input kernel weight data and image data or previous layer arithmetic results. Execute. The output result of the arithmetic circuit is accumulated by an accumulation circuit block (ACC) 19. The accumulated result is nonlinearly converted by a look-up table (LUT) 21 through a multiplexer (MUX) 20 and stored again in the memory circuit block (MEM) 10.

  By repeating the above processing several times for features and several times for layers, the arithmetic processing of the hierarchical convolutional neural network is realized.

  As described above, according to the configuration of the present embodiment, the operation executed by the arithmetic circuit in the third embodiment can be digitally executed. Note that, among the circuit blocks constituting the LSI chip described above, the detailed circuit configuration of the arithmetic circuit block (D-MAC) 29 is the same as that described in the second embodiment, and thus detailed description thereof is omitted. . At this time, the arithmetic processing system in the present embodiment is the same as that of the third embodiment except for the change accompanying the digital circuit of the arithmetic circuit block described above. Therefore, a method for connecting LSI chips in a matrix and inputting / outputting data to be calculated is the same as in the third embodiment, and a description thereof is omitted.

It is the figure which showed the arithmetic processing system typically. It is the figure which showed typically the process which performs the position detection of a face using a hierarchical convolutional neural network. 1 is a block diagram schematically showing a circuit configuration of an LSI chip. It is the block diagram which showed typically the flow of the data between circuit blocks. It is the figure which showed the circuit structure of the arithmetic circuit block. It is the figure which showed typically the mode of the convolution calculation by an arithmetic circuit block. It is the figure which showed typically a mode that the convolution calculation with a kernel was performed with respect to two-dimensional data using four LSI chips. It is the figure which showed typically the memory reading order and the flow of data at the time of reading the data used as a calculation target from the memory circuit block of an adjacent LSI chip. It is the figure which showed exemplarily the structure where the input / output wiring with an adjacent LSI chip was shared. It is the figure which showed illustratively the structure which wiring shared within the chip. 1 is a block diagram schematically showing a circuit configuration of an LSI chip. It is the block diagram which showed typically the flow of the data between circuit blocks. It is the figure which showed the circuit structure of the arithmetic circuit block. It is the figure which showed the circuit structure of the arithmetic circuit block. It is the figure which showed the arithmetic processing system typically. 1 is a block diagram schematically showing a circuit configuration of an LSI chip. It is the figure which showed typically a mode that the convolution calculation with a kernel was performed with respect to two-dimensional data using 12 LSI chips. It is the figure which showed typically the memory reading order and the flow of data at the time of reading the data used as a calculation target from the memory circuit block of an adjacent LSI chip. It is the figure which showed typically the memory reading order and the flow of data at the time of reading the data used as a calculation target from the memory circuit block of an adjacent LSI chip. It is the block diagram which showed typically the flow of the data between circuit blocks. It is the block diagram which showed typically the circuit structure of the LSI chip which comprises an arithmetic processing system. It is the block diagram which showed typically the flow of the data between circuit blocks. It is the figure which showed typically the example of a structure of the arithmetic processing system formed by connecting the some board | substrate with which the arithmetic circuit was mounted. It is the figure which showed typically the data flow in the LSI chip which performs a convolution operation with respect to two-dimensional data.

Claims (3)

An arithmetic processing system that connects a plurality of LSI chips and performs a convolution operation between two-dimensional target data and a two-dimensional kernel,
Each of the LSI chips is
A target data memory for holding the target data;
A register that holds the target data required for the operation;
Kernel memory holding the kernel;
An arithmetic circuit that performs convolution arithmetic processing based on the target data held in the register and the kernel;
An input / output wiring connected to an output line from the target data memory to the register, and for inputting / outputting the target data to and from the adjacent LSI chip;
A switch circuit for switching connection with the adjacent LSI chip in the input / output wiring,
The target data is distributed and held in the target data memory provided in each of the plurality of adjacent LSI chips,
In each of the plurality of LSI chips, when the target data is output from the target data memory provided in the LSI chip to the register provided in the LSI chip, in the calculation in the arithmetic circuit of the adjacent LSI chip Necessary target data that does not exist in the target data memory provided in the adjacent LSI chip is output to the register provided in the same LSI chip as the target data memory that holds the data. In addition, an operation processing system characterized in that, by switching the connection in the input / output wiring by the switch circuit , the output is simultaneously output to the register provided in the adjacent LSI chip via the input / output wiring. The arithmetic processing system according to claim 1, wherein the LSI chips are arranged in one row. The arithmetic processing system according to claim 1, wherein the LSI chips are arranged in a matrix format.

Images