TWI727643B - Artificial intelligence accelerator and operation thereof - Google Patents

Abstract

An artificial intelligence accelerator receives an input data set in binary and a selected layer of overall weight pattern layers. The artificial intelligence accelerator includes multiple processing tiles and a summation output circuit. Each processing tile receives one of input data subsets of the input data set and performs convolution operation respectively on weight blocks of each sub weight pattern of the overall weight pattern to obtain weight operation values and then obtain a weight output as expected by direct convolution operation on the input data subset with the sub weight pattern through performing multiple stages of shifting and adding operation on the weight operation values. The summation output circuit obtains a summation value as expected by direct convolution operation on the input data set with the overall weight pattern through performing multiple stages of shifting and adding operation in sum on the weight output values of the processing tiles.

Description

Artificial intelligence accelerator and its processing method

The present invention relates to the technology of artificial intelligence accelerators, and particularly relates to artificial intelligence accelerators composed of divided input bits and divided weight blocks.

The application of artificial intelligence accelerators includes, for example, the role of a filter, which distinguishes how well the shape represented by the input data matches the known shape. An application, for example, uses an artificial intelligence accelerator to identify whether the captured image contains information such as eyes, nose, face, and so on.

The data required by the artificial intelligence accelerator is, for example, the data of all the pixels of an image, that is, the input data is a large number of bits of data. After these data are input in parallel, they are stored in the artificial intelligence accelerator for comparison in various forms. Operation. These patterns are stored in a large number of memory cells in the form of weights. The memory cell structure is a 3D structure, which is composed of multiple 2D memory cell layers. Each layer represents a characteristic shape, which is stored in a memory cell array of one layer in the form of a weight value, which is sequentially selected by the word line control The memory cell array of one layer to be processed. The input data is input by bit lines. Input data and memory cell array for convolution operation (convolution operation) obtains the degree of coincidence of the characteristic patterns of the memory cell array corresponding to this layer.

Artificial intelligence accelerators have to deal with a large amount of calculations. If a multi-layer memory cell array is assembled in one unit and the bit is used as the processing unit, the overall circuit will be very large. However, the operation speed will be delayed and consume energy. From the point of view that artificial intelligence accelerators require fast processing to filter and distinguish the content of the input image, the design of a single circuit chip generally needs to continue to be improved, such as the consideration of operating speed.

The embodiment of the present invention provides an artificial intelligence accelerator. The artificial intelligence accelerator is composed of divided input bits and divided weight blocks, and then the value combination of the respective parallel operations is restored to The expected calculation result for a single chip. In this way, the processing speed of the artificial intelligence accelerator can be effectively increased, and energy consumption can be reduced.

In one embodiment, the present invention provides an artificial intelligence accelerator that receives a two-bit input data set and a selected one of the multi-layer overall weight forms to perform a convolution operation. The input data group is multiple sub-data groups. The artificial intelligence accelerator includes multiple processing chips and a total output circuit. Each of the processing chips includes: receiving end components, which respectively receive a sub-data group. The weight storage section stores a partial weight form of the overall weight form, wherein the weight storage section includes a plurality of weight blocks, and each of the weight blocks stores a block part of the partial weight form in a bit order, wherein the weight The memory cell array structure of the storage unit is planned to perform a convolution operation on the data group and each part of the block relative to the corresponding data group. Multiple weight calculation values in sequence. The block-by-block output circuit includes multiple shifters and multiple adders. The sum of the multiple weight calculation values is obtained through multi-level shift and addition operations. The convolution calculation is directly performed on the data group and the part of the weight form. The expected weight output value. The summation output circuit includes multiple shifters and multiple adders. The sum of the multiple weighted output values is obtained by multi-level shift and addition operations. The input data group and the overall weight form are directly convoluted. The expected sum value.

In one embodiment, for the artificial intelligence accelerator, the input data group includes i bits, divided into p sub-data groups, i and p are integers, and each sub-data group contains i/p bits yuan.

In one embodiment, for the artificial intelligence accelerator, the input data group includes i bits, the number of the plurality of processing slices is p, the input data group is p sub-data groups, and the i and p is an integer greater than or equal to 2, the i is greater than the p, and each sub-data group contains i/p bits.

In one embodiment, for the artificial intelligence accelerator, the number of weight blocks included in the weight storage section is q, and q is an integer greater than or equal to 2, and the weight storage section includes j bits, and j and q are integers greater than or equal to 2, the j is greater than the q, and each weight block contains j/q memory cells.

In one embodiment, for the artificial intelligence accelerator, the total number of memory cells of the plurality of weight storage parts of the plurality of processing slices is p*q*2 ^(i/p+j/q) .

In one embodiment, for the artificial intelligence accelerator, for the block-by-block output circuit, each stage of the shift and add operation includes at least one shifter and at least one adder. For multiple input values of each order, take two adjacent ones as A processing unit, in which the input value belonging to the higher bit passes through the shifter and the input value of the lower bit is added by the adder and then output to the next stage, where the single output value of the last stage is regarded as a pair The weight output value of the slice should be processed.

In one embodiment, for the artificial intelligence accelerator, the shift amount of the shifter in the first stage is j/q memory cells, and the shift amount of the shifter in the later stage is that of the previous stage Twice the amount of shift of this shifter.

In one embodiment, for the artificial intelligence accelerator, for the sum output circuit, at least one shifter and at least one adder are included in the shift and add operation of each stage. For multiple input values of each level, two adjacent ones are used as a processing unit, where the input value belonging to the higher bit passes through the shifter and the lower bit input value is added by the adder Output to the next level, where the single output value of the last level is regarded as the total value.

In one embodiment, for the artificial intelligence accelerator, the shift amount of the shifter in the first stage is i/p bits, and the shift amount of the shifter in the later stage is that of the previous stage Twice the amount of shift of this shifter.

In one embodiment, for the artificial intelligence accelerator, it further includes a normalization processing circuit, which normalizes the total value to obtain a normalized total value, and a quantization processing circuit, which normalizes the normalized total value by a base number Turn it into an integer value.

In one embodiment, for the artificial intelligence accelerator, the processing circuit includes a plurality of sense amplifiers, which respectively sense each part of the block to perform a convolution operation to obtain a plurality of sensed values, which are used as the multiple weighted calculation values.

In one embodiment, the present invention further provides a processing method for artificial intelligence accelerators. The artificial intelligence accelerator receives a two-bit input data group and a selected one of the multi-layer overall weight forms to perform a convolution operation, and the input data group is a plurality of sub-data groups. The processing method includes the use of multiple processing slices, each of which includes execution. Use the receiving end component to receive one of this data group respectively. A weight storage unit is used to store a partial weight form of the overall weight form, wherein the weight storage part includes a plurality of weight blocks, and each of the weight blocks stores a block part of the partial weight form in a bit order, wherein the weight storage part contains a plurality of weight blocks. The memory cell array structure of the weight storage part is planned to perform a convolution operation on the data group with each part of the block to obtain a plurality of weight operation values in sequence with respect to the corresponding data group. Using a block-by-block output circuit that includes multiple shifters and multiple adders, the sum of the multiple weight calculation values is obtained by multi-level shift and addition operations. The data group and the part of the weight form are directly convolved. Calculate the expected weight output value. Using a summation output circuit that includes multiple shifters and multiple adders, the sum of the multiple weighted output values is obtained by multi-level shift and addition operations. The input data group and the overall weight form are directly convolved The sum value expected by the operation.

In one embodiment, for the processing method of the artificial intelligence accelerator, the input data group includes i bits, divided into p sub-data groups, i and p are integers, and each sub-data group includes i/ p bits.

In one embodiment, for the processing method of the artificial intelligence accelerator, the input data group includes i bits, the number of the plurality of processing slices is p, and the input data group is p sub-data groups, The i and p are integers greater than or equal to 2, the i is greater than p, and each sub-data group contains i/p bits.

In one embodiment, for the processing method of the artificial intelligence accelerator, the number of the plurality of weight blocks included in the weight storage unit is q, and q is an integer greater than or equal to 2, and the weight storage unit includes j bits Element, the j and q are integers greater than or equal to 2, the j is greater than the q, each weight block contains j/q memory cells, and the memory of the multiple weight storage parts of the multiple processing slices The total amount of cells is p*q*2 ^(i/p+j/q) .

In one embodiment, for the processing method of the artificial intelligence accelerator, for the operation of the block-by-block output circuit, each step of the shift and addition operation includes using at least one shifter and at least one Adder. For multiple input values of each level, two adjacent ones are used as a processing unit, where the input value belonging to the higher bit passes through the shifter and the lower bit input value is added by the adder Output to the next level, where the last level output a single value as the weighted output value corresponding to the processed slice.

In one embodiment, for the processing method of the artificial intelligence accelerator, the shift amount of the shifter in the first stage is j/q memory cell, and the shift amount of the shifter in the later stage is the former The first-order shift is twice the amount of shift of the shifter.

In one embodiment, for the processing method of the artificial intelligence accelerator, for the operation of the summing output circuit, each step of the shift and add operation includes at least one shifter and at least one adder Device. For multiple input values of each level, two adjacent ones are used as a processing unit, where the input value belonging to the higher bit passes through the shifter and the lower bit input value is added by the adder Output to the next The first order, where the last order outputs a single value as the total value.

In one embodiment, for the processing method of the artificial intelligence accelerator, the shift amount of the shifter in the first stage is i/p bits, and the shift amount of the shifter in the later stage is the former The first-order shift is twice the amount of shift of the shifter.

In one embodiment, the processing method of the artificial intelligence accelerator further includes: using a normalization processing circuit to normalize the total value to obtain a normalized total value; and using a quantization processing circuit to convert The normalized sum value is quantified into an integer value.

In one embodiment, for the processing method of the artificial intelligence accelerator, the processing circuit includes a plurality of sense amplifiers, and the multiple sense amplifiers are used to respectively sense each part of the block and perform a convolution operation to obtain a plurality of sensed values. Take it as the multiple weight calculation values.

In order to make the above-mentioned features and advantages of the present invention more comprehensible, the following specific embodiments are described in detail in conjunction with the accompanying drawings.

20: Artificial Intelligence Accelerator

50: Input data

52: receiving end components

54: storage unit

56: Memory cell array structure

58: output data

60: induction amplifier

100_1, 100_2, 100_(p-1), 100_p: processing slices

102_1, 102_2, 102_p: secondary input data group

104_1, 104_2, 104_p: output data

300: storage unit

302: weight block

308, 314, 316: shifter

312, 318: Adder

350, 354, 356: shifter

352, 358: Adder

400: Processing circuit

402: Multiplier

404: constant

406: Adder

408: offset

500: quantization circuit

502: Divider

504: base

600: Overall system

602: Artificial Intelligence Accelerator

604: Control Unit

700: memory

S100, S102, S104, S106, S108: steps

FIG. 1 is a schematic diagram of the basic architecture of an artificial intelligence accelerator according to an embodiment of the present invention.

FIG. 2 is a schematic diagram of the operation mechanism of an artificial intelligence accelerator according to an embodiment of the present invention.

FIG. 3 is a schematic diagram of the planning of an artificial intelligence accelerator according to an embodiment of the present invention.

FIG. 4 is a schematic diagram of the planning of an artificial intelligence accelerator according to an embodiment of the present invention.

FIG. 5 is a schematic diagram of a memory cell structure of a storage unit according to an embodiment of the present invention.

FIG. 6 is a schematic diagram of a mechanism for summing multiple weight blocks for one processing slice according to an embodiment of the present invention.

FIG. 7 is a schematic diagram of the operation mechanism of the summing circuit between multiple processing slices according to an embodiment of the present invention.

FIG. 8 is a schematic diagram of the overall application configuration of an artificial intelligence accelerator according to an embodiment of the present invention.

FIG. 9 is a schematic flowchart of an artificial intelligence accelerator processing method according to an embodiment of the present invention.

The embodiment of the present invention provides an artificial intelligence accelerator, which is composed of divided input bits and divided weight blocks. The divided input bits are used in parallel with the divided weight blocks, and the values of the parallel operations are combined by shift and addition operations to restore the expected results of a single chip. In this way, the processing speed of the artificial intelligence accelerator can be effectively increased, and energy consumption can be reduced.

A number of embodiments are provided below to illustrate the present invention, but the present invention is not limited to the illustrated embodiments.

FIG. 1 is a schematic diagram of the basic architecture of an artificial intelligence accelerator according to an embodiment of the present invention. Referring to FIG. 1, the artificial intelligence accelerator 20 includes a 3D structure configured The NAND storage unit 54 includes a multi-layer 2D memory array. Each memory cell of the memory array of each layer stores a weight value. All the weight values of the memory array of each layer constitute a weight form according to predetermined characteristics. The weight form is, for example, the form data to be recognized, such as the form data of the face, ears, eyes, nose, mouth, or objects. Each weighting form is stored in a layer of the 3D NAND memory cell 54 in a 2D memory array.

With the cell array structure 56 of the artificial intelligence accelerator 20 corresponding to the input data in a line arrangement, the weight form stored in the memory cell can be convoluted with the input data 50 received and converted by the receiving terminal element 52. This convolution operation is generally, for example, a matrix multiplication operation to obtain an output value. The output data 58 is obtained by performing a convolution operation on the memory cell array structure 56 according to the weight form of a layer. The output data 58 may indicate the degree of matching between the input data 50 and the weight pattern. The convolution operation can be performed according to the general method in the field, without any restrictions. The detailed operation will not be described here. In terms of performance, the weight form of each layer is similar to the filter layer of an object, and the identification function is achieved by identifying the degree to which the input data 50 matches.

FIG. 2 is a schematic diagram of the operation mechanism of an artificial intelligence accelerator according to an embodiment of the present invention. Referring to FIGS. 1 and 2, the input data 50 is, for example, digital data of an image. Taking a dynamically detected image as an example, part or all of the actual image captured by the camera at any time will be identified by the artificial intelligence accelerator 20 whether it contains at least one of the various objects stored in the storage unit 54. Due to the improvement of image resolution, the data of an image contains a large amount of data. The structure of the storage unit 54 is a 3D multi-layer 2D Memory cell array. The memory cell array of one layer includes i bit lines for inputting data and j selection lines corresponding to the weight rows. That is, the storage unit 54 for storing the weight is composed of a multi-layer i*j matrix. The parameters i and j are very large integers. The input data 50 is received by the bit lines of the storage unit 54 which respectively receive the pixel data of the image. Through the processing circuit arranged in the periphery, the input data 50 and the weight are subjected to a convolution operation including matrix multiplication, and the output data 58 after the operation is output.

As for the more direct convolution operation, it can use a single bit and a single weight to perform one-by-one operations. However, due to the large amount of data to be processed, the overall storage unit is very large, which constitutes a relatively large processing chip. In operation, its speed will be slower. In addition, the energy consumption (heat) generated during the operation of large-size wafers will also be greater. As for the functions required by the artificial intelligence accelerator, it needs a faster recognition speed, and at the same time, it is also hoped to reduce the energy consumption of the operation.

FIG. 3 is a schematic diagram of the planning of an artificial intelligence accelerator according to an embodiment of the present invention. Referring to Fig. 3, the present invention further proposes an artificial intelligence accelerator operation planning method. The artificial intelligence accelerator of the present invention maintains the overall input data 50 that receives parallel input, but divides the input data 50 (also referred to as the input data group) into a plurality of secondary input data groups 102_1,..., 102_p. Each sub-input data group is respectively processed by a processing tile (processing tile) 100_1,..., 100_p to perform a convolution operation on the sub-input data group 102_1,..., 102_p. Each processing slice 100_1,..., 100_p is only a part of the processing of the overall convolution operation. If the input resource 50 includes i bit lines as an example, the i bit lines are divided into p groups, and p is 2 or an integer greater than 2. Such a processing chip contains i/p bit lines, and receives the secondary input data sets 102_1,..., 102_p, that is, the secondary input data group contains i/p bits of data. Here, the relationship between the parameter i and p is that i can be divisible by p. However, if the p processing slices cannot divide the i bit lines, the last processing slice only processes the remaining bit lines, which is planned according to actual needs without limitation.

According to the architecture of FIG. 3, p processing slices 100_1,..., 100_p are used to process the weight form of the currently selected layer to perform the convolution operation. The overall input data corresponding to the p processing slices is also divided into p secondary input data groups 102_1,..., 102_p and input to the corresponding processing slices 100_1,..., 100_p. The output values 104_1,..., 104_p are obtained after the convolution operation processed by the p processing slices 100_1,..., 100_p, which are, for example, current values. Afterwards, using the shift and addition processing described later, the result corresponding to the convolution operation with the overall input data group and the overall weight form can be obtained.

Regarding the segmentation method of Figure 3, the weight form of the part stored in the processing slice is directly convoluted with the secondary input data set. The efficiency of the convolution calculation may be further improved. An embodiment of the present invention proposes a further block planning for weights.

FIG. 4 is a schematic diagram of the planning of an artificial intelligence accelerator according to an embodiment of the present invention. Referring to Fig. 4, the overall predetermined input data group includes i data sorted from 0 to i-1, for example, a _{binary value of a 0} ... a _i-1 , and each bit a is regarded as The input data of one bit line is thus input via i bit lines. In one embodiment, for example, it divides i data into p groups, that is, input data groups 102_1, 102_2... one by one. One sub-input data group 102_1, 102_2..., for example, contains i/p data, but multiple processing slices 100_1, 100_2... are arranged in sequence. The processing slices 100_1, 100_2... receive the corresponding secondary input data sets 102_1, 102_2... in the order corresponding to the overall input data set. For example, the first processing slice receives data from a ₀ to a _i/p-1 , the next processing slice receives data from a _i/p to a _2*i/p-1 , and so on. The secondary input data sets 102_1, 102_2... are received by the receiving end component 66. The receiving end component 66 includes, for example, a sense amplifier 60 to sense digital input data. The bit line decoding circuit 62 obtains the corresponding logic output, and the voltage switch 64 inputs the data. The receiving end element 66 is set according to actual needs, and the present invention does not limit the circuit configuration of the receiving end element 66.

Each input data group 102_1, 102_2... is convoluted by a corresponding processing piece 100_1, 100_2... The convolution operation of the processing slices 100_1, 100_2... is a part of the overall convolution operation. Each processing piece 100_1, 100_2... corresponds to the received secondary input data group 102_1, 102_2... and is processed in parallel respectively. The secondary input data groups 102_1, 102_2... enter the associated memory cell in the storage unit 90 through the receiving end element 66.

In one embodiment, the number of rows of memory cells storing weights is, for example, j, where j is a large integer. In other words, there are j memory cells corresponding to a bit line. Each memory cell stores a weight value. Herein, the memory cell array can also be referred to as a selection line. In one embodiment, j memory cells are divided into q weight blocks 92, for example. In the embodiment where j is divisible by q, a weight block contains j/q memory cells. For a memory cell, from the output end, it is also equivalent to one bit of a binary number string. These weight blocks are also sorted according to the weight, and are divided into q weight blocks 92 from 0 to j-1.

For the entire convolution operation, it is necessary to get the total value, expressed in Sum, which is shown in formula (1): Sum =Σ a * W (1) where " a " represents the input data group, and W represents the selection in the storage unit A two-dimensional array of layer weights.

For the input sub-input data group, take data containing 8 bits as an example. The binary number string is represented as [a ₀ a ₁ ... a ₇ ], for example, [10011010], which corresponds to a decimal The numerical value. Similarly, the weight block is represented by a bit string. For example, the first weight block contains [W ₀ ...W _j/q-1 ], and the last weight block in sequence is [W _{(q-1 )*j/q} ...W _j-1 ], which respectively represent a decimal value.

In this way, the entire convolution operation is expressed as formula (2): SUM=(W ₀ ...W _j/q-1 *2 ⁰ +...+W _(q-1)*j/q ...W _{j- 1} *2 ^j*(q-1)/q )*2 ⁰ *a ₀ ...a _i/p-1 +(W ₀ ...W _j/q-1 *2 ⁰ +...+W _(q-1)*j/q ...W _j-1 *2 ^j*(q-1)/q )*2 ^i/p *a _i/p ...a _2*i/p-1 + ...+(W ₀ ...W _j/q-1 *2 ⁰ +...+W _(q-1)*j/q ...W _j-1 *2 ^{j*(q-1) /q} )*2 ^i*(p-1)/p *a _(p-1)*i/p ...a _i-1 (2)

For the weight patterns stored in the i*j two-dimensional array as shown in Figure 2, Sum is _{the value predicted by the weight pattern and the overall input data set (a 0} ... a _i-1 ) through the convolution operation. The convolution operation is integrated in the planning of the memory cell array structure. Through wiring, the input data of the bit and the weight form stored in the memory cell of the selected layer are convoluted. The detailed actual operation of the matrix convolution operation is understood in the technical field and will not be described in detail here. According to the embodiment of the present invention, the weight data is divided into multiple processing slices 100_1, 100_2... for parallel operation according to the convolution operation, and each processing slice 100_1, 100_2... is divided into multiple weight blocks 92 , Can also be operated in parallel respectively. An embodiment of the present invention also proposes that for each processing slice, a method of shifting and adding is used to restore the divided multiple weight blocks to the desired result of a single weight block as a whole. In addition, for a plurality of processing slices, the divided processing slices have been summed up by means of shift and addition to obtain the overall required operation value.

Each processing slice 100_1, 100_2... is also provided with a processing circuit 70 to perform convolution operations. In addition, the processing chips 100_1, 100_2... are also provided with a block-wise output circuit 80, which includes multi-level shift and addition operations. For parallel zero-level output data, according to the bit (memory cell) In order, the data corresponding to [W ₀ ...W _j/q-1 ], ... are obtained. Shift and addition operations are also used between processing slices to obtain the final overall convolution operation.

Under the above plan, the storage used for processing a weight block of a processing slice is 2 ^(i/p+j/q) . As a whole, it contains p processing slices, and each processing is q weight blocks, so the number of memory cells required can be reduced to p*q*2 ^(i/p+j/q) .

The following describes in more detail the result of the overall operation based on the divided weight blocks and the divided processing slices.

FIG. 5 is a schematic diagram of a memory cell structure of a storage unit according to an embodiment of the present invention. Referring to Figure 5, for a processing slice storage unit, it corresponds to a bit line BL_1, BL_2... will contain multiple memory cell strings, which are vertically connected to the bit line BL to form a 3D structure. Each memory cell in the memory cell string is a memory cell array belonging to a layer, and stores one of the weight values in the weight form. The cell strings on the bit lines BL_1, BL_2... are activated by the selection line SSL. The memory cells corresponding to multiple selection lines SSL constitute a weight block, as indicated by Block_n. The input data is input by the bit line BL, and flowed into the corresponding memory cell under control, so that the convolution operation is performed. It is then combined and output by the output terminal SL_n. The storage unit will contain q blocks, as indicated by Block_n*q.

FIG. 6 is a schematic diagram of a mechanism for summing multiple weight blocks for one processing slice according to an embodiment of the present invention. Referring to FIG. 6, in the storage unit 300 of a processing slice, it is divided into a plurality of weight blocks 302. Each weight block 302 performs a convolution operation with the secondary input data set, and outputs the calculated value of each weight block 302 in parallel, as indicated by the thick arrow. After that, a sense signal is output respectively through the sense of the sense amplifier SA, which is, for example, a current value. Because the weight is in accordance with the binary arrangement and is output in parallel. To obtain the decimal value, an embodiment of the present invention proposes a block-by-block output circuit configuration. The adder 312 is used to add the output values of adjacent phases, and the output value belonging to the higher bit of the two output values is passed first. The shifter 308, which can shift a predetermined number of bits, shifts it to the corresponding bit sequence. For example, a weight block contains j/q bits (memory cells), and the output value of higher bits needs to be shifted and increased by j/q bits, so the first-order shift and addition operation shift The device 308 has the effect of shifting by j/q bits. After the addition of the first-order adder, it represents a value of 2*j/q bits. In this way, for the subsequent second-level shift and addition operations, the mechanism is the same, but the shift amount of the shifter 314 is 2*j/q bits. By analogy, the input value of the last stage is only two, so only one shifter 316 is needed, but the shift amount is, for example, 2 ^(log ₂ ^q-1) *j/q bits, and one Process the result of the convolution operation of the slice.

In addition, it should be explained that the weight block of the weight form of a layer can also be scattered by multiple different processing slices, which are based on the planning and combination of the weight blocks. That is, the weight blocks stored in a processing slice do not need the same layer of weight data. On the other hand, the weight blocks of the weight data of one layer are distributed to multiple processing slices. Therefore, processing slices can be operated in parallel. That is, each of the involved multiple processing slices only needs to perform calculations on the blocks belonging to one layer to be processed, and then combine calculation data belonging to the same layer.

The following describes the shift and addition operations that integrate multiple processing slices. FIG. 7 is a schematic diagram of the operation mechanism of the summing circuit between multiple processing slices according to an embodiment of the present invention. Referring to FIG. 7, p processing slices 100_1, 100_2, ..., 100_p perform shift and addition operations respectively according to the output value of FIG. 6. Here, each processing slice 100_1, 100_2,..., 100_p refers to the convolution calculation result of the corresponding input data in the same layer of weight form.

Similar to the situation in FIG. 6, the input data group is a binary input string, but for each sub-input data group, for example, there are i/p bits. Therefore, the first-order shift and addition operation also takes each pair of adjacent outputs to be added by the adder 352, in which the higher-bit value is first shifted by i/p bits through the shifter 350. Bit. The shifter 354 for the next-order shift and addition operation is a shift of 2*i/p bits. The shift amount of the last-stage shifter 356 is 2 ^(log ₂ ^p-1) *i/p bits. After the last-order shift and addition operations, the sum value Sum as in formula (1) can be obtained.

常數α 404是縮放值通過乘法器402調整總和值Sum，其後再使用加法器406調整偏移量β 408。 The sum value Sum at this stage is a preliminary value. In practical applications, it needs to be normalized later. For example, the normalization processing circuit 400 is used to normalize the sum value to obtain the normalized sum value. The normalization processing circuit includes, for example, the operation of formula (3):

The constant α 404 is the scaling value adjusted by the multiplier 402 to adjust the sum value Sum, and then the adder 406 is used to adjust the offset β 408.

其中0.5表示取整數的運算。一般而言，如果輸入資料組愈能吻合此層的特徵形態，其數量化的值a’會愈大。 The normalized sum value is then passed through the quantization circuit 500, which uses the divider 502 to divide by the base d 504 for quantization, as shown in formula (4):

Where 0.5 represents the operation of taking integers. In general, if the input data set can match more morphological characteristics of this layer, the number of values a 'will be larger.

After completing the convolution operation of the weight form of one layer, it uses the character line to continue to select the weight form of the next layer to perform the convolution calculation.

FIG. 8 is a schematic diagram of the overall application configuration of an artificial intelligence accelerator according to an embodiment of the present invention. Referring to FIG. 8, the artificial intelligence accelerator 602 of the overall system 600 and the control unit 604 that can communicate with the host in two-way communication. The control unit 604 of the host, for example, obtains input data, such as digital data of an image, from the external memory 700, and inputs it to the artificial intelligence accelerator 602 for identification processing of characteristic shapes, and then responds To the control unit 604 of the host. The application of the overall system 600 can be configured according to actual needs, and is not limited to the configuration described above.

An embodiment of the present invention also provides a processing method for an artificial intelligence accelerator. FIG. 9 is a schematic flowchart of an artificial intelligence accelerator processing method according to an embodiment of the present invention.

Referring to FIG. 9, an embodiment of the present invention further provides a processing method for an artificial intelligence accelerator. The artificial intelligence accelerator receives a two-bit input data group and a selected one of the multi-layer overall weight forms to perform a convolution operation, and the input data group is a plurality of sub-data groups. The processing method includes step S100 using multiple processing slices, each of which includes execution. In step S102, the receiving end component is used to respectively receive one of the secondary data groups. Step S104: Use a weight storage part to store a partial weight form of the overall weight form, wherein the weight storage part includes a plurality of weight blocks, and each of the weight blocks stores a block part of the partial weight form in a bit order, The memory cell array structure of the weight storage part is planned to perform a convolution operation with each of the blocks to obtain a plurality of weight calculation values in sequence with respect to the corresponding sub-data group. In step S106, a block-by-block output circuit including multiple shifters and multiple adders is used, and the sum of the multiple weight calculation values is obtained by multi-level shift and addition operations to obtain a direct result from the sub-data group and the partial weight form. The weight output value expected by the convolution operation. In step S108, a summation output circuit including multiple shifters and multiple adders is used, and the sum of the multiple weight output values is obtained by multi-level shift and addition operations to obtain a direct result of the input data set and the overall weight form. The sum value expected by the convolution operation.

As described above, the embodiment of the present invention proposes to divide the weight data of the storage unit into multiple processing slices to perform the convolution operation, and at the same time, the storage unit of each processing slice is also divided into multiple weight-driven blocks for processing separately. Afterwards, by shifting and adding operations, the final overall total value can be obtained. Since the circuit of the processing chip is smaller, the calculation speed can be increased, and the energy consumed during processing of the processing chip, such as heat generation, can be reduced.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the relevant technical field can make some changes and modifications without departing from the spirit and scope of the present invention. The scope of protection of the present invention shall be determined by the scope of the attached patent application.

100_1, 100_2, 100_(p-1), 100_p: processing slices

350, 354, 356: shifter

352, 358: Adder

400: Processing circuit

402: Multiplier

404: constant

406: Adder

408: offset

500: quantization circuit

502: Divider

504: base

Claims (10)

An artificial intelligence accelerator that receives a two-bit input data group and a selected one of the multi-layered overall weight forms to perform a convolution operation. The input data group is a plurality of sub-data groups, and the artificial intelligence accelerator includes a plurality of processes Slices, each of the processing slices includes a receiving end component, which receives one of the sub-data groups; a weight storage part, which stores part of the weight form of the overall weight form, wherein the weight storage part includes a plurality of weight blocks, each of the weights The block stores a block part of the partial weight form according to the bit sequence, wherein the memory cell array structure of the weight storage part is planned to correspond to the corresponding sub-data group such that the sub-data group and each of the sub-data groups are respectively The block part performs the convolution operation to obtain multiple weight calculation values in sequence; and the block-by-block output circuit includes multiple shifters and multiple adders, and the multiple weights are obtained through multi-level shift and addition operations The calculated value is summed to obtain the weight output value expected by the direct convolution operation of the sub-data group and the weight form; and the summation output circuit includes multiple shifters and multiple adders, through multi-stage shift and phase The addition operation sums the output values of the multiple weights to obtain the sum value expected by the direct convolution operation between the input data group and the overall weight form. The artificial intelligence accelerator according to claim 1, wherein the input data group contains i bits, divided into p sub-data groups, i and p are integers, and each sub-data group contains i/p bits . The artificial intelligence accelerator according to claim 1, wherein the input data group includes i bits, the number of the plurality of processing slices is p, the input data group is p of the sub-data group, and the i and p It is an integer greater than or equal to 2, the i is greater than the p, and each sub-data group contains i/p bits. The artificial intelligence accelerator according to claim 3, wherein the number of the plurality of weight blocks included in the weight storage part is q, and q is an integer greater than or equal to 2, and the weight storage part contains j bits, and the j , Q is an integer greater than or equal to 2, the j is greater than the q, and each weight block contains j/q memory cells. The artificial intelligence accelerator according to claim 4, wherein the total amount of memory cells of the plurality of weight storage parts of the plurality of processing slices is p*q*2 ^(i/p+j/q) . A processing method for an artificial intelligence accelerator. The artificial intelligence accelerator receives a two-bit input data group and a selected one of the multi-layer overall weight forms to perform a convolution operation, and the input data group is a plurality of sub-data groups , The processing method includes: using a plurality of processing slices, each of the processing slices includes executing: using a receiving end component to respectively receive one of the sub-data groups; using a weight storage unit to store part of the weight form of the overall weight form, wherein the weight is stored The part includes a plurality of weight blocks, and each of the weight blocks stores a block part of the partial weight form according to the bit sequence, wherein the memory cell array structure of the weight storage part is relative to the corresponding sub-data group, It is planned to perform a convolution operation on the data group with each part of the block to obtain a plurality of weight calculation values in sequence; and Using a block-by-block output circuit including multiple shifters and multiple adders, the sum of the multiple weight calculation values is obtained through multi-level shift and addition operations. The convolution calculation is directly performed on the data group and the weight form. The expected weight output value; and the sum output circuit including multiple shifters and multiple adders is used, and the sum of the multiple weight output values is obtained by multi-stage shift and addition operations to obtain the input data set The sum value expected by the convolution operation directly with the overall weight pattern. The processing method for an artificial intelligence accelerator according to claim 6, wherein the input data group contains i bits, divided into p sub-data groups, i and p are integers, and each sub-data group contains i /p bits. The processing method for an artificial intelligence accelerator according to claim 6, wherein the input data group includes i bits, the number of the plurality of processing slices is p, and the input data group is p of the sub-data group , The i and p are integers greater than or equal to 2, the i is greater than the p, and each sub-data group contains i/p bits. The processing method for an artificial intelligence accelerator according to claim 8, wherein the number of the plurality of weight blocks included in the weight storage unit is q, and q is an integer greater than or equal to 2, and the weight storage unit includes j Bit, the j and q are integers greater than or equal to 2, the j is greater than the q, and each weight block contains j/q memory cells, wherein the multiple weight storage parts of the multiple processing slices The total number of memory cells is p*q*2 ^(i/p+j/q) . The processing method for an artificial intelligence accelerator according to claim 9, wherein for the operation of the block-by-block output circuit, the shift and addition operation of each stage includes using at least one shifter and at least one The adder, For the multiple input values of each level, two adjacent ones are used as a processing unit. The input value belonging to the higher bit passes through the shifter and is added to the input value of the lower bit by the adder. Then output to the next stage, where a single value of the last stage output is regarded as the weighted output value corresponding to the processed slice.

Images