TWI852888B - Accumulator and memory device for in-memory computing and operation method thereof - Google Patents

Abstract

An accumulator provided by the present disclosure comprises: a counter for counting partial results generated by computing, to generate counting results; a weighting and extension unit for extending bits of each values of the counting results to generate bit extension results, and for moving corresponding bits of the bit extension results to generate weighting results based on a weight & extension table; a bit-flip unit for flipping the corresponding bits of the weighting results to generate flipping results based on a bit-flip table; a result summing unit for summing the flipping results to generate summing results; and a bias add unit for adding a bias value to the summing results to obtain final results.

Description

Accumulator for in-memory calculation, memory device and operation method thereof

The present invention relates to an accumulator for in-memory computing (IMC), a memory device and an operation method thereof.

Artificial intelligence (AI) has become a highly effective solution in many fields. The key operation of AI is to perform multiply-and-accumulate (MAC) operations on a large amount of input data (such as input feature maps) and weight values, which also includes vector-vector multiplication (VVM) operations. However, with the current AI architecture, such as the Von Neumann architecture of CPU/GPU with external memory, it is easy to encounter input/output bottlenecks (I/O bottlenecks) and inefficient MAC operation processes. However, as the amount of data required for calculation increases, a large amount of computing resources are consumed in data input/output (I/O), making the I/O bottleneck more serious and the efficiency will be lower.

The first aspect of the present disclosure provides an accumulator for in-memory computing, comprising: a counter for counting corresponding bits having a first bit value among a plurality of bits of a plurality of partial results generated by the in-memory computing to generate a counting result; a weighting and extension The invention relates to a bit-extension unit, coupled to the counter, for extending each bit in the counting result to a preset bit number using a plurality of second bit values to generate a bit extension result, and for moving each bit having a first bit value in the bit extension result to a corresponding Wth bit in each preset bit number according to a weight and extension data table to generate a weight result; a bit-flip unit, coupled to the weight and extension unit, for flipping each bit in the corresponding bit value in the weight result according to a bit-flip data table to generate a flip result, and the bit-flip data table has P first bit values; a result summing unit, coupled to the bit-flip unit, for summing each value in the flip result to generate a summed result; and a bias summing unit. The add) unit is coupled to the result summing unit and is used for adding the summing result to the bias value to generate the accumulated result, and the bias value is P. P and W are positive integers greater than or equal to 0.

The second aspect of the present disclosure provides a memory device for in-memory operations, comprising: multiple memory planes, each of the multiple memory planes comprising: a cell array for storing multiple weights; a page buffer coupled to the cell array, the page buffer for reading multiple weights located in the cell array and generating multiple partial results based on multiple input operations; a multiplexer coupled to the page buffer for selecting corresponding partial results from the multiple partial results located in the page buffer; and an accumulator provided in the first aspect of the present disclosure, coupled to the multiplexer.

A third aspect of the present disclosure provides a memory device for in-memory computing, comprising: a plurality of memory planes, each of the plurality of memory planes comprising: a cell array for storing a plurality of weights; a page buffer coupled to the cell array, the page buffer for reading a plurality of weights in the cell array and generating a plurality of partial results according to a plurality of input operations; a multiplexer coupled to the page buffer for selecting a corresponding partial result of a plurality of partial results in a page buffer; and a counter coupled to the multiplexer, the counter being used to count a plurality of corresponding bits having a first bit value among a plurality of bits of the plurality of partial results to generate a counting result; and an accumulator coupled to a plurality of memory planes, the accumulator comprising: a register for temporarily storing a plurality of counting results in each memory plane; a weight and extension unit coupled The weight and extension unit is connected to the register, and is used to extend each bit in each counting result to a preset bit number using a plurality of second bit values to generate a bit extension result, and is used to move each bit with a first bit value in the bit extension result to a corresponding Wth bit in the preset bit number according to the weight and extension data table to generate a weight result; the bit flip unit is coupled to the weight and extension unit, and is used to move each bit with a first bit value in the bit extension result to a corresponding Wth bit in the preset bit number according to the weight and extension data table The bit flip data table flips each bit in the corresponding bit value in the weight result to generate a flip result, and the bit flip data table has P first bit values; the result summing unit is coupled to the bit flip unit and is used to sum each value in the flip result to generate a summed result; and the offset summing unit is coupled to the result summing unit and is used to add the offset value to the summed result to generate an accumulated result, and the offset value is P. P and W are positive integers greater than or equal to 0.

According to various embodiments of the first aspect of the present disclosure, the bandwidth of each of one or more buses from the counter to the bitwise summing unit corresponds to the weight of the data and the N-bit, M-dimension and carry bit of the input, where N and M are integers greater than or equal to 1.

According to multiple embodiments of the third aspect of the present disclosure, the bandwidth of each of one or more buses between the counter and the offset summing unit corresponds to the number O of each memory plane, the N bits of input, the M dimension and the carry bit, and M, N and O are integers greater than or equal to 1.

According to various embodiments of the first aspect or the third aspect of the present disclosure, the result summing unit has one or more registers, and the number of the one or more registers is inversely proportional to the number of cycles required to sum the values in the flipped result to generate the summed result.

According to various embodiments of the first aspect or the third aspect of the present disclosure, the accumulator further includes at least one shift-and-sum unit corresponding to data encoded with error-bit-tolerance.

The fourth aspect of the present disclosure provides an operation method for an accumulator, including: counting corresponding multiple bits with first bit values in multiple bits of multiple partial results generated by calculation in a memory to generate a counting result; extending each bit in the counting result by a preset bit number using multiple second bit values to generate a bit extension result; moving each bit with a first bit value in the bit extension result to the corresponding Wth bit in the preset bit number according to a weight and extension data table to generate a weight result; flipping each bit in the corresponding bit value in the weight result according to a bit flip data table to generate a flip result, and the bit flip data table has P first bit values; summing up each value in the flip result to generate a summed result; and adding a bias value to the summed result to generate an accumulated result, and the bias value is P. P and W are positive integers greater than or equal to 0.

According to multiple embodiments of the fourth aspect of the present disclosure, data corresponding to the error bit tolerant data encoding and the partial results generated by the operation on the data are decoded and summed to generate a summed result.

Various variations of the aforementioned aspects optionally include hardware, methods or programs, or computer software on a computer-accessible medium.

In order to better understand the above and other aspects of the present invention, the following embodiments are specifically described in detail with reference to the accompanying drawings:

The technical terms in this specification refer to the customary terms in this technical field. If this specification explains or defines some terms, the interpretation of these terms shall be subject to the explanation or definition in this specification. Each embodiment of the present disclosure has one or more technical features. Under the premise of possible implementation, a person with ordinary knowledge in this technical field can selectively implement part or all of the technical features in any embodiment, or selectively combine part or all of the technical features in these embodiments.

FIG. 1 shows a schematic diagram of an example memory device 100. The memory device 100 may be used for in-memory computing (IMC). The memory device 100 has a cell array 110, a page buffer 130, and an accumulator 140. The memory device 100 may use the page buffer 130 and the accumulator 140 to perform a vector-vector multiplication (VVM) operation of the IMC. When performing the VVM operation, data in a selected page 121 of a page 120 in the cell array 110 is read into the page buffer 130. The data form of the data may be a vector and include an input 101 and a weight 102. In this example memory device 100, when performing the accumulation operation of the VVM operation, the input 101 needs to be multiplied by the weight 102 step by step. When the data is a vector with multiple sets of multiple dimensions, multiple loops are required to complete the data processing (for example, the data format is 64 sets of 4-bit 512-dimension (64sets of 4-bit 512-dimension), with a capacity of 16KB), which means that more time and power consumption are required.

FIG. 2 is a schematic diagram showing the architecture of an accumulator 240 of an example memory device 200. The memory device 200 is similar to the memory device 100 of FIG. 1 and can also be used for in-memory computing (IMC) and perform VVM operations. The memory device 200 is similar to the memory device 100 of FIG. 1 and has a cell array 210, a page buffer 230, and an accumulator 240. The difference from the memory device 100 of FIG. 1 is that the accumulator 240 of the memory device 200 is provided with a plurality of shift and add operations 241 for processing the product of the input 201 and the weight 202 respectively. Since there are multiple shift-and-sum 241, the accumulator 240 can process these input data simultaneously when executing the VVM operation, thereby reducing the time and power consumption required for the operation. However, in order to simultaneously process a large amount of data (for example, the data is in the form of 64 groups of 4-bit 512 dimensions and the capacity is 16KB), the additional shift-and-sum 241 will increase a lot of costs and increase the size of the accumulator 240.

In order to solve the cost and size problem of using a large number of accumulators for partial multiplication of a large number of data in the above-mentioned in-memory operation, the memory device and the accumulator of the memory device proposed in the present disclosure use a systolic input and a counter-co-work operation architecture, and use a bit-flip table to more widely support integer operations of signed data. The technology of the memory device and the accumulator of the memory device of multiple embodiments of the present disclosure will be described in detail below.

Techniques disclosed herein provide for performing VVM, MAC, HDC, or SUM-only operations in a semiconductor device using counter-co-work support. For the purposes described, the semiconductor device is illustrated as a memory device. It should be noted that these techniques may be applied to any circuit, device, or system that performs VVM or MAC operations. For example, in addition to memory devices, these techniques may also be applied to any other storage device or logic device such as a neural network accelerator.

These techniques can be applied to several types of nonvolatile memory devices, such as NOR flash memory, NAND flash memory, erasable programmable read-only memory (EPROM), ferroelectric random access memory (FeRAM), magnetic random access memory (PCRAM), and others. These techniques can be applied to several memory formats, such as single-level cell (SLC) devices, multi-level cell (MLC) devices such as 2-level cell devices, triple-level cell (TLC) devices, quad-level cell (QLC) devices, or penta-level cell (PLC) devices. These techniques may additionally or alternatively be applied to various types of devices and systems, such as secure digital (SD) cards, embedded multimedia cards (eMMC), or solid-state drives (SSDs), embedded systems, or network computer devices such as network routers or network processors, cache controllers and address translation registers, lookup tables, database engines, data compression hardware, artificial neural networks, intrusion prevention systems, custom computers, and others.

FIG. 3 is a schematic diagram showing the architecture of an accumulator 340 of a memory device 300 according to an embodiment of the present disclosure. The memory device 300 is similar to the memory device 100 of FIG. 1 and the memory device 200 of FIG. 2 and can also be used for in-memory computing (IMC) and perform VVM or MAC operations. The memory device 300 has a cell array 310, a page buffer 330, a multiplexer 315, and an accumulator 340. The difference between the memory device 100 of FIG. 1 and the memory device 200 of FIG. 2 is that the accumulator 340 of the memory device 300 has a counter 341, a register 342, a weight and extension unit 343, a bit flip unit 344, a result summing unit 345, a biased summing unit 346, and an output register 347. When performing a VVM or MAC operation, the accumulator 340 of the memory device 300 receives data as a partial result (not shown in the figure, please refer to the content described in FIG. 4), and uses the counter 341 to enable systolic input and counter-co-work operations. The weight and extension unit 343, the bit flip unit 344, the result summing unit 345 and the offset summing unit 346 of the accumulator 340 are used to support integer operations on signed data through a bit-flip table (not shown in the figure, please refer to the content described in FIG. 6). The specific functions of each element in the accumulator of the semiconductor device according to various embodiments of the present disclosure will be described in detail in the following FIGS. 4 to 7.

FIG. 4 is a schematic diagram showing the architecture of a memory device 400 having an accumulator 440 according to an embodiment of the present disclosure. In many embodiments, before performing a VVM operation, data (such as vector data) may be pre-processed. Before pre-processing, the input data may be in the form of L groups of N-bit M-dimensional vectors. As an example, the pre-processed data may be encoded with L groups of N-bit M-dimensional weights to obtain encoded weights 402, where L, M, and N are positive integers greater than or equal to 1. In this example, 64 groups of 4-bit 512-dimensional (L=64, N=4, M=512) vectors (with a total capacity of 16KB) are used as an example for ease of explanation. The weight 402 may be pre-processed by data, and the original 64 groups of 4-bit 512-dimensional data forms are re-encoded according to the weights into N different corresponding weight values in each L group (such as weights W ₃ , W ₂ , W ₁ and W ₀ in the figure), and each corresponding weight value has M bits, that is, 64 groups of 4 weights and each weight has a 512-bit data form, as shown in the figure. In order to correspond to the data form of each weight having 512 bits, the input 401 can be operated with the weight through N loops (corresponding to N weights) of M-bit form (corresponding to M bits of weight), and each loop segment should produce multiple partial results, as shown in the figure. According to the data preprocessing disclosed in the present invention, the data can be coded and calculated (such as multiplication) first, so that the data can be used in the aforementioned pulse input (systolic The weight stored in the cell array (such as the cell array 310 in FIG. 3 ) is read through the page buffer and operated with the input to generate multiple partial results, and then the corresponding partial results are selected by the multiplexer (such as the page buffer 330 and the multiplexer 315 in FIG. 3 ), which can be further used in the aforementioned counter-co-work operation architecture of the present disclosure (counter 441, weight and extension unit 443, bit flip unit 444, result summing unit 445, offset summing unit 446 and output register 447 that can be used in conjunction with the accumulator 440). The specific functions of other components will be described in detail in the following figures.

FIG. 5 is a schematic diagram of the operation of weights and extensions 500 according to an embodiment of the present disclosure. As mentioned above, weights used for multiple operations (such as VVM or MAC operations) can be stored in a cell array (such as the cell array 310 in FIG. 3), and the read weights are operated according to the input through a page buffer (such as the page buffer 330 in FIG. 3) to generate multiple partial results, and a multiplexer (such as the multiplexer 315 in FIG. 3) selects the corresponding partial results and outputs them in batches to an accumulator (such as the accumulator 440 in FIG. 4) for processing. For ease of explanation, in this example, the input and weights used for calculation are input 501 in a 4-bit data format: and weight 502: As an example, the value of input 501 is -3 (1101) and the value of weight 502 is -1 (1111), as shown in the figure.

When the input 501 and the weight 502 are multiplied, the weight 502 can be stored in a cell array (such as the cell array 310 in FIG. 3 ), and the weight 502 is read through the page buffer and operated according to the input 501. The multiplexer (such as the page buffer 330 and the multiplexer 315 in FIG. 3 ) can select the corresponding partial result to be output to the counter in the accumulator (such as the accumulator 440 and the counter 441 in FIG. 4 ), that is, the partial result 510 (such as the accumulator 440 and the counter 441 in FIG. 4 ) in FIG. 5 ). The plurality of partial results 510 are counted by a counter (e.g., the counter 441 in FIG. 4 ) to obtain a count result 520, as shown in the figure. The counting is to count the bits having the first bit value. For the convenience of explanation, the first bit value is 1 and the second bit value is 0 as an example, but the present disclosure is not limited to this.

Then, a weighting and extension unit (e.g., weighting and extension unit 443 in FIG. 4 ) extends each bit in the count result 520 to a preset number of bits using a plurality of second bit values (0) to generate a bit extension result 530. The preset number of bits is associated with the bit of the input data and weight, the data dimension, and the carry bit. In some examples, the default number of bits can be calculated according to the following equation (1): ^{2Default number of bits} = (2 ^{( input bit (Ni) – 1)} * 2 ^{( weight bit (Nw) – 1)} ) * (( ^2m ＞ data dimension (M)) * 2 ¹ ( carry bit)) Equation (1) wherein the input bit (Ni), weight bit (Nw) and data dimension (M) are all integers greater than or equal to 1. It should be noted that ( ^2m ＞ data dimension (M)) means that ^2m needs to be greater than the data dimension, and m is also an integer greater than or equal to 1. For example, when the data dimension (M) = 1, ^2m=1 =2 is greater than the data dimension (M)(1), so m is 1; and when the data dimension (M) = 2, ^2m=1 =2, which is not greater than the data dimension (M) (which is 2), so it is necessary to express it as 2 ^m=2 , so that 2 ² is greater than the data dimension (M) (which is 2). The calculation method for other m values is similar. In this example, since the bits of input 501 and weight 502 are both 4 ( and Corresponding to the input bit Ni and the weight bit Nw), and the data dimension is 1 (M=1), so each bit in the count result 520 is extended to 8 (2 ⁸ ) bits. For example, the part framed in the extended result 530 in the figure, The original 1 (the first bit value) is extended to 8 bits using 7 zeros (the second bit value). Since the numerical value disclosed in the present invention can be carried after calculation, a carry bit 531 is included, as shown in the figure.

Next, referring to the weight and extension data table 550 shown in FIG. 5, which includes and The corresponding weight displacement data, for example, W00~W03 in the table correspond to 0~3 respectively, representing the position (Wth in the default bit number) to which the bit with the first bit value (1 in this example) in the extended result 530 needs to be moved. For example, in the weight and extended data table 550, The value of is 2 ^6(W33) , that is, W33 is 6, so, The first digit value (in this example, 1) is originally at position 0 (1st from the right) and will be extended based on the weight and data in table 550 ( The value of is 2 ^6(W33) ), and it moves to the W33th position (W33=6, which is the 7th bit from the right). Similarly, the corresponding bits (for example, except ), similarly according to the weight and extension data table 550, the corresponding bit of the first bit value (1) is moved to the corresponding Wth (W00~W33) bit in the preset bit number to generate the weight result 540, as shown in FIG. 5 .

FIG. 6 is a schematic diagram of the operation of a bit flip 600 according to an embodiment of the present disclosure. In order to simplify the components required for the product operation in the accumulator of the present disclosure, in many embodiments, the bit flip unit (such as the bit flip unit 444 of FIG. 4) has a bit flip data table. The bit flip data table corresponds to the following equation: Equation (2)

Through equation (2), the bit positions that need to be bit flipped in the bit flip data table can be obtained (as indicated in the bit flip data table 605 of FIG. 6). In this example, it can be known from the bit flip data table 605 that the bits that need to be bit flipped are A ₃ (B ₂ B ₁ B ₀ ) and B ₃ (A ₂ A ₁ A ₀ ), where the first bit value (in this example, 1) is used. In the bit flip data table, the number of the first bit values mentioned above is P, that is, there are P first bit values. In this example, the number (P) of the first bit values (in this example, 1) in the bit flip data table 605 is 6, which can be represented by (0110). The number of the first bit values in the bit flip data table can be used as the offset value P in other subsequent operations (refer to the description of FIG. 7).

As shown in FIG. 6 , the corresponding bits A ₃ (B ₂ B ₁ B ₀ ) and B ₃ (A ₂ A ₁ A ₀ ) of the weight result 640 (which is the same as the weight result 540 in FIG. 5 in this embodiment) are flipped according to the corresponding columns of the bit flip data table 605, flipping the first bit value into the second bit value (e.g., flipping 1 into 0) and flipping the second bit value into the first bit value (e.g., flipping 0 into 1) to obtain the flip result 660.

FIG. 7 is a schematic diagram showing the operation of result summing 700 according to an embodiment of the present disclosure. The operation of result summing 700 can be implemented by a result summing unit (e.g., result summing unit 445 in FIG. 4 ), which first performs group summing 770 on each value in flip result 760 (which is the same as flip result 660 in FIG. 6 in this embodiment) and then sums them to generate a summed result 780 (e.g., 1111101, as shown in the figure).

Since the bit flipping operation as described above is performed in many embodiments of the present disclosure, the accumulator of the present disclosure has a bias add unit (such as the bias add unit 446 in Figure 4) for adding a bias value to the summed result to generate an accumulated result.

As shown in FIG. 7 , the summed result 780 (e.g., 1111101) is added to the aforementioned offset value P (the sample of the first bit value in the bit flip data table 605), and P is 00000110 (representing 6). The cumulative result 790 after adding the two is 00000011 as shown in the figure, that is, the final cumulative result is 3. As mentioned above, in this example, the input value is -3 (1101) (e.g., input 501) and the weight value is -1 (1111) (e.g., weight 502), so the cumulative result is 3, which is equal to the product of the input value -3 and the weight value -1.

Therefore, according to the above-mentioned multiple embodiments, the memory device and the accumulator architecture of the memory device proposed in the present disclosure can achieve the accumulation operation of partial results of VVM or MAC with systolic input and counter-co-work operation architecture, without the need to configure a large number of shift-sum elements, effectively reducing the time, power consumption and cost of IMC operation.

In addition to the aforementioned memory device and accumulator architecture of the memory device according to the present disclosure, the present disclosure can further adjust the bus bandwidth between components according to the processed VVM data format to increase the efficiency and flexibility of the operation. The following will be explained in detail with reference to FIG. 8 and FIG. 9.

FIG. 8 is a schematic diagram showing an exemplary bus bandwidth design of an accumulator 840 of a memory device 800 according to an embodiment of the present disclosure. In this example, as shown in the figure, one or more bus bandwidths (as indicated by the dashed box in the figure) between the weight and extension unit 843 in the accumulator 840 and the output register 847 can be designed according to the input and weight bits (Ni and Nw), the data dimension and the carry bits, that is, according to the following equation: 2 ^{Bus bandwidth} = (2 ^{( input bit (Ni) – 1)} * 2 ^{( weight bit (Nw) – 1)} ) * data dimension (M) * 2 ¹ (carry bit) Equation (3)

For example, when the data format is 4-bit 512-dimensional (equal to 2 ⁹ ), the bus bandwidth is (2 ^(4-1) * 2 ^(4-1) ) * 2 ⁹ * 2 ¹ = 2 ¹⁶ according to the above equation (3), and is designed to be 16, as shown in FIG. 8 .

FIG. 9 is a schematic diagram showing another example bus bandwidth of an accumulator 940 of a memory device 900 according to an embodiment of the present disclosure. Similar to the example of FIG. 8, in this example, when the data format is 8-bit 512-dimensional (equal to 2 ⁹ ), the bus bandwidth is (2 ^(8-1) * 2 ^(8-1) ) * 2 ⁹ * 2 ¹ = 2 ²⁴ according to the above equation (3), and is designed to be 24, as shown in the figure. Therefore, the various embodiments of the present disclosure can adjust the bus bandwidth design between components according to the VVM data format being processed to increase the efficiency and flexibility of the operation.

FIG. 10 is a schematic diagram showing an example register 1042 of a result summing unit 1045 of an accumulator 1040 of a memory device 1000 according to an embodiment of the present disclosure. In various embodiments, the present disclosure further designs the number of example registers 1042 of the result summing unit 1045 according to requirements. For example, if the data format is 64 sets of 4-bit 512-dimensional vectors and the page buffer capacity is 16KB, if the number of registers is designed to be 64 (strategy 1), 64 sets of partial values can be stored, and 16KB can be processed at a time, so fewer loops are needed; if the number of registers is designed to be 16 (strategy 2), 16 sets of partial values can be stored, and 4KB can be processed at a time, so more loops are needed; if the number of registers is designed to be 1 (strategy 3), 1 set of partial values can be stored, and 256B can be processed at a time, so more loops are needed. However, more registers mean more cost but fewer cycles (computation time), while fewer registers mean lower cost but more cycles (computation time). Based on repeated experiments, the computing speed obtained under the above conditions is about 1306us for strategy 1, about 1306us for strategy 2, and about 1351us for strategy 3.

The parameters and calculation methods used in Strategy 1, Strategy 2 and Strategy 3 are as follows: Strategy 1 : Assume the latency parameters = Req_VVM = 512 t _Read = 70us, one-time t _{Input_import} = 30.72us, Global t _{Logic-AND then to CDL} = 5us, t _{Internal-transfer} = 5us, t _{Accumulation-base} = 30.72us t _Accumulation = (t _{Accumulation-base} /(Exe_VMM_PB))*N (N = 64) Exe_VMM_PB = Page buffer size/Required memory cells Request_Read = Req_VVM/ Exe_VMM_PB Computing time = Input_bit*(t _{Input_import} ) + (Request_Read * (((Exe_VMM_PB/N) * (Input_bit*(t _{Logic-function then to CDL} + t _Accumulation ))) + t _{Internal-transfer} )) = (4 * 30.72) + (8*(((64/64) * (4 * (5 + 30.72))) + 5)) ~= 1306us Strategy 2 : Assume the latency parameters = Req_VVM = 512 t _Read = 70us, one-time t _{Input_import} = 30.72us, Local t _{Logic-AND then to CDL} = 1. 25us, t _{Internal-transfer} = 5us, t _{Accumulation-base} = 30.72us t _Accumulation = (t _{Accumulation-base} /(Exe_VMM_PB))*N (N = 16) Exe_VMM_PB = Page buffer size/Required memory cells Request_Read = Req_VVM/ Exe_VMM_PB Computing time = Input_bit*(t _{Input_import} ) + (Request_Read * (((Exe_VMM_PB/N) * (Input_bit*(t _{Logic-function then to CDL} + t _Accumulation ))) + t _{Internal-transfer} )) = (4 * 30.72) + (8*(((64/16) * (4 * (1.25 + 7.68))) + 5)) ~= 1306us Strategy 3 : Assume the latency parameters = Req_VVM = 51 2 t _Read = 70us, one-time t _{Input_import} = 30.72us, Local t _{Logic-AND then to CDL} = 0.1us, t _{Internal-transfer} = 5us, t _{Accumulation-base} = 30.72us t _Accumulation = (t _{Accumulation-base} /(Exe_VMM_PB))*N (N = 1) Exe_VMM_PB = Page buffer size/Required memory cells Request_Read = Req_VVM/ Exe_VMM_PB Computing time = Input_bit*(t _{Input_import} ) + (Request_Read * (((Exe_VMM_PB/N) * (Input_bit*(t _{Logic-function then to CDL} + t _Accumulation ))) + t _{Internal-transfer} )) = (4 * 30.72) + (8*(((64/1) * (4 * (0.1 + 0.48))) + 5)) ~= 1351us

In summary, it can be understood that the number of registers provided by the present disclosure is inversely proportional to the number of cycles required (the number of cycles required to sum the values in the flip result to generate the summed result).

In addition to the number of registers in the result summing unit in the various embodiments disclosed herein being designed to be adjustable according to demand, the number of output registers of the accumulator in the various embodiments disclosed herein (e.g., output register 1047 in FIG. 10 ) can also be adjusted according to demand. For example, fewer output registers have a lower cost, but require the controller of the memory device (e.g., the state machine controller, not shown in the figure) to fetch the VVM output result at any time before the operation is completed; more output registers have a higher cost, but the memory controller can fetch the VVM output result together after the operation is completed.

FIG. 11 is a schematic diagram showing the architecture of an accumulator 1140 of a memory device 1100 according to another embodiment of the present disclosure. The aforementioned data preprocessing (not shown) can perform operations after error-bit-tolerance data encoding to obtain corresponding partial results. Error-bit-tolerance data encoding uses input data duplication and weight data flattening technology. As shown in the figure, the accumulator 1140 may further include a result summing unit and a shift summing unit (such as shift summing unit 1160) to correspondingly process the partial results calculated by the error-bit-tolerance data encoding. Comparing this embodiment with the embodiment illustrated in FIG. 4 , it can be seen that the accumulator architecture described in the present disclosure can be substantially the same and can be adjusted (such as adding a shift-sum unit) to correspond to different data pre-processing (coding) methods. The corresponding operation methods and functions of other components in the accumulator 1140 in the figure are similar to those described above and will not be repeated here (such as the descriptions of FIG. 1 to FIG. 10 ).

FIG. 12 is a schematic diagram showing the architecture of an accumulator 1240 of a memory device 1200 according to yet another embodiment of the present disclosure. In addition to the aforementioned counter-co-working operation architecture, the accumulator of the present disclosure can also be applied to different operation architectures, such as a ping-pong buffer-co-working operation architecture. As shown in the figure, compared to the embodiment illustrated in FIG. 4 , instead of the counter 441, the accumulator 1240 further includes a plurality of page buffers 1231 and 1232, and a plurality of weight and extension units 1242 and 1243) to correspond to the architecture of Page ADC (Page Analog-to-Digital Converter) design. By comparing this embodiment with the embodiment illustrated in FIG. 4 , it can be seen that the accumulator architecture described in the present disclosure can be substantially the same and can be adjusted (such as adding multiple page buffers and multiple weight and extension units to replace the counter) to implement different accumulation operation architectures. The corresponding operation methods and functions of other components in the accumulator 1240 in the figure are similar to the above description and will not be repeated here (such as the description of FIG. 1 to FIG. 10 ).

FIG. 13 is a schematic diagram showing the architecture of a memory device 1300 according to another embodiment of the present disclosure. As shown in the figure, the memory device 1300 includes a memory plane 1311, a memory plane 1312, a memory plane 1313, and a memory plane 1314. Each memory plane (e.g., the memory plane 1311) includes a cell array 1310, a page buffer 1330, a multiplexer 1315, and an accumulator 1340. In this embodiment, since each memory plane has the architecture of the accumulator according to multiple embodiments of the present disclosure, VVM operations can be directly performed on each memory plane. The corresponding operation methods and functions of other components in the accumulator 1340 in the figure are similar to the above description (such as the description of Figures 1 to 12), and will not be repeated here.

FIG. 14 is a schematic diagram showing the architecture of a memory device according to yet another embodiment of the present disclosure. As shown in the figure, the memory device 1400 includes a memory plane 1411, a memory plane 1412, a memory plane 1413, and a memory plane 1414, each of which includes a cell array 1410, a page buffer 1430, a multiplexer 1415, and a counter 1441. The memory plane 1411, the memory plane 1412, the memory plane 1413, and the memory plane 1414 are coupled to an accumulator 1440, and each of the memory planes may correspond to a different weight. In this configuration, when executing VVM, each memory plane with different weights may first generate a counting result (such as the counting result 520 in FIG. 5 ) by the counter of each memory plane (such as the counter 1441), and the counting result of each plane may first be stored in the register 1442 in the accumulator 1440, and then accumulated by other components to obtain the overall accumulation result of these memory planes (such as the accumulation result 790 in FIG. 7 ). The corresponding operation methods and functions of other components in the accumulator 1440 in the figure are similar to the above description (such as the description of FIG. 1 to FIG. 10 ), and will not be repeated here.

FIG. 15 is a flow chart of an operation method 1500 of an accumulator according to an embodiment of the present disclosure. In step S1501, a counter (e.g., counter 441 in FIG. 4) may be used to count the corresponding multiple bits of the multiple bits of the partial result generated by the operation that have the first bit value (e.g., 1) to generate a counting result (e.g., counting result 520 in FIG. 5). Then, in step S1502, each bit in the counting result is extended to a preset number of bits using multiple second bit values (e.g., 0) by a weight and extension unit (e.g., weight and extension unit 443 in FIG. 4) to generate a bit extension result (e.g., bit extension result 530 in FIG. 5). In step S1503, according to the weight and extension data table (e.g., weight and extension data table 550 in FIG. 5 ), each bit having the first bit value (e.g., 1) in the bit extension result is moved to the corresponding Wth bit in the default bit number to generate a weight result (e.g., weight result 540 in FIG. 5 ).

In step 1504, the corresponding bits in the weight result may be flipped by a bit flip unit (e.g., the bit flip unit 444 of FIG. 4) according to a bit flip data table (e.g., the bit flip data table 605 of FIG. 6) to generate a flip result (e.g., the flip result 660 of FIG. 6), and the bit flip data table has P first bit values (e.g., 1). Then, in step 1505, the values in the flip result may be summed up by a result summing unit (e.g., the result summing unit 445 of FIG. 4) to generate a summed result (e.g., the summed result 780 of FIG. 7). Finally, in step S1506, the offset value P may be added to the summed result by a offset and summing unit (eg, the offset and summing unit 446 in FIG. 4 ) to generate a cumulative result (eg, the summed result 790 in FIG. 7 ).

The above-mentioned multiple embodiments of the present disclosure can be applied to the IMC architecture design that needs to run VVM operations, and can be applied to AI technologies, such as but not limited to, fully-connection layer, convolution layer, multiple layer perceptron, support vector machine and other AI technologies.

The above-mentioned embodiments disclosed in the present invention can be applied not only to application-driven in-memory computing, but also to in-memory search applications and hyperdimensional computing (HDC).

Although many details may be described herein, these details should not be construed as limitations on the scope of the invention that is or may be claimed, but rather as descriptions of features directed to specific embodiments. Certain features described herein in separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although several features may be described above as acting in certain combinations and even initially claimed as such, one or more features from the claimed combinations may in some cases be excluded from the combinations, and the claimed combinations may be directed to subcombinations or variations of subcombinations. Similarly, while several operations are depicted in the drawings in a particular order, it should not be understood that such operations must be performed in the particular order depicted or in sequential order, or that all illustrated operations must be performed to achieve desirable results.

In summary, although the present invention has been disclosed as above by way of embodiments, it is not intended to limit the present invention. Those with ordinary knowledge in the technical field to which the present invention belongs can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope defined in the attached patent application.

100,200,300,800,900,1000,1100,1200,1300,1400:Memory device 101,201,401,501:Input 102,202,402,502:Weight 110,210,310,810,910,1010,1110,1210,1310,1410:Cell array 120,220:Page 121:Selected page 130,230,330,830,930,1030,1130,1230,1231,1232,1330,1430:Page buffer 140,240,340,400,840,940,1040,1140,1240,1340,1440:Accumulator 241:Shift-to-sum 341,441,841,941,1041,1141,1341,1441:Counter 342,842,942,1042,1142,1161,1248,1342,1442,1449:Register 343,443,843,943,1043,1143,1242,1243,1245,1343,1443:Weight and extension unit 344,444,844,944,1044,1144,1244,1344,1444: Bit flip unit 345,445,845,945,1045,1145,1245,1345,1445: Result summing unit 346,446,846,946,1046,1146,1246,1346,1446: Bit summing unit 347,447,847,947,1047,1147,1247,1347,1447: Output register 315,415,815,915,1015,1115,1215,1315,1415: Multiplexer 500: Weight and extension 550: Weight and extension data table 510: Partial results 520: Count results 530: Extension results 531: Carry bit 540,640: Weight results 600: Bit flip 605,805,905,1005,1105,1205,1305,1405: Bit flip data table 660,760: Flip results 700: Result summation 770: Group summation 780: Sum results 790: Accumulated results 1160: Shift sum unit 1311,1312,1313,1314,1411,1412,1413,1414: Memory plane 1500: Operation method S1501, S1502, S1503, S1504, S1505, S1506: Steps P: Deviation value

FIG. 1 shows a schematic diagram of an example memory device. FIG. 2 shows a schematic diagram of the architecture of an accumulator of an example memory device. FIG. 3 shows a schematic diagram of the architecture of an accumulator of a memory device according to an embodiment of the present disclosure. FIG. 4 shows a schematic diagram of the architecture of a memory device having an accumulator according to an embodiment of the present disclosure. FIG. 5 shows a schematic diagram of the operation of weight and extension according to an embodiment of the present disclosure. FIG. 6 shows a schematic diagram of the operation of bit flipping according to an embodiment of the present disclosure. FIG. 7 shows a schematic diagram of the operation of result summing according to an embodiment of the present disclosure. FIG. 8 shows a schematic diagram of an example bus width design of an accumulator of a memory device according to an embodiment of the present disclosure. FIG. 9 is a schematic diagram showing another example bus bandwidth of an accumulator of a memory device according to an embodiment of the present disclosure. FIG. 10 is a schematic diagram showing an example register of a result summing unit of an accumulator of a memory device according to an embodiment of the present disclosure. FIG. 11 is a schematic diagram showing a structure of an accumulator of a memory device according to another embodiment of the present disclosure. FIG. 12 is a schematic diagram showing a structure of an accumulator of a memory device according to yet another embodiment of the present disclosure. FIG. 13 is a schematic diagram showing a structure of a memory device according to yet another embodiment of the present disclosure. FIG. 14 is a schematic diagram showing a structure of a memory device according to yet another embodiment of the present disclosure. FIG. 15 is a flow chart showing an operation method of an accumulator according to an embodiment of the present disclosure. The same reference numbers and names in the various figures represent the same elements. It should also be understood that the various exemplary embodiments shown in the figures are merely illustrative representations and are not necessarily drawn to scale.

300: Memory device

310: Cell array

315:Multiplexer

330: Page buffer

340:Accumulator

341:Counter

342: Register

343: Weight and extension unit

344: Bit flip unit

345: Result sum unit

346: offset summing unit

347: Output register

Claims (11)

An accumulator for in-memory computing comprises: A counter, for counting the corresponding bits having a first bit value in a plurality of bits of a plurality of partial results generated by the in-memory computing to generate a counting result; A weighting and extension unit, coupled to the counter, for extending each bit in the counting result to a preset bit number using a plurality of second bit values to generate a bit extension result, and for moving each bit having the first bit value in the bit extension result to the corresponding Wth bit in the preset bit number according to a weighting and extension data table to generate a weight result; A bit-flip unit, coupled to the weight and extension unit, is used to flip each bit in the corresponding bit value in the weight result according to a bit-flip data table to generate a flip result, and the bit-flip data table has P first bit values; A result summing unit, coupled to the bit-flip unit, is used to sum each value in the flip result to generate a summed result; and A bias add unit, coupled to the result summing unit, is used to add a bias value to the summed result to generate an accumulated result, and the bias value is P, Wherein, P and W are positive integers greater than or equal to 0. An accumulator as described in claim 1, wherein a bandwidth of each of the one or more buses between the counter and the offset summing unit corresponds to a weight of data and an input of N bits, M dimensions and a carry bit, N and M are integers greater than or equal to 1. An accumulator as described in claim 1, wherein the result summing unit has one or more registers, the number of which is inversely proportional to the number of cycles required to sum each of the values in the flipped result to generate the summed result. The accumulator as described in claim 1, further comprising at least one shift-and-sum unit corresponding to data encoded with error-bit-tolerance. A memory device for in-memory computing, comprising: A plurality of memory planes, each of the memory planes comprising: A cell array for storing a plurality of weights; A page buffer coupled to the cell array, the page buffer for reading the weights in the cell array and generating a plurality of partial results according to a plurality of input operations; A multiplexer coupled to the page buffer for selecting and outputting the partial results in the page buffer; and An accumulator as described in any one of claim items 1 to 4, coupled to the multiplexer. A memory device for in-memory operations, comprising: A plurality of memory planes, each of which comprises: A cell array for storing a plurality of weights; A page buffer coupled to the cell array, the page buffer for reading the weights in the cell array and generating a plurality of partial results according to a plurality of input operations; A multiplexer coupled to the page buffer for selecting and outputting the partial results in the page buffer; ; and A counter coupled to the multiplexer, the counter being used to count the corresponding bits having a first bit value among the plurality of bits of the selected partial results to generate a counting result; and An accumulator coupled to the memory planes, the accumulator comprising: A register for temporarily storing the counting results in each of the memory planes; A weight and extension unit, coupled to the register, for extending each bit in each of the counting results by a preset bit number using a plurality of second bit values to generate a bit extension result, and for moving each bit having the first bit value in the bit extension result to the corresponding Wth bit in the preset bit number according to a weight and extension data table to generate a weight result; A bit flip unit, coupled to the weight and extension unit, for flipping each bit in the corresponding bit value in the weight result according to a bit flip data table to generate a flip result, and the bit flip data table has P first bit values; A result summing unit, coupled to the bit flipping unit, is used to sum the values in the flipping result to generate a summed result; and A bias summing unit, coupled to the result summing unit, is used to add a bias value to the summed result to generate an accumulated result, the bias value is P, wherein P and W are positive integers greater than or equal to 0. A memory device as described in claim 6, wherein a bandwidth of each of one or more buses between a plurality of units in the accumulator corresponds to the number O of the memory planes, N bits of input, M dimension and a carry bit, and M, N and O are integers greater than or equal to 1. A memory device as described in claim 6, wherein the result summing unit has one or more registers, the number of which is inversely proportional to the number of cycles required to sum each of the values in the flip result to generate the summed result. A memory device as described in claim 6, wherein the accumulator further includes at least one shift-and-sum unit corresponding to data encoded with error-bit-tolerant data. A method for operating an accumulator, comprising: Counting the corresponding bits having a first bit value in a plurality of bits of a plurality of partial results generated by calculation in a memory to generate a counting result; Extending each bit in the counting result to a preset bit number using a plurality of second bit values to generate a one-bit extension result; According to a weight and extension data table, moving each bit having the first bit value in the bit extension result to the corresponding Wth bit in the preset bit number to generate a weight result; Flipping each bit in the corresponding bit value in the weight result according to a one-bit flip data table to generate a flip result, and the bit flip data table has P first bit values; Adding up each value in the flip result to generate a summed result; and Add a bias value to the summed result to generate a cumulative result, the bias value is P, wherein, P and W are positive integers greater than or equal to 0. An operating method as described in claim 10, wherein data corresponding to the error bit tolerant data encoding and the partial results generated by the operation based on the data are decoded and summed to generate the summed result.

Images