TWI796977B - Memory device and operation method thereof - Google Patents

Abstract

A memory device and an operation method thereof are provided. A memory device and an operation method thereof are provided. The operation method includes: encoding an input data, sending an encoded input data to at least one page buffer, and reading out the encoded input data in parallel; encoding a first part and a second part of a weight data into an encoded first part and an encoded second part of the weight data, respectively, writing the encoded first part and the encoded second part of the weight data into a plurality of memory cells of the memory device, and reading out the encoded first part and the encoded second part of the weight data in parallel; multiplying the encoded input data with the encoded first part and the encoded second part of the weight data respectively to parallel generate a plurality of partial products; and accumulating the partial products to generate an operation result.

Description

Memory device and method of operation thereof

The present invention relates to a memory device with In-Memory-Computing (IMC) and an operating method thereof.

Artificial intelligence (AI) has become a highly effective solution in many fields. The key operation of AI is to perform a multiply-and-accumulate (MAC) operation on a large amount of input data (such as input feature maps) and weight values.

However, with the current AI architecture, it is easy to encounter an IO bottleneck and an inefficient MAC operation process.

To achieve high accuracy, MAC operations can be performed with multi-bit inputs and multi-bit weight values. However, the I/O bottleneck becomes more severe and will be less efficient.

In-Memory-Computing (IMC) can be used to accelerate MAC operations, because IMC can reduce the complex arithmetic logic unit (ALU) required under the central processing architecture, and provide MAC in memory High parallelism of operations.

When performing IMC, the multiplication operations of unsigned integers and signed integers are described as follows.

For example, you want to multiply two unsigned numbers (both 8 bits): a[7:0] and b[7:0]. Then 8 unit multiplications can be performed to generate 8 partial products (partial product) p0[7:0]~p7[7:0], each of which is related to each bit of the multiplicand a , the 8 partial products can be expressed as follows: p0[7:0] = a[0] × b[7:0] = {8{a[0]}} & b[7:0] p1[7:0] = a[1] × b[7:0] = {8{a[1]}} & b[7:0] p2[7:0] = a[2] × b[7:0] = {8{a[2]}} & b[7:0] p3[7:0] = a[3] × b[7:0] = {8{a[3]}} & b[7:0] p4[7:0] = a[4] × b[7:0] = {8{a[4]}} & b[7:0] p5[7:0] = a[5] × b[7:0] = {8{a[5]}} & b[7:0] p6[7:0] = a[6] × b[7:0] = {8{a[6]}} & b[7:0] p7[7:0] = a[7] × b[7:0] = {8{a[7]}} & b[7:0]

Among them, {8{a[0]}} means to repeat a[0] 8 times, and the rest can be deduced in this way.

To obtain the product, add the 8 partial products p0[7:0]~p7[7:0], as shown in FIG. 1A. Figure 1A shows the multiplication of two unsigned numbers (both 8 bits).

Among them, P0=p0[0]+0+0+0+0+0+0+0, and P1=p0[1]+p1[0]+0+0+0+0+0+0, the rest can be So on and so forth.

The product P[15:0] is obtained by multiplying P0~P15. The product P[15:0] represents a 16-bit unsigned product obtained by multiplying two unsigned numbers (both 8-bit).

And if b is a signed number, the partial product needs to be sign-extended to the width of the product before summing. If a is also signed, the partial product P7 is subtracted from the final sum, not added.

Figure 1B shows the multiplication of two signed numbers (both 8 bits). In Fig. 1B, the symbol "~" represents complement, for example, ~p1[7] represents the complement number of p1[7].

When performing IMC, if the "operation speed" can be accelerated and the capacity requirement can be reduced, it will be beneficial to the performance of IMC.

According to an example of this case, a memory device is proposed, including: a plurality of memory chips, each of which includes a plurality of memory planes, a plurality of page buffers and an accumulation circuit, and each of these memory planes It includes a plurality of memory cells. Wherein, an input data is encoded, an encoded input data is transmitted to at least one page buffer, and the encoded input data is read in parallel from the at least one page buffer; a first A part and a second part are respectively encoded as an encoded first part of the weight data and an encoded second part of the weight data, and written into the memory cells of the memory device In, and, read out the coded first part of the weight data and the coded second part of the weight data in parallel; multiply the coded input data by the coded first part of the weight data respectively Parts and the encoded second part of the weight data are used to generate a plurality of partial products in parallel; and these partial products are accumulated to generate an operation result.

According to another example of the present case, a method for operating a memory device is proposed, including: encoding an input data, transmitting an encoded input data to at least one page buffer, and reading in parallel from the at least one page buffer input data after outputting the coding; a first part and a second part of a weight data are respectively coded as a coded first part of the weight data and a coded second part of the weight data, and writing the encoded first part of the weight data and the encoded second part of the weight data into a plurality of memory cells of the memory device, and reading out the weight data in parallel The coded first part and the coded second part of the weight data; multiply the coded input data by the coded first part of the weight data and the coded second part of the weight data respectively Parts are used to generate a plurality of partial products in parallel; and these partial products are accumulated and accumulated to generate an operation result.

In order to have a better understanding of the above-mentioned and other aspects of the present invention, the following specific examples are given in detail with the accompanying drawings as follows:

The technical terms in this specification refer to the customary terms in this technical field. If some terms are explained or defined in this specification, the explanations or definitions of these terms shall prevail. Each embodiment of the disclosure has one or more technical features. On the premise of possible implementation, those skilled in the art may selectively implement some or all of the technical features in any embodiment, or selectively combine some or all of the technical features in these embodiments.

FIG. 2 shows a flow chart of the operation method of the memory device according to an embodiment of the present invention. In step 210, the input data is encoded and the encoded input data (which is a vector) is sent to a plurality of page buffers, and read out from the page buffers in parallel. The details of how to encode the input data will be explained below.

In step 220, encode the weight data and write the encoded weight data (which is a vector) into a plurality of memory cells of the memory device, and read out the encoded weight data in parallel. The details of how to encode the weight data will be explained below. Wherein, when encoding, the most significant bit (most significant bit, MSB) and the least significant bit (least significant bit, LSB) of the weight data are respectively encoded.

In step 230, the input data are respectively multiplied by the MSB and LSB of the encoded weight data to generate a plurality of partial products in parallel.

In step 240, the partial products are summed (accumulated) to generate a multiply-and-accumulate (MAC) result or a Hamming distance (Hamming distance) result.

An embodiment of the present case discloses a memory device capable of implementing digital MAC operations, which has error-bit-tolerance data encoding to tolerate error bits and reduce area requirements. Error-tolerant data encoding uses input data duplication and weighted data flattening techniques. In addition, the sensing technology of the present embodiment includes standard single level cell (SLC) readout and logical AND (AND) function to perform bit multiplication to generate partial products. In other possible embodiments of the present application, if the page buffer does not remove the input data stored in the latch unit during the sensing process, the SLC read can be replaced by a select bit read, or by a multi-bit It is replaced by read operations of Multi-Level Cell (MLC), Triple Level Cell (TLC), and Quad-level Cells (QLC). In addition, the digital MAC operation in an embodiment of the present case uses a high bandwidth weighted accumulator to generate an output result, and the high bandwidth weighted accumulator can repeatedly use the fail-bit-count (fail-bit-count, FBC) circuit to implement weighted accumulation (weighted accumulation).

Another embodiment of the present case discloses a memory device capable of implementing Hamming distance calculation, which has error-bit-tolerant data encoding to tolerate error bits. Error-tolerant data encoding uses input data duplication and weighted data flattening techniques. In addition, the sensing technology of the present embodiment includes a standard single level cell (SLC) read and logical exclusive OR (EXOR) function to perform bit multiplication to generate partial products. In other possible embodiments of this case, if the page buffer does not remove the input data stored in the latch unit during sensing, the SLC read can be replaced by a select bit read, or by MLC, Superseded by TLC, QLC read operations. And the logical exclusive OR (EXOR) function can be replaced by the logical anti-mutual exclusive OR (XNOR) and logical inverse. In addition, the digital Hamming distance operation of an embodiment of the present case uses a high bandwidth unweighted accumulator to generate output results. The high bandwidth unweighted accumulator can be used repeatedly to bit-count (FBC) circuit to implement unweighted accumulation (unweighted accumulation).

Figures 3A and 3B show error-tolerant data encoding in this embodiment. For example but not limited thereto, the input data and weight data are floating point 32 (floating point 32) data. In FIG. 3A, the input data and the weight data are quantized into 8-bit binary integers, wherein both the input data and the weight data are 8-bit vectors with N dimensions (N is a positive integer). The input data and weight data can be expressed as _Xi (7:0) and W _i (7:0) respectively.

In FIG. 3B, the N-dimensional 8-bit weight vectors are divided into MSB vectors and LSB vectors. The MSB vector of the 8-bit weight vector includes 4 bits W _i (7:4), and the LSB vector includes 4 bits W _i (3:0).

Next, each bit of the MSB vector and the LSB vector of the 8-bit weight vector is represented by unary coding (ie, a value format). For example, the bit W _i=0 (7) of the MSB vector of the 8-bit weight vector can be expressed as 8 bits (replicated 8 times), and the bit W _i=0 (6) of the MSB vector of the 8-bit weight vector Can be expressed as 4 bits (copy 4 times), bit W _i=0 (5) of MSB vector of 8-bit weight vector can be expressed as 2 bits (copy 2 times), MSB vector of 8-bit weight vector The bit W _i=0 (4) of can be expressed as 1 bit (copy once), and the spare bit (spare bit) (0) is added to the bit W _i= of the MSB vector of the 8-bit weight vector After ₀ (4). In this way, the 4-bit MSB vector of the 8-bit weight vector can be encoded into a 16-bit unary coding (Unary coding) format.

Similarly, the 4-bit LSB vector of the 8-bit weight vector can be encoded into a 16-bit unary coding (Unary coding) format.

In the first embodiment of the present case, the error bit tolerance can be improved through the previous encoding method.

FIG. 4A shows an 8-bit unsigned integer multiplication operation in one embodiment of the present case, and FIG. 4B shows an 8-bit signed integer multiplication operation in an embodiment of the present case.

As shown in Figure 4A, when multiplying 8-bit unsigned numbers, at cycle 0, multiply the input data _Xi (7) (the input data has been encoded into a unary encoding format) by the MSB of the weight data Vector W _i (7:4) (the MSB vector of the weight data has been encoded into a unary encoding format) to obtain the first MSB partial product. Similarly, X _i (7) of the input data is multiplied by the LSB vector W _i (3:0) of the weight data (the LSB vector of the weight data has been encoded into a unary encoding format) to obtain the first LSB partial product. The first MSB partial product is shifted by 4 bits and added to the first LSB partial product to obtain the first partial product.

In the first cycle, multiply Xi ₍ 6) of the input data by the MSB vector W _i (7:4) of the weight data (the MSB vector of the weight data has been encoded into a unary encoding format) to obtain the second MSB part product. Similarly, X _i (6) of the input data is multiplied by the LSB vector W _i (3:0) of the weight data (the LSB vector of the weight data has been encoded into a unary encoding format) to obtain the second LSB partial product. The second MSB partial product is shifted by 4 bits and added to the second LSB partial product to obtain the second partial product. In addition, the first part of the product is shifted by 1 bit and added to the second part of the product to obtain the updated second part of the product. The operations of the remaining cycles (the second cycle to the seventh cycle) can be deduced in a similar manner, which will not be repeated here.

That is, the multiplication of 8-bit unsigned numbers can be completed in 8 cycles.

As shown in FIG. 4B, when performing multiplication of 8- _{bit signed numbers, at cycle 0, multiply Xi (7) of input data by MSB vector W i (} 7) of weight data (weight data of The MSB vector has been encoded into a unary encoding format), _{and the input data Xi (7) is multiplied by the MSB vector W i (} 6:4) of the weight data (the MSB vector of the weight data has been encoded into a unary encoding format) and reversed , the two are added to obtain the first MSB partial product. Multiply _Xi (7) of the input data by the LSB vector W _i (3:0) of the weight data (the LSB vector of the weight data has been encoded into a unary encoding format) and invert to obtain the first LSB partial product. The first MSB partial product is shifted by 4 bits and added to the first LSB partial product to obtain the first partial product.

In the first cycle, the input data Xi ₍ 6) is multiplied by the MSB vector W _i (7) of the weight data (the MSB vector of the weight data has been encoded into a unary encoding format) and inverted, and the input data X _i (6) is multiplied by the MSB vector W _i (6:4) of the weight data (the MSB vector of the weight data has been encoded into a unary encoding format), and the two are added to obtain the second MSB partial product. Similarly, X _i (6) of the input data is multiplied by the LSB vector W _i (3:0) of the weight data (the LSB vector of the weight data has been encoded into a unary encoding format) to obtain the second LSB partial product. The second MSB partial product is shifted by 4 bits and added to the second LSB partial product to obtain the second partial product. In addition, the first part of the product is shifted by 1 bit and added to the second part of the product to obtain the updated second part of the product. The operations of the remaining cycles (the second cycle to the seventh cycle) can be deduced in a similar manner, which will not be repeated here.

That is, the multiplication of 8-bit signed numbers can be completed in 8 cycles.

The above method needs 8 cycles to complete the 8-bit unsigned number multiplication operation and the 8-bit signed number multiplication operation.

FIG. 5A shows a schematic diagram of the operation of multiplication of unsigned numbers according to an embodiment of the present invention. FIG. 5B shows a schematic diagram of the operation of multiplication of signed numbers according to an embodiment of the present invention. FIG. 5A and FIG. 5B take an example in which both the input data and the weight data are 8-bit for illustration, but it should be understood that the present case is not limited thereto.

In Figures 5A and 5B, the input data is also encoded, and the MSB vector and LSB vector of the weight data are encoded into a unary encoding format.

In FIG. 5A and FIG. 5B, the input data is input into the page buffer, and the weight data is written into a plurality of memory cells.

In FIG. 5A, the input data is read in parallel from the page buffers, and the weight data are read in parallel from the memory cells and multiplied in parallel to obtain partial products.

In detail, the bit X _i (7) of the input data is multiplied by the MSB vector W _i (7:4) of the weight data to obtain the first MSB partial product. The bit X _i (6) of the input data is multiplied by the MSB vector W _i (7:4) of the weight data to obtain a second MSB partial product. The rest can be deduced in the same way until the bit _Xi (0) of the input data is multiplied by the MSB vector W _i (7:4) of the weight data to obtain the eighth MSB partial product. For example, in Fig. 5A, the bit _Xi (7) of the input data is copied 15 times and a spare bit is added to form a 16-bit multiplier "0000000000000000". The 16-bit multiplier "00000000000000000" is multiplied by the MSB vector W _i (7:4) "1111111100001100" of the weight data to obtain the first MSB partial product "0000000000000000". The rest can be deduced by analogy. All MSB partial products can be combined into an input stream M.

Similarly, X _i (7) of the input data is multiplied by the LSB vector W _i (3:0) of the weight data to obtain the first LSB partial product. Multiply _Xi (6) of the input data by the LSB vector W _i (3:0) of the weight data to obtain a second LSB partial product. The rest can be deduced in the same way until the bit _Xi (0) of the input data is multiplied by the LSB vector W _i (3:0) of the weight data to obtain the eighth LSB partial product. All LSB partial products can be combined to form the input stream L.

Afterwards, the first to eighth MSB partial products are combined with the first to eighth LSB partial products, and the number of bits 1 of the combined value is counted to obtain the MAC operation of the unsigned multiplication operation result.

In FIG. 5B, the input data is read in parallel from the page buffers, and the weight data are read in parallel from the memory cells and multiplied in parallel to obtain partial products.

In detail, the bit X _i (7) of the input data is multiplied by the MSB vector W _i (7:4) of the weight data to obtain the first MSB partial product. The bit X _i (6) of the input data is multiplied by the MSB vector W _i (7:4) of the weight data to obtain a second MSB partial product. The rest can be deduced in the same way until the bit _Xi (0) of the input data is multiplied by the MSB vector W _i (7:4) of the weight data to obtain the eighth MSB partial product.

Similarly, X _i (7) of the input data is multiplied by the LSB vector W _i (3:0) of the weight data to obtain the first LSB partial product. Multiply _Xi (6) of the input data by the LSB vector W _i (3:0) of the weight data to obtain a second LSB partial product. The rest can be deduced in the same way until the bit _Xi (0) of the input data is multiplied by the LSB vector W _i (3:0) of the weight data to obtain the eighth LSB partial product.

Afterwards, combining the products of the first to eighth MSB parts with the products of the first to eighth LSB parts, and counting the number of bits 1 of the combined value, the MAC operation of multiplication with signed numbers can be obtained result.

FIG. 6 shows a functional block diagram of a memory device according to an embodiment of the present invention. The memory device 600 includes a plurality of memory dies (die) 615 . In FIG. 6, the memory device 600 includes four memory chips 615 as an example for illustration, but it should be understood that the present application is not limited thereto.

The memory die 615 includes a plurality of memory planes (MP) 620 , a plurality of page buffers 625 and an accumulation circuit 630 . In FIG. 6, the memory die 615 includes four memory planes 620 and four page buffers 625 as an example for illustration, but it should be understood that the present invention is not limited thereto. The memory plane 620 includes a plurality of memory cells (not shown). Weight data is stored in the memory cells.

In each memory die 615 , the accumulation circuit 630 is shared by the memory planes 620 , so the accumulation circuit 630 executes the accumulation operation of the memory planes 620 sequentially. In addition, each memory die 615 can independently execute the above-mentioned digital MAC operation and digital Hamming distance operation in this embodiment.

Input data can be input into the page buffers 625 through a plurality of word lines.

The page buffer 625 includes a sensing circuit 631 , a plurality of latch units 633 - 641 , and a plurality of logic gates 643 and 645 .

The sensing circuit 631 is coupled to the bit line BL to sense the current on the bit line BL.

The latch units 633-641 are, for example but not limited to, data latches (data latch, DL) 633, latches (L1) 635, latches (L2) 637, latches (L3) 639 and a common data latch (CDL) 641 . The latch units 633-641 are, for example but not limited to, single cell latches.

The data latch 633 is used to latch the weight data and output the weight data to logic gates 643 and 645 .

The latch (L1) 635 and the latch (L3) 639 are used for decoding.

The latch ( L2 ) 637 is used to latch the input data and output the input data to logic gates 643 and 645 .

The common data latch 641 is used to latch the data transmitted from the logic gate 643 or 645 .

Logic gates 643 and 645 are, for example but not limited to, logic AND gates and logic XOR gates, respectively. The logic gate 643 performs a logic AND operation on the input data and the weight data, and writes the logic operation result into the common data latch 641 . The logic gate 645 performs a logic XOR operation on the input data and the weight data, and writes the logic operation result into the common data latch 641 . The logic gates 643 and 645 are respectively controlled by enable signals AND_EN and XOR_EN. For example, when the digital MAC operation is performed, the logic gate 643 is enabled by the enable signal AND_EN; and when the digital Hamming distance operation is performed, the logic gate 645 is enabled by the enable signal XOR_EN.

As illustrated in Figure 5A or Figure 5B, one bit of the bit _Xi (7) of the input data is input to the latch (L2) 637, and the MSB vector of the weight data that has been encoded into a unary encoding format One bit of _Wi (7:4) is input to the data latch 633 . The input data of the latch (L2) 637 and the weight data of the data latch 633 are logically operated by the logic gate 643 or 645, and the common data latch 641 is used to latch the data transmitted from the logic gate 643 or 645. data of. The common data latch 641 can also be regarded as the data output path of the bit line.

The accumulation circuit 630 includes: a partial product accumulation unit 651 , a single-dimensional product generation unit 653 , a first multi-dimensional accumulation unit 655 , a second multi-dimensional accumulation unit 657 and a weight accumulation control unit 659 .

The partial product accumulation unit 651 is coupled to the page buffer 625 to receive a plurality of logical operation results transmitted from the plurality of common data latches 641 of the page buffer 625 to generate a plurality of partial products.

For example, taking FIG. 5A or FIG. 5B, the partial product accumulation unit 651 generates the first to eighth MSB partial products and the first to eighth LSB partial products.

The single-dimensional product generation unit 653 is coupled to the partial product accumulation unit 651 and accumulates the partial products generated by the partial product accumulation unit 651 to generate a single-dimensional product.

For example, taking Fig. 5A or Fig. 5B as an example, the single-dimensional product generating unit 653 generates the first to eighth MSB partial products and the first to eighth LSB partial products generated by the partial product accumulation unit 651 Partial multiplications are accumulated to produce single-dimensional products.

For example, after the <0> dimension product is generated in the 0th cycle, the <1> dimension product can be generated in the 1st cycle, and so on.

The first multi-dimensional accumulation unit 655 is coupled to the single-dimensional product generating unit 653, and accumulates the multiple single-dimensional products generated by the single-dimensional product generating unit 653 to obtain a multi-dimensional product accumulation result.

For example but not limited thereto, the first multi-dimensional accumulation unit 655 accumulates the multiplication and accumulation of the <0>th to <7th> dimensions generated by the single-dimensional product generation unit 653 to obtain an 8-dimensional <0:7> multiplication and accumulation result. Next, the first multi-dimensional accumulation unit 655 accumulates the multiplication and accumulation of the <8th> to <15th> dimensions generated by the single-dimensional product generation unit 653 to obtain another 8-dimensional <8:15> multiplication and accumulation result.

The second multi-dimensional accumulating unit 657 is coupled to the first multi-dimensional accumulating unit 655 and accumulates a plurality of multi-dimensional multi-dimensional accumulation results generated by the first multi-dimensional accumulating unit 655 to obtain an output accumulating value. For example but not limited thereto, the second multi-dimensional accumulation unit 657 accumulates 64 8-dimensional multiplication and accumulation results generated by the first multi-dimensional accumulation unit 655 to obtain a 512-dimensional output accumulation value.

The weight accumulation control unit 659 is coupled to the partial product accumulation unit 651 , the single-dimensional product generation unit 653 , and the first multi-dimensional accumulation unit 655 . According to performing a digital MAC operation or a digital Hamming distance operation, the weight accumulation control unit 659 is enabled or disabled. For example, but not limited to, when performing digital MAC operation, the weight accumulation control unit 659 is enabled; and when performing digital Hamming distance operation, the weight accumulation control unit 659 is disabled. When the weight accumulation control unit 659 is enabled, the weight accumulation control unit 659 outputs control signals to the partial product accumulation unit 651 , the single-dimensional product generation unit 653 , and the first multi-dimensional accumulation unit 655 according to the weight accumulation enable signal WACC_EN.

A single page buffer 620 in FIG. 6 is coupled to a plurality of bit lines BL. For example but not limited thereto, each page buffer 620 is coupled to 131072 bit lines BL, and data results on 128 bit lines BL are selected in each cycle to be accumulated by the accumulation circuit 630 . In this case, it takes 1024 cycles to send the data on the 131072 bit lines BL.

In addition, in the above description, the partial multiply-accumulate unit 651 receives 128 bits at a time, the first multi-dimensional accumulate unit 655 produces an 8-dimensional multiply-accumulate result, and the second multi-dimensionally accumulate unit 657 produces a 512-dimensional output accumulated value. But this case is not limited to this. In another possible embodiment, the partial multiply-accumulate unit 651 receives 64 bits at a time (2 bits are 1 group), the first multi-dimensional accumulate unit 655 generates a 16-dimensional multiply-accumulate result, and the second multi-dimensionally accumulate unit 657 Produces an output accumulation of 512 dimensions.

FIG. 7 shows a sequence diagram of the MAC operation flow comparing an embodiment of the present case with the prior art. As shown in Fig. 7, input data is received during input broadcasting time. Afterwards, bit multiplication and bit accumulation are performed on the input data and the weight data to generate a MAC operation result.

In the prior art, a long operation time is required. On the contrary, in the present embodiment, (1) the partial product of the input vector and the MSB vector of the weight data, and (2) the partial product of the input vector and the LSB vector of the weight data are generated by parallel multiplication. In this way, the unsigned number multiplication operation and/or the signed number multiplication operation can be completed within one cycle. Therefore, the operation speed of the embodiment of the present case is faster than that of the conventional technology.

Fig. 8 shows an operation method of a memory device according to an embodiment of the present invention, including: encoding an input data, transmitting an encoded input data to at least one page buffer, and, from the at least one page buffer Read out the coded input data in parallel (810); code a first part and a second part of a weight data into a coded first part of the weight data and a coded code of the weight data respectively The second part is written into a plurality of memory cells of the memory device, and the encoded first part of the weight data and the encoded second part of the weight data are read in parallel (820); respectively multiplying the encoded input data by the encoded first part of the weight data and the encoded second part of the weight data to generate a plurality of partial products in parallel (830); and The partial products are accumulated and accumulated to generate an operation result (840).

As mentioned above, in the embodiment of the present case, error bits can be reduced through the bit error tolerant coding method, the accuracy can be improved and the demand for memory capacity can be reduced.

In addition, the digital MAC operation in an embodiment of the present case uses a high-bandwidth weight accumulator to generate output results. The high-bandwidth weight accumulator can implement weighted accumulation by repeatedly using the faulty bit counting circuit, so the accumulation speed can be improved.

The digital Hamming distance operation operation of an embodiment of the present case uses a high-bandwidth unweighted accumulator to generate output results. The high-bandwidth unweighted accumulator can implement unweighted accumulation by reusing the faulty bit counting circuit, so it can improve accumulation. acceleration.

The above embodiments of the present case can be applied to NAND flash memory, or memory devices sensitive to error bits, such as but not limited to, NOR flash memory, phase change (PCM) flash memory, Magnetic RAM or resistive RAM.

In the above embodiment, the accumulation circuit 630 can receive 128 partial products transmitted from the page buffer 625, but in other embodiments of this case, the accumulation circuit 630 can receive 2, 4, 8, 16...512 partial products (which are powers of 2), which are also within the spirit of this case.

In the above embodiments, the accumulation circuit 630 can support the addition function, but in other embodiments of the present application, the accumulation circuit 630 can support the subtraction function, which is also within the spirit of the present application.

In the above embodiment, although the MAC operation of INT8 or UNIT8 is used as an example for illustration, in other embodiments of this case, the MAC operation of INT2, UNIT2, INT4, and UNIT4 can also be supported, which is also within the spirit of this case.

Although in the above embodiments, the weight data is divided into MSB vectors and LSB vectors (two vectors), the present application is not limited thereto. In other possible embodiments of this case, the weight data can also be divided into more vectors, which is also within the spirit of this case.

The above-mentioned embodiments of this case can be applied to the design of AI models that require MAC operations, such as but not limited to, fully-connection layer, convolution layer, multi-layer perceptron (multiple layer Perceptron), Among AI technologies such as support vector machine.

The above in this case can be applied not only to computing usage, but also to similarity search, analysis usage, clustering analysis, etc.

To sum up, although the present invention has been disclosed by the above embodiments, it is not intended to limit the present invention. Those skilled in the art of the present invention can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be defined by the scope of the appended patent application.

210-240: Steps 600: memory device 615: memory die 620: memory plane 625: page buffer 630: accumulation circuit 631: induction circuit 633-641: Latch unit 643, 645: logic gate 651: Partial multiply-accumulate unit 653:Single-dimension product generation unit 655: The first multi-dimensional accumulation unit 657: The second multi-dimensional accumulation unit 659: Weight accumulation control unit 810-840: Steps

Figure 1A shows the multiplication of two unsigned numbers. Figure 1B shows the multiplication of two signed numbers. FIG. 2 shows a flow chart of the operation method of the memory device according to an embodiment of the present invention. Figures 3A and 3B show error-tolerant data encoding in this embodiment. FIG. 4A shows the multiplication operation of 8-bit unsigned numbers in one embodiment of the present invention, and FIG. 4B shows the multiplication operation of signed 8-bit numbers in one embodiment of the present case. FIG. 5A shows a schematic diagram of the operation of multiplication of unsigned numbers according to an embodiment of the present invention. FIG. 5B shows a schematic diagram of the operation of multiplication of signed numbers according to an embodiment of the present invention. FIG. 6 shows a functional block diagram of a memory device according to an embodiment of the present invention. FIG. 7 shows a sequence diagram of the MAC operation flow comparing an embodiment of the present case with the prior art. FIG. 8 shows a method of operating a memory device according to an embodiment of the present invention.

810-840: Steps

Claims (10)

A memory device comprising: a plurality of memory dies, each of the memory dies includes a plurality of memory planes, a plurality of page buffers and an accumulation circuit, each of the memory planes includes a plurality of memory cells, Wherein, an input data is encoded, an encoded input data is transmitted to at least one page buffer, and the encoded input data is read out in parallel from the at least one page buffer; Encoding a first part and a second part of a weight data into a coded first part of the weight data and a coded second part of the weight data respectively, and writing them into the memory In the memory cells of the device, and read the encoded first part of the weight data and the encoded second part of the weight data in parallel; multiplying the encoded input data by the encoded first part of the weighting data and the encoded second part of the weighting data, respectively, to generate a plurality of partial products in parallel; and The partial products are accumulated and accumulated to generate an operation result. The memory device as claimed in item 1, wherein, The first part of the weight data is a most significant bit (MSB), and the second part of the weight data is a least significant bit (LSB). The memory device as claimed in item 1, wherein, During encoding, the input data and the weight data are respectively quantized into binary integer vectors; copy each bit of the input data a plurality of times and add a spare bit; separate the weighting data into the first part and the second part; and Each bit of the first part and the second part of the weight data is represented by a unitary code, so as to obtain the coded first part of the weight data and the coded second part of the weight data. The memory device as claimed in item 1, wherein, The operation result includes a multiply-and-accumulate (MAC) operation result or a Hamming distance (Hamming distance) operation result; and, accumulating the partial products belonging to the same dimension to obtain a single-dimensional product; Multiplying and accumulating multiple single-dimensions to obtain a multi-dimensional multiplication-accumulation result; A plurality of multi-dimensional multiply-accumulate results are accumulated to generate the operation result. The memory device as described in claim 4, wherein, When performing a multiply-accumulate operation, perform a logical AND operation on each bit of the encoded input data and each bit of the encoded first part of the weight data; and When the Hamming distance calculation is performed, a logical exclusive OR operation is performed on each bit of the encoded input data and each bit of the encoded first part of the weight data. A method of operating a memory device, comprising: Encoding an input data, transferring an encoded input data to at least one page buffer, and reading the encoded input data in parallel from the at least one page buffer; A first part and a second part of a weight data are respectively coded as a coded first part of the weight data and a coded second part of the weight data, and the weight data of the writing the encoded first part and the encoded second part of the weight data into a plurality of memory cells of the memory device, and reading out the encoded first part of the weight data in parallel with the encoded second part of the weight data; multiplying the encoded input data by the encoded first part of the weighting data and the encoded second part of the weighting data, respectively, to generate a plurality of partial products in parallel; and The partial products are accumulated and accumulated to generate an operation result. The method of operating a memory device according to claim 6, wherein, The first part of the weight data is a most significant bit (MSB), and the second part of the weight data is a least significant bit (LSB). The method of operating a memory device according to claim 6, wherein, During encoding, the input data and the weight data are respectively quantized into binary integer vectors; copy each bit of the input data a plurality of times and add a spare bit; separate the weighting data into the first part and the second part; and Each bit of the first part and the second part of the weight data is represented by a unitary code, so as to obtain the coded first part of the weight data and the coded second part of the weight data. The method of operating a memory device according to claim 6, wherein, The operation result includes a multiply-and-accumulate (MAC) operation result or a Hamming distance (Hamming distance) operation result; and accumulating the partial products belonging to the same dimension to obtain a single-dimensional product; Multiplying and accumulating multiple single-dimensions to obtain a multi-dimensional multiplication-accumulation result; A plurality of multi-dimensional multiply-accumulate results are accumulated to generate the operation result. The method of operating a memory device according to claim 9, wherein, When performing a multiply-accumulate operation, perform a logical AND operation on each bit of the encoded input data and each bit of the encoded first part of the weight data; and When the Hamming distance calculation is performed, a logical exclusive OR operation is performed on each bit of the encoded input data and each bit of the encoded first part of the weight data.

Images