TW202211217A - Memory device and operation method thereof - Google Patents

Abstract

A memory device and an operation method thereof are provided. The memory device includes: a memory array including a plurality of memory cells for storing a plurality of weights; a multiplication circuit for performing bitwise multiplication on a plurality of input data and the weights to generate a plurality of multiplication results, wherein in performing bitwise multiplication, the memory cells generate a plurality of memory cell currents; a digital accumulating circuit for performing a digital accumulating on the multiplication results; an analog accumulating circuit for performing an analog accumulating on the memory cell currents to generate a first MAC operation result; and a decision unit for deciding whether to perform the analog accumulating, the digital accumulating or a hybrid accumulating, wherein in performing the hybrid accumulating, whether the digital accumulating circuit is triggered is based on the first MAC operation result.

Description

Memory device and method of operating the same

The present invention relates to a memory device with In-Memory-Computing (IMC) and an operation method thereof.

Artificial Intelligence (AI) has become a highly effective solution in many fields. The key operation of AI is to perform multiply-and-accumulation (MAC) operations on a large amount of input data (such as input feature maps) and weight values.

However, with the current AI architecture, it is easy to encounter an IO bottleneck and an inefficient MAC operation flow.

To achieve high accuracy, MAC operations with multi-bit inputs and multi-bit weight values can be performed. However, the I/O bottleneck becomes more severe and the efficiency will be lower.

In-Memory-Computing (IMC) can be used to accelerate MAC operations, because IMC can reduce the complex Arithmetic logic unit (ALU) required under the central processing architecture, and provide an in-memory MAC High parallelism of operations.

In terms of non-volatile memory IMC (NVM-based IMC), its advantages are, for example, non-volatile storage and reduced data transfer.

When performing IMC, it will be helpful for IMC performance if the "operation speed" and "operation accuracy" can be taken into account at the same time.

According to an example of the present application, a memory device is proposed, comprising: a memory array including a plurality of memory cells, which can be used to store a plurality of weight values in the memory cells of the memory array; a multiplication circuit coupled to connected to the memory array, the multiplication circuit multiplies a plurality of input data and the weight values to obtain a plurality of multiplication results, wherein when the multiplication is performed, the memory cells generate a plurality of memory cell currents; a A digital accumulation circuit, coupled to the multiplication circuit, performs a digital accumulation on the multiplication results; an analog accumulation circuit, coupled to the memory array, performs an analog accumulation on the memory cell currents generating a first multiply-accumulate (MAC) operation result; and a determination unit, coupled to the digital-accumulation circuit and the analog-accumulation circuit, determines to perform the analog-accumulation, the digital-accumulation or a hybrid-accumulation, Wherein, when the hybrid accumulation is performed, whether to trigger the digital accumulation circuit is determined according to the operation result of the first multiply-accumulate operation.

According to another example of the present application, a method for operating a memory device is proposed, comprising: storing a plurality of weight values in a plurality of memory cells of a memory array of the memory device; comparing a plurality of input data and the weight values performing a bitwise multiplication to obtain a plurality of multiplication results, wherein when performing the multiplication, the memory cells generate a plurality of memory cell currents; and deciding to perform an analog accumulation, a digital accumulation or a hybrid accumulation, wherein , when performing the analog accumulation, perform the analog accumulation on the memory cell currents to generate a first multiply-accumulate operation (MAC) operation result; when performing the digital accumulation, perform the multiplication results on the multiplication results. The digital accumulation generates a second multiply-accumulate operation result; and when the hybrid accumulation is performed, whether to trigger the digital-accumulation is determined according to the first multiply-accumulate operation result.

In order to have a better understanding of the above-mentioned and other aspects of the present invention, the following specific examples are given and described in detail in conjunction with the accompanying drawings as follows:

The technical terms in this specification refer to the common terms in the technical field. If some terms are described or defined in this description, the interpretations of these terms are subject to the descriptions or definitions in this description. Each embodiment of the present disclosure has one or more technical features. Under the premise of possible implementation, those skilled in the art can selectively implement some or all of the technical features in any embodiment, or selectively combine some or all of the technical features in these embodiments.

Please refer to FIG. 1 , which shows a functional block diagram of a memory device 100 having an In-Memory-Computing (IMC) function according to an embodiment of the present invention. The memory device 100 with in-memory computing function includes: a memory array 110 , a multiplying circuit 120 , an I/O circuit 130 , a digital accumulation circuit 135 , an analog accumulation circuit 160 , a determination unit 170 and a comparator 163 . The digital accumulating circuit 135 includes a grouping circuit 140 and a counting unit 150 . The analog accumulation circuit 160 includes an analog to digital conversion unit 161 . Among them, the memory array 110, the multiplication circuit 120 and the analog accumulation circuit 160 and the analog to digital conversion unit 161 are analog, while the digital accumulation circuit 135, the grouping circuit 140 and the counting unit 150 are digital.

The memory array 110 includes a plurality of memory cells 111 . In an embodiment of the present application, the memory unit 111 is, for example, but not limited to, a non-volatile memory unit. When performing MAC operations, the memory unit 111 may be used to store weights.

The multiplying circuit 120 is coupled to the memory array 110 . The multiplying circuit 120 includes a plurality of unitary multiplying units 121 . Each unit multiplication unit 121 includes an input latch 121A, a sense amplifier (SA) 121B, an output latch 121C, and a common data latch (CDL) 121D. The input latch 121A is coupled to the memory array 110 . The sense amplifier 121B is coupled to the input latch 121A. The output latch 121C is coupled to the sense amplifier 121B. The common data latch 121D is coupled to the output latch 121C.

The I/O circuit 130 is coupled to the multiplying circuit 120 , the grouping circuit 140 and the counting unit 150 for receiving input data and outputting the output data obtained by the memory device 100 .

The digital accumulation circuit 135 is used for digital accumulation, the details of which will be described below.

The analog accumulation circuit 160 is used for analog accumulation, and the details of which will be described below.

The determination unit 170 determines whether the memory device 100 performs analog accumulation, digital accumulation or hybrid accumulation. The determining unit 170 can output the enabling signals EN1 and EN2 to the analog accumulating circuit 160 and the digital accumulating circuit 135 respectively to determine whether to enable the analog accumulating circuit 160 or the digital accumulating circuit 135 .

"Analog accumulation" means that the analog accumulation circuit 160 is enabled but the digital accumulation circuit 135 is not enabled. "Digital accumulation" means that the digital accumulation circuit 135 is enabled but the analog accumulation circuit 160 is not enabled. "Hybrid accumulation" means that the digital accumulation circuit 135 and the analog accumulation circuit 160 are enabled.

The analog-to-digital conversion unit 161 is coupled to the memory cells 111 of the memory array 110 . The cell currents of the memory cells 111 can be accumulated and input to the analog-to-digital conversion unit 161 to be converted into the first MAC operation result OUT1 .

The comparator 163 is coupled to the analog-to-digital conversion unit 161 for comparing the first MAC operation result OUT1 with a trigger reference value. When "Hybrid accumulation" is selected, when the first MAC operation result OUT1 is lower than the trigger reference value, the comparator 163 does not output a trigger signal TS to the digital accumulation circuit 135 (that is, the digital accumulation circuit 135 does not output a trigger signal TS to the digital accumulation circuit 135). will be triggered); and, when the first MAC operation result OUT1 is higher than the trigger reference value, the comparator 163 outputs the trigger signal TS to the digital accumulation circuit 135 to trigger the digital accumulation circuit 135 to perform digital accumulation. When the “analog accumulation” is selected, the trigger signal TS output by the comparator 163 will be ignored by the digital accumulation circuit 135 .

In an embodiment of the present application, the "analog accumulation" can be used to quickly filter out useless data, so as to speed up the operation speed of the MAC. The "digital accumulation" can accumulate the data that is not filtered out to increase the accuracy of the MAC. The "hybrid accumulation" uses low-resolution quantization operations, which can reduce the variation influence. In addition, it can avoid the accumulation of useless data and maintain the resolution. That is, "hybrid accumulation" takes into account the advantages of "analog accumulation" and "hybrid accumulation", while reducing its disadvantages.

The grouping circuit 140 is coupled to the multiplying circuit 120 . The grouping circuit 140 includes a plurality of grouping units 141 . The grouping units 141 perform grouping operations on the multiplication results of the unit element multiplication units 121 to obtain a plurality of grouping results. In a possible embodiment of the present case, the grouping operation can be implemented by, for example, a majority technique, such as a majority function technique, and the grouping circuit 140 is performed by a majority grouping circuit according to the majority function technique. circuit), the grouping unit 141 is implemented by a decentralized majority grouping unit, but the present case is not limited to this. The clustering technique can be implemented by other similar techniques. In an embodiment of the present application, the grouping circuit 140 can be selectively provided.

The counting unit 150 is coupled to the grouping circuit 140 or the multiplying circuit 120 . In an embodiment of the present application, the counting unit 150 is used to perform bitwise counting or bitwise accumulation on the multiplication result of the multiplication circuit 120 to generate the second MAC operation result OUT2 (when the memory device 100 when the grouping circuit 140 is not included). Alternatively, the counting unit 150 is configured to perform bit count or bit accumulation on the grouping result (eg, the majority result) of the grouping circuit 140 to generate the second MAC operation result OUT2 (when the memory device 100 includes the grouping circuit 140 ) ). In an embodiment of the present application, the counting unit 150 may be implemented by a known counting circuit, such as, but not limited to, a ripple counter. In the description of this case, counting and accumulation basically have the same meaning, and counter and accumulator basically have the same meaning.

Specifically, when the determination unit 170 determines that the memory device 100 performs the analog accumulation, the first MAC operation result OUT1 is used as the MAC operation result. When the determination unit 170 determines that the memory device 100 performs digital accumulation, the second MAC operation result OUT2 is used as the MAC operation result. When the determination unit 170 determines that the memory device 100 performs the hybrid accumulation, before the digital accumulation circuit 135 is not triggered by the trigger signal TS, the first MAC operation result OUT1 is used as the MAC operation result; After the trigger signal TS is triggered, the second MAC operation result OUT2 is used as the MAC operation result.

Please refer to FIG. 2 , which shows a schematic diagram of data mapping according to an embodiment of the present application. As shown in FIG. 2, each input data (or each weight value) has an 8-bit element with N dimensions (N is a positive integer) as an example (but it should be understood that this case is not limited to this).

The following is an example of the data mapping of the input data, but it should be known that this case is not limited to this. The following instructions also apply to the data mapping of weight values.

When the input data is represented by binary 8-bit, the input data (or weight value) is divided into the most significant bit (most significant bit, MSB) vector (vector) and the least significant bit (least significant bit, LSB) vector . The most significant bit vector of the 8-bit input data (or weight value) includes 4 bits B7~B4, and the least significant bit vector includes 4 bits B3~B0.

The bits of the MSB vector and the LSB vector of the input data are represented by unary coding (ie, value format). For example, the bit B7 of the most significant bit vector of the input data can be expressed as B7 ₀ ~B7 ₇ , the bit B6 of the most significant bit vector of the input data can be expressed as B6 ₀ ~B6 ₃ , the most significant bit vector of the input data Bit B5 can be represented as B5 ₀ ~B5 ₁ , and bit B4 of the most significant bit vector of the input data is also represented as B4.

Each bit element of the MSB vector of the input data represented by the unary code (in numerical form) and each bit cell of the LSB vector of the input data are repeated multiple times to be in the form of unfolding dot product (unFDP). For example, the bits of the MSB of the input data are repeated (2 ⁴ -1) times, and similarly, the bits of the LSB of the input data are repeated (2 ⁴ -1) times. In this way, the input data can be expressed in the form of a spread product.

Multiply the input data (in the form of product product) and the weight value to obtain the result of the multiplication operation.

For the convenience of understanding, the following example is used to illustrate, but it should be understood that it is not used to limit this case.

Please refer now to FIG. 3A, which shows an example of one-dimensional data mapping according to an embodiment of the present application. As shown in Figure 3A, the input data=(IN ₁ , IN ₂ )= (2, 1) , and the weight value=(We ₁ , We ₂ )= (1, 2). The MSB and LSB of the input data are expressed in binary form, so IN ₁ =10, and IN ₂ =01, similarly, the MSB and LSB bits of the weight value are expressed in binary form, so, We ₁ = 01, and We ₂ =10.

The MSB and LSB of the input data, as well as the MSB and LSB of the weight value, are encoded to be represented by a unary code (in numerical form). That is, the MSB of the input data is coded as 110, the LSB of the input data is coded as 001, similarly, the MSB of the weight value is coded as 001, and the LSB of the weight value is coded as 110.

After that, the bits of the MSB(110) of the input data encoded as the unary code and the bits of the LSB(001) of the input data of the unary code are repeated many times to become the unfolding dot product (unFDP) form . For example, the bits of the MSB (110) of the input data are repeated three times, so the expanded product form of the MSB of the input data is 111111000. The bits of the LSB(001) of the input data are repeated 3 times, so the LSB of the input data is obtained in the form of the expanded product of 000000111.

The MAC operation is performed on the input data (in the form of the product of expansion) and the weight value to obtain the result of the MAC operation. The result of the MAC operation is: 1*0=0, 1*0=0, 1*1=1, 1*0=0, 1*0=0, 1*1=1, 0*0=0, 0*0 =0, 0*1=0, 0*1=0, 0*1=0, 0*0=0, 0*1=0, 0*1=0, 0*0=0, 1*1=1 , 1*1=1, 1*0=0. Adding these values together gives: 0+0+1+0+0+1+0+0+0+0+0+0+0+0+0+1+1+0=4.

As can be seen from the above, if the input data is i-bit and the weight value is j-bit (both i and j are positive integers), the number of memory cells used is: (2 ⁱ -1)*(2 ^j -1) .

Please refer now to FIG. 3B, which shows another possible example of data mapping according to an embodiment of the present application. In Figure 3B, the input data is (IN ₁ )=(2), and the weight value is (We ₁ )=(1). Input data and weight values are 4 bits.

When the input data is expressed in binary format, IN ₁ =0010. Similarly, when the weight value is expressed in binary format, We ₁ =0001.

Encode the input data and weight values into a unary code (numerical form). For example, the most significant bit "0" of the input data is encoded as "00000000", and the least significant bit "0" of the input data is encoded as "0", and so on. Similarly, the most significant bit "0" of the weight value is encoded as "00000000", and the least significant bit "1" of the weight value is encoded as "1".

The bits of the input data coded into the unary code are copied multiple times to form the spread product. For example, the most significant bit 301A of the input data encoded in the unary encoding is copied 15 times to become the bit 303A; and the least significant bit 301B of the input data encoded in the unary encoding is copied 15 times to become the bit 303B.

The weight value 302 coded into the unary code is also replicated 15 times to represent it in product form.

A multiplication operation is performed on the input data expressed in the form of the spread product and the weight value expressed in the form of the spread product to generate the MAC operation result. Specifically, the bit 303A of the input data is multiplied by the weight value 302; the bit 303B of the input data is multiplied by the weight value 302, and so on. Summing the multiplication values can produce the MAC operation result ("2").

Please refer now to FIG. 3C, which shows another possible example of data mapping according to an embodiment of the present application. In Figure 3C, the input data is (IN ₁ )=(1), and the weight value is (We ₁ )=(5). Input data and weight values are 4 bits.

When the input data is expressed in binary format, IN ₁ =0001. Similarly, when the weight value is expressed in binary format, We ₁ =0101.

Encode the input data and weight values into a unary code (numerical form).

The bits of the input data coded into the unary code are copied multiple times to form the spread product. In Fig. 3C, when duplicating each bit of the input data and each bit of the weight value, a bit "0" is added. For example, the most significant bit 311A of the input data encoded in the unary code is copied 15 times and the bit "0" is added to become the bit 313A; and the least significant bit 311B of the input data encoded in the unary code is copied 15 times and Bit "0" is added to become bit 313B. In this way, the input data is represented in the expanded product form.

Similarly, the weight values 312 encoded as unary codes are also replicated 15 times, and an additional bit of "0" is added to each weight value 314 . Thereby, the weight value is expressed in the product form.

A multiplication operation is performed on the input data expressed in the form of the spread product and the weight value expressed in the form of the spread product to generate the MAC operation result. Specifically, the bit 313A of the input data is multiplied by the weight value 314; the bit 313B of the input data is multiplied by the weight value 314, and so on. Summing the multiplication values can produce the MAC operation result ("5").

In the prior art, the MAC operation is performed on the 8-bit input data and the 8-bit weight value. If the direct MAC algorithm is used, the number of memory cells used is 255*255*512=33,292,822.

On the contrary, as described above, in the embodiment of this case, the MAC operation is performed on the 8-bit input data and the 8-bit weight value, and the number of memory cells used is 15*15*512*2=115,200*2=230,400 . Therefore, the number of memory cells used in the MAC operation in this embodiment is about 0.7% of that of the prior art.

In the embodiment of the present application, the unFDP-type data mapping can be used to reduce the number of memory cells used in the operation, thereby reducing the operation cost and the error correction code (ECC, error correction code) cost. In addition, fail-bit effects can also be tolerated.

Please refer to Figure 1 again. In the embodiment of this case, when performing the multiplication operation, the weight value (transduction value) is stored in the memory cells 111 of the memory array 110 , and the input data (voltage) is read by the I/O circuit 130 Fetch and transmit to common data latch 121D. Common data latch 121D transmits input data to input latch 121A.

For a better understanding of the multiplication operation of the embodiment of the present application, please refer to FIG. 4 , which shows a schematic diagram of an exemplary multiplication operation of the embodiment of the present application. Figure 4 applies to a memory device supporting the selected bit-line read function. In FIG. 4 , the input latch 121A includes a latch (first latch) 405 and a bit line switch 410 .

As shown in Fig. 4, the weight value is represented by a unary code (numerical form) (as shown in Fig. 2). Therefore, the highest bit of the weight value is stored in 8 memory cells 111 , the second highest bit of the weight value is stored in 4 memory cells 111 , and the third highest bit of the weight value is stored in 2 memory cells 111 . , the lowest bit of the weight value is stored in one memory unit 111 .

Similarly, the input data is represented by unary code (in the form of numerical value) (as shown in Figure 2), therefore, the most significant bit of the input data is stored in the eight common data latches 121D, and the second most significant bit of the input data is stored in 4 In the common data latches 121D, the third high-order bit of the input data is stored in the two common data latches 121D, and the least significant bit of the input data is stored in one common data latch 121D. Input data is sent to latch 405 from common data latch 121D.

In FIG. 4 , the plurality of bit line switches 410 are coupled between the memory unit 111 and the sense amplifier 121B. Bit line switch 410 is controlled by latch 405 . For example, when the latch 405 outputs bit 1, the bit line switch 410 is on, and when the latch 405 outputs bit 0, the bit line switch 410 is off.

In addition, when the weight value in the memory cell 111 is bit 1 and the bit line switch 410 is turned on (the input data is bit 1), the sense amplifier 121B will sense the memory cell current to generate the multiplication result "1" . When the weight value in the memory cell 111 is bit 0 and the bit line switch 410 is turned on (the input data is bit 1), the sense amplifier 121B cannot sense the memory cell current. When the weight value in the memory cell 111 is bit 1 and the bit line switch 410 is off (input data is bit 0), the sense amplifier 121B cannot sense the current of the memory cell to generate the multiplication result "0". When the weight value in the memory cell 111 is bit 0 and the bit line switch 410 is off (the input data is bit 0), the sense amplifier 121B cannot sense the memory cell current.

That is, through the layout of FIG. 4, when the input data is bit 1 and the weight value is bit 1, the sense amplifier 121B senses the memory cell current to generate a multiplication result “1”. As for other cases, the sense amplifier 121B cannot sense the current of the memory cell, so that the multiplication result "0" is generated.

The memory cell currents IMC generated by the memory cells 111 are jointly input to the analog-to-digital conversion unit 161 .

As for the relationship between input data, weight value, numerical multiplication result and analog memory cell current IMC, the following table shows: input data Weights result of multiplication IMC 0 0(HVT) 0 0 0 +1 (LVT) 0 0 1 0(HVT) 0 IHVT 1 +1 (LVT) 1 ILVT

In the above table, HVT and LVT represent high threshold memory cells and low threshold memory cells, respectively. The IHVT and ILVT respectively represent the analog memory generated by the high-threshold memory cell and the low-threshold memory cell (with weight values of 0 (HTV) and +1 (LTV), respectively) when the input data is logic 1 Cell current IMC.

In the embodiment of the present application, the selected bit line read (SBL-read) instruction can be used repeatedly during the multiplication operation. Therefore, the embodiment of the present case can reduce the variation influence caused by the single-bit representation.

Please refer to FIG. 5A , which shows a schematic diagram of grouping operation (majority decision operation) and bitwise counting according to an embodiment of the present application. As shown in FIG. 5A, reference symbol GM1 represents the first multiplication result obtained by performing bitwise multiplication on the first MSB vector of the input data and the weight value; reference symbol GM2 represents the second multiplication result of the input data. The second multiplication result obtained by the bitwise multiplication of the MSB vector and the weight value; the reference symbol GM3 represents the third multiplication result obtained by the bitwise multiplication of the third MSB vector of the input data and the weight value; the reference symbol GL represents the fourth multiplication result obtained by performing the bitwise multiplication of the LSB of the input data and the weight value. After the grouping operation (majority decision operation), the result of grouping the first multiplication result GM1 is the first grouping result CB1 (its cumulative weight is 2 ² ); the result of grouping the second multiplication result GM2 is the second grouping The result CB2 (its cumulative weight is 2 ² ); the result of grouping the third multiplication result GM3 is the third grouping result CB3 (its cumulative weight is 2 ² ); and the result of grouping the fourth multiplication result GL is the The quartile results CB4 (its cumulative weight is 2 ⁰ ).

Figure 5B shows a cumulative example of Figure 3C. Please refer to Figure 3C and Figure 5B. As shown in Figure 5B, the bit 313B of the input data (Figure 3C) is multiplied by the weight value 314. The first four bits ("0000") of the multiplication result resulting from multiplying bits 313B of the input data (FIG. 3C) by the weight value 314 are grouped into the first multiplication result "GM1". Similarly, the fifth to eighth bits ("0000") of the multiplication result resulting from multiplying bit 313B of the input data (FIG. 3C) by the weight value 314 are grouped into the second multiplication result "GM2". The ninth to twelfth bits ("1111") of the multiplication result resulting from multiplying bits 313B of the input data (FIG. 3C) by the weight value 314 are grouped into a third multiplication result "GM3". The thirteenth to sixteenth bits ("0010") of the multiplication result generated by multiplying the weight value 314 by the bit 313B of the input data (FIG. 3C) are counted directly.

After the grouping operation (majority decision operation), the first grouping result CB1 is "0" (its cumulative weight is 2 ² ); the second grouping result CB2 is "0" (its cumulative weight is 2 ² ); the third grouping result The result CB3 is "1" (its cumulative weight is 2 ² ). During counting, these clustering results CB1 to CB4 are multiplied by the individual cumulative weights and accumulated to generate the MAC operation result. For example, as shown in FIG. 5B, the MAC operation result (the second MAC operation result OUT2) is CB1*2 ² +CB2*2 ² +CB3*2 ² +CB4*2 ⁰ = 0*2 ² +0*2 ² +1*2 ² +1*2 ⁰ =0000 0000 0000 0000 0000 0000 0000 0101=5.

In an embodiment of this case, the grouping principle (majority rule) can be as follows: group bit Group result (majority result) 1111 (Condition A) 1 1110 (Condition B) 1 1100 (Condition C) 1 or 0 1000 (Condition D) 0 0000 (Condition E) 0

In the above table, in the case of condition A, since the groups are all correct ("1111" has no error bits), the majority result is 1. In case E, since the groups are all correct ("0000" has no error bits), the majority result is 0.

In case B, since 1 bit in the group is wrong ("0" in "1110" is wrong), "1110" can be determined as "1" by majority vote. In case D, since 1 bit in the group is wrong ("1" in "0001" is wrong), "0001" can be determined as "0" by majority vote.

In case C, 2 bits in the group are wrong ("00" in "1100" is wrong, or "11" in "1100" is wrong), by majority decision, it is possible to Determine "1100" as "1" or "0".

Therefore, in the embodiment of the present application, through the grouping (majority decision) function, the erroneous bits can be reduced.

The grouping result of the grouping circuit 140 is input to the counting unit 150 for bit counting.

When the count is performed, the count result of the multiplication operation result of the MSB vector and the count result of the multiplication operation result of the LSB vector are accumulated. In the case of Figure 5A, two types of accumulators are used. The first type of accumulator is to be assigned a higher accumulation weight value (eg 2 ² ). The first accumulator is to accumulate: (1) "The result of grouping (majority decision) of multiplication operation result GM1: 1 bit" plus "grouping (majority decision) of multiplication result GM2" ) of the grouping (majority decision) result: 1 bit" plus "the grouping (majority decision) result of the multiplication result GM3: 1 bit". The count result obtained by the first accumulator is then multiplied by a higher accumulative weight value (eg, 2 ² ). The second type of accumulator is to be assigned a lower accumulation weight value (eg 2 ⁰ ). The second accumulator directly counts the multiplication result GL (multi-bit). The MAC result is obtained by adding the two cumulative results weighted by the cumulative weight. For example, the grouping result obtained by grouping the multiplication result GM1 is 1 (1 bit), the grouping result of grouping the multiplication result GM2 is 0 (1 bit), and the grouping result of grouping the multiplication result GM3 is 1 (1 bit). The count result (1+0+1) obtained by the first accumulator is multiplied by 2 ² , which is equal to 2*2 ² =8. The multiplication result GL is 4 (3 bits), which can be counted directly. Adding the two cumulative results weighted by the cumulative weight, the MAC result is 8+4=12.

As can be seen from the above, in the embodiment of the present case, since the input data has been expanded into the unFDP format, the data stored in the common data latch can be grouped (that is, divided into MSBs) during the counting or accumulation. vector and LSB vector), the error bits in MSB vector/LSB vector can be reduced by grouping mechanism (majority decision mechanism).

In addition, in the present embodiment, even if a conventional accumulator (counter) is used, the counting/accumulating time can still be reduced, because the present embodiment uses a digit count instruction (error bit count), and for different vectors ( The cumulative results of MSB vector and LSB vector) are given different cumulative weights. In one example, the cumulative computing time can be reduced to about 40%.

FIG. 6 shows the MAC operation flow of an embodiment of the present application. In FIG. 6, DMAC represents the first type of digital accumulation (but does not perform grouping operations, that is, the memory device 100 does not include the grouping circuit 140), and mDMAC represents the second type of digital accumulation (performs grouping operations, that is, memory The device 100 includes a grouping circuit 140), AMAC stands for "Analog Accumulation" and HMAC stands for "Hybrid Accumulation".

For the first digital accumulation operation process of the embodiment of the present application, the input data is transmitted to the memory device. Bit line setting and word line setting are performed at the same time. Sensing is performed after the bit line setting is completed. Perform a digital accumulation operation. And return the result of the digital accumulation operation. Repeat the above operation until all input data have been calculated.

For the second digital accumulation operation flow of the embodiment of the present application, the operation speed of the digital accumulation can be accelerated through the grouping operation.

According to the analog accumulation operation flow of the embodiment of the present application, when sensing is performed, ADC conversion and comparison operations can be completed at the same time, so the MAC operation can be further improved.

In terms of the hybrid accumulation operation flow of the embodiment of the present application, since the analog accumulation and digital accumulation are performed, the operation speed of the hybrid accumulation is slower than that of the analog accumulation but faster than that of the digital accumulation. However, hybrid accumulation can be nearly as accurate as digital accumulation and higher than analog accumulation.

It can be seen from FIG. 6 that the MAC operation in the embodiment of this case can be divided into two sub-operation types. The first sub-operation type is the multiplication operation, which multiplies the input data by the weight value according to the selected bit line read command. The second type of sub-operation is accumulation (data counting), especially fail bit counting. In other possible embodiments of this case, more counting units may be added to speed up the counting/accumulation operation.

The operation time of the digital accumulation mainly depends on the accumulation speed of the counting unit 150 because the counting unit 150 calculates bit by bit. The quantization accuracy of the analog-to-digital conversion unit 161 mainly depends on the variation tolerance of the memory cells. Therefore, digital accumulation has higher accuracy but lower accumulation speed than analog accumulation.

In addition, in this embodiment, the read voltage can also be adjusted. FIG. 7A shows a flowchart of programming a fixed memory page in the embodiment of the present invention, and FIG. 7B shows a flowchart of adjusting the read voltage in the embodiment of the present invention.

In FIG. 7A, in step 710, a known input data is programmed into a fixed memory page. For example and without limitation, the bit ratio of the known input data is: 75% is bit 0 and 25% is bit 1.

In FIG. 7B, in step 720, the fixed memory page is read and the ADC is enabled. In step 730, it is determined whether the output value of the ADC is close to the reference test value (if the ADC is 8 bits, the reference test value is 127, but this case is not limited to this). If step 730 is no, the flow proceeds to step 740 . If step 730 is YES, the flow proceeds to step 750 .

In step 740, if the output value of the ADC is less than the reference test value, the read voltage is increased; and if the output value of the ADC is greater than the reference test value, the read voltage is decreased. After step 740 ends, the flow returns to step 720 .

In step 750, the current read voltage is recorded for use in subsequent read operations.

As is known, reading the voltage will affect the ADC output value and the reading of bit 1. Therefore, in the present embodiment, the read voltage can be periodically corrected according to operating conditions (such as, but not limited to, programming cycle, temperature, or read disturbance, etc.) to maintain high accuracy and reliability.

FIG. 8 shows a MAC operation flow according to an embodiment of the present application. In step 810, a plurality of weight values are stored in a plurality of memory cells of a memory array of the memory device. In step 820, bitwise multiplication is performed on the plurality of input data and the weight values to obtain a plurality of multiplication results, wherein the memory cells generate a plurality of memory cell currents during the multiplication. In step 830, it is determined to perform an analog accumulation, a digital accumulation or a hybrid accumulation. In step 840, when performing the analog accumulation, analog accumulation is performed on the memory cell currents to obtain a first multiply-accumulate (MAC) operation result. In step 850, when the digital accumulation is performed, the multiplication results are digitally accumulated to obtain a second multiply-accumulate (MAC) operation result. In step 860, when the hybrid accumulation is performed, whether to trigger the digital accumulation is determined according to the result of the first multiply-accumulate operation.

The embodiments of this case can be applied to NAND-type flash memory, or memory devices that are sensitive to retention and thermal changes, such as, but not limited to, NOR-type flash memory, phase-change (PCM)-type flash memory, Magnetic random access memory (magnetic RAM) or resistive RAM.

The embodiments of this case can be applied to 3D memory and 2D memory, such as, but not limited to, 2D/3D NAND flash memory, 2D/3D NOR flash memory, 2D/3D phase change (PCM) ) type flash memory, 2D/3D magnetic random access memory (magnetic RAM) or 2D/3D resistive RAM.

Although in the above-mentioned embodiment, the input data and/or the weight value are divided into MSB vector and LSB vector (two vectors), but the present case is not limited to this. In other possible embodiments of this case, the input data and/or weight values can also be divided into more vectors, which are also within the scope of the spirit of this case.

In the embodiment of this case, not only the majority decision grouping technique but also other grouping techniques can be applied to accelerate the accumulation.

The embodiments of this case can be applied to, for example, but not limited to, AI technologies such as face recognition.

In this embodiment, the analog-to-digital conversion unit 161 may be a current-mode analog-to-digital conversion unit, a voltage-mode analog-to-digital conversion unit, or a mixed-mode analog-to-digital conversion unit.

The embodiment of this case can be applied not only to the serial MAC operation, but also to the parallel MAC operation.

To sum up, although the present invention has been disclosed by the above embodiments, it is not intended to limit the present invention. Those skilled in the art to which the present invention pertains can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be determined by the scope of the appended patent application.

100: Memory device 110: Memory array 120: Multiplication circuit 130: I/O circuit 140: Grouping circuit 150: counting unit 111: Memory unit 121: Identity Multiplication Unit 121A: Input Latch 121B: Sense Amplifier 121C: Output Latch 121D: Common data latch 141: Grouping unit 135: Digital accumulator circuit 160: Analogue accumulator circuit 170: Decision Unit 161: Analog-to-digital conversion unit 163: Comparator 301A, 303A, 301B, 303B, 311A, 313A, 311B, 313B: Bit 302, 312, 314: Weight value 405: Latch 410: Bit line switch 710-750: Procedure 810-860: Procedure

FIG. 1 is a functional block diagram of a memory device with an in-memory computing function according to an embodiment of the present application. FIG. 2 shows a schematic diagram of data mapping according to an embodiment of the present application. Figures 3A to 3C show several examples of data mapping according to an embodiment of the present application. FIG. 4 shows a schematic diagram of two exemplary examples of the multiplication operation in the embodiment of the present invention. 5A and 5B show schematic diagrams of grouping operation (majority decision operation) and counting according to an embodiment of the present invention. FIG. 6 shows a comparison of several MAC operation flows of an embodiment of the present application. FIG. 7A shows a flowchart of programming a fixed memory page in the embodiment of the present invention, and FIG. 7B shows a flowchart of adjusting the read voltage in the embodiment of the present invention. FIG. 8 shows a MAC operation flow according to an embodiment of the present application.

810-860: Procedure

Claims (10)

A memory device comprising: a memory array including a plurality of memory cells for storing a plurality of weight values in the memory cells of the memory array; a multiplication circuit, coupled to the memory array, the multiplication circuit multiplies a plurality of input data and the weight values to obtain a plurality of multiplication results, wherein when the multiplication is performed, the memory cells generate a plurality of memories body current; a digital accumulating circuit, coupled to the multiplying circuit, and performing a digital accumulating on the multiplication results; an analog accumulation circuit, coupled to the memory array, performs analog accumulation on the memory cell currents to generate a first multiply-accumulate (MAC) operation result; and a determining unit, coupled to the digital accumulating circuit and the analog accumulating circuit, determines to perform the analog accumulating, the digital accumulating or a hybrid accumulating, Wherein, when the hybrid accumulation is performed, whether to trigger the digital accumulation circuit is determined according to the operation result of the first multiply-accumulate operation. The memory device of claim 1, wherein, When performing the analog accumulation, the determination unit enables the analog accumulation circuit but not the digital accumulation circuit; When performing the digital accumulation, the determination unit enables the digital accumulation circuit but disables the analog accumulation circuit; and When performing the hybrid accumulation, the determination unit enables the digital accumulation circuit and the analog accumulation circuit. The memory device of claim 1, further comprising: a comparator, coupled to the analog accumulating circuit and the digital accumulating circuit, compares the first MAC operation result with a trigger reference value to output a trigger signal to the digital accumulating circuit to trigger the digital accumulating circuit Do this digital accumulation, Wherein, the analog accumulation circuit includes an analog-digital conversion unit coupled to the memory array, and the memory cell currents of the memory cells are accumulated and input to the analog-digital conversion unit for conversion into the first analog-digital conversion unit MAC operation result. The memory device of claim 1, wherein the digital accumulating circuit comprises: A counting unit, coupled to the multiplication circuit, performs bit counting on the multiplication results to obtain a second MAC operation result. The memory device of claim 4, further comprising a grouping circuit coupled to the multiplying circuit and the counting unit, the grouping circuit performs a grouping operation on the multiplication results of the multiplying circuit to obtain a plurality of grouping results , and input the grouping results into the counting unit. A memory device as claimed in claim 1, the plurality of bits of each of the input data or each of the weight values is divided into a plurality of bit vectors; Convert the bits of these bit vectors from a binary form to a unary code representation; repeating the bits of the bit vectors represented by the unary code a plurality of times to form a product product; and The multiplication circuit multiplies the input data in the spread product form and the weight values in the spread product form to obtain the multiplication results. The memory device of claim 5, wherein, When performing the grouping operation on the multiplication results, the grouping circuit respectively performs the grouping operation on the plurality of multiplication results of the vectors to obtain the grouping results; When performing bit counting, different accumulation weights are given to the grouping results and then accumulated to obtain the second multiplication accumulation result; and The grouping circuit is a majority circuit, including a plurality of majority units. A method of operating a memory device, comprising: storing a plurality of weight values in a plurality of memory cells of a memory array of the memory device; performing a bitwise multiplication on a plurality of input data and the weight values to obtain a plurality of multiplication results, wherein the memory cells generate a plurality of memory cell currents during the multiplication; and Decide to perform an analog accumulation, a digital accumulation, or a hybrid accumulation, in, When performing the analog accumulation, the analog accumulation is performed on the memory cell currents to generate a first multiply-accumulate (MAC) operation result; When performing the digital accumulation, performing the digital accumulation on the multiplication results to generate a second multiply-accumulate operation result; and When the hybrid accumulation is performed, whether to trigger the digital accumulation is determined according to the operation result of the first multiply-accumulate operation. The operation method of a memory device according to claim 8, wherein the memory cell currents of the memory cells are accumulated and then converted into the first MAC operation result by analog-digital conversion; and The first MAC operation result is compared with a trigger reference value to determine whether to trigger the digital accumulation. The operating method of a memory device as claimed in claim 8, wherein, dividing each of the input data or each of the weight values into a plurality of bit vectors; Convert the bits of these bit vectors from a binary form to a unary code representation; repeating the bits of the bit vectors represented by the unary code a plurality of times to form a product product; and performing a multiplication operation on the input data in the extended product form and the weight values in the extended product form to obtain the multiplication operation results; When performing bit accumulation, different accumulation weights are given to the grouping results and then accumulation is performed to obtain the second multiplication accumulation operation result; and Performing a grouping operation on the multiplication results performs a majority operation on the multiplication results.

Images