TWI798798B - In-memory computing method and in-memory computing apparatus - Google Patents

Abstract

An in-memory computing method and an in-memory computing apparatus, adapted for a processor to perform MAC operations on a memory, are provided. In the method, a preprocessing is respectively performed on an input data and a weight data to be written to input lines and memory cells of the memory to divide the input data and the weight data into a primary portion and a secondary portion. Next, the input data and the weight data divided into the primary portion and the secondary portion are written into the input lines and the memory cells in batches to perform the MAC operations and obtain multiple calculation results. Then, the calculation results are filtered according to a numeral value of each calculation result. Finally, a post-processing is performed on the filtered calculation results according to the portions to which the calculation results correspond, and accordingly obtain output data.

Description

In-memory computing method and device

The present disclosure relates to a computing method and device, and more particularly to an in-memory computing method and device.

When a traditional computing system executes a data-intensive application, it needs to perform a large number of calculations and frequently move data between the processor and the memory. Among them, performing a large number of calculations will result in reduced system performance, and a large amount of data movement will result in high power consumption.

In order to solve the above problems of performance limitation and power consumption, new algorithms and/or memory architectures have been proposed in recent years, including Nearest Neighbor Search, Decision tree learning, distributed system , In-memory computing, etc. However, decision tree learning still requires a large amount of data transfer, distributed systems have problems of high cost and communication between devices, and in-memory computing cannot support complex operations.

In view of the above, the present disclosure provides an in-memory computing method and an in-memory computing device capable of improving the performance of a computing system.

The disclosure provides an in-memory operation method, which is suitable for a processor to perform a multiply-add (MAC) operation on a memory. The memory includes a plurality of input lines and a plurality of output lines intersecting with each other, a plurality of memory units respectively arranged at intersection points of the input lines and output lines, and a plurality of sense amplifiers respectively connected to the output lines. The method includes the following steps: pre-processing the input data and weight data to be written into the input line and the memory unit respectively, so as to distinguish them into main parts and secondary parts; The weight data is written into the input line and the memory unit in batches for multiplication and addition operations to obtain multiple operation results; the operation results are filtered out according to the numerical value of each operation result; The calculation results are post-processed to obtain output data.

In an embodiment of the present disclosure, the step of filtering the calculation results according to the numerical value of each calculation result includes filtering the calculation results whose numerical value is not greater than the preset threshold value, sorting the filtered calculation results, and selecting Sort the results of at least one previous operation for postprocessing.

In an embodiment of the present disclosure, the method further includes encoding the input data and weight data when pre-processing the input data and weight data, and encoding As a result of the operation, a weighting operation corresponding to encoding is performed.

In an embodiment of the present disclosure, the step of performing a weighting operation corresponding to the encoding on the operation result includes multiplying the operation result by a first weight in response to the operation result corresponding to the main part of the input data and the main part of the weight data To obtain a first product, in response to the result of the operation corresponding to the major part of the input data and the minor part of the weighted data, multiply the result of the operation by a second weight to obtain a second product, in response to the result of the operation corresponding to the minor part of the input data The main part of the part and weight data, the operation result is multiplied by the third weight to obtain the third product, and the result of the operation is multiplied by the fourth weight in response to the operation result corresponding to the sub part of the input data and the sub part of the weight data Obtaining the fourth product, accumulating the first product, the second product, the third product and the fourth product obtained by weighting the operation results, and outputting the accumulated result as output data.

The disclosure provides an in-memory computing device, which includes a memory and a processor. The memory includes a plurality of input lines and a plurality of output lines intersecting each other, a plurality of units respectively arranged at intersection points of the input lines and output lines, and a plurality of sense amplifiers respectively connected to the output lines. The processor is coupled to the memory and is configured to pre-process the input data and weight data to be written into the input line and the memory unit, respectively, to be divided into primary and secondary Part of the input data and weight data are written into the input line and memory unit in batches to perform multiplication and addition operations, and the sensed values of the sense amplifier are accumulated to obtain multiple operation results, and are filtered according to the numerical value of each operation result The operation result is removed, and the filtered operation result is post-processed according to the corresponding part of the operation result to obtain output data.

In an embodiment of the present disclosure, the main part is the most significant bit (Most significant bit, MSB) of the multi-bits of the processed data, and the secondary part is the least significant bit of the multi-bits of the processed data Effective bit (Least significant bit, LSB).

In an embodiment of the present disclosure, the in-memory computing device further includes a filter for filtering out computing results whose values are not greater than a preset threshold, wherein the processor includes sorting the filtered computing results and selecting Sort the results of at least one previous operation for postprocessing.

In an embodiment of the present disclosure, the processor includes encoding the input data and the weight data when performing pre-processing on the input data and weight data, and encoding the input data and weight data when performing post-processing on the filtered calculation results As a result of the operation, a weighting operation corresponding to encoding is performed.

In an embodiment of the present disclosure, the processor includes a main part corresponding to the input data and a main part of the weight data in response to the operation result, and the operation result is multiplied by the first weight to obtain the first product, which is reflected in the operation result Corresponding to the main part of the input data and the secondary part of the weight data, the operation result is multiplied by the second weight to obtain the second product, in response to the fact that the operation result corresponds to the secondary part of the input data and the main part of the weight data, for the operation The result is multiplied by a third weight to obtain a third product, in response to the result of the operation corresponding to the subsection of the input data and the subsection of the weighted data, the result of the operation is multiplied by a fourth weight to obtain a fourth product, and the cumulative pair operation The first product, the second product, the third product and the fourth product obtained by performing weighting operations on the results, and outputting the accumulated results as output data.

In order to make the aforementioned features and advantages of the present disclosure more comprehensible, the following describes the embodiments of the accompanying drawings in detail.

FIG. 1 is a schematic diagram of an in-memory computing device according to an embodiment of the disclosure. Please refer to FIG. 1 , the in-memory computing device 10 of the present embodiment is, for example, a memristor configured to implement process-in-memory (PIM), which is suitable for data-intensive applications such as face search. application. The computing device 10 includes a memory 12 and a processor 14, and its functions are described as follows:

Memory 12 is, for example, NAND flash memory, NOR flash memory, phase change memory (phase change memory; PCM), spin transfer torque random access memory (spin -transfer torque random-access memory; STT-RAM), or 2D or 3D structured resistive random-access memory (ReRAM), which is not limited herein. In some embodiments, various volatile memories such as static random access memory (static random access memory; SRAM), dynamic random access memory (dynamic random access memory; DRAM), and various non-volatile memories can be integrated. Memory, such as ReRAM, PCM, Flash, magnetoresistive RAM, ferroelectric RAM, for in-memory operations, without limitation herein.

The memory 12 includes a plurality of input lines IL _i and a plurality of output lines OL _j intersecting each other, and a plurality of memory cells (represented by resistors R _ij ) respectively arranged at intersections of the input lines IL _i and output lines OL _j , And a plurality of sense amplifiers SA respectively connected to the output line OL _j for sensing the current I _j output from the output line OL _j . In some embodiments, input line IL _i is a word line and output line OL _j is a bit line, and in some embodiments, input line IL _i is a bit line and output line OL _j is a word line, where Not limited herein.

The processor 14 is, for example, a central processing unit (central processing unit; CPU) or other programmable general-purpose or special-purpose microprocessors, microcontrollers (microcontroller; MCU), programmable controllers, application-specific integrated circuits (application specific integrated circuit (ASIC), programmable logic device (programmable logic device; PLD) or other similar devices or a combination of these devices, which is not limited in this embodiment. In this embodiment, the processor 14 is configured to execute instructions for performing in-memory operations. The described in-memory operations can be implemented into various artificial intelligence (AI) applications, such as fully connected layers, convolution layers, multi-layer perception, support vector machines, or other applications implemented using memristors, here Not limited herein.

FIG. 2 is a flowchart of an in-memory computing method according to an embodiment of the disclosure. Please refer to FIG. 1 and FIG. 2 , the method of this embodiment is suitable for the above-mentioned in-memory computing device 10 , and the detailed steps of the in-memory computing method of this embodiment will be described below with reference to various devices and components of the in-memory computing device 10 .

First, in step S202, the processor 14 performs pre-processing on the input data and weight data to be written into the input line and the memory unit respectively, so as to distinguish them into main parts and secondary parts. In one embodiment, the processor 14 distinguishes the input data into a multi-bit most significant bit (Most significant bit, MSB) and a multi-bit least significant bit (Least significant bit, LSB), and the weight The data area is divided into multi-bit MSB and multi-bit LSB. In the case that the input data is 8 bits, the processor 14, for example, divides the input data into 4-bit MSB and 4-bit LSB, and divides the weight data into 4-bit MSB and 4-bit LSB . In other cases, the processor 14 can distinguish the input data and the weight data into the same number or different numbers of one or more bits of MSB and one or more bits of LSB according to implementation requirements, and this embodiment does not set limit. In other embodiments, the processor 14 may also mask or filter out one or more unimportant bits (ie, secondary parts) in the input data, and only keep the more important bits (ie, the main part) for subsequent operations, and this embodiment is not limited.

In other embodiments, the processor 14 may further encode the input data and weight data, for example, convert the multi-bit MSB and multi-bit LSB of the input data or weight data from a binary format to The value format of the unary code. The processor 14 can then copy the converted unary code to unfold (unfold) into a dot product format.

For example, FIG. 3 is an example of data encoding according to an embodiment of the disclosure. Please refer to FIG. 3 . In this embodiment, it is assumed that there are N-dimensional input data and weight data to be written, wherein N is a positive integer, and each data has 8 bits B0~B7 expressed in binary. Taking N-dimensional input data <1>~<N> as an example, in this embodiment, each input data <1>~<N> is divided into MSB vector and LSB vector, wherein the MSB vector includes 4 bits MSB B7~B4, The LSB vector includes 4 bits LSB B3~B0. Next, convert each bit of the MSB vector and LSB vector into a unary code according to the value, for example, convert bit B7 into bit B7 ₀ ~B7 ₇ , convert bit B6 into B6 ₀ ~B6 ₃ , convert bit B5 is converted to B5 ₀ ~B5 ₁ , and bit B4 remains unchanged. Then, copy the converted unary code to expand into the inner product format, for example, copy (2 ⁴ -1) converted unary codes of the MSB vector of each input data (2 ⁴ -1) times to expand into 225 bits to generate data in the unfolding dot product (unFDP) format shown in FIG. 3 . Similarly, the weight data can also be pre-processed in the encoding manner of the above-mentioned input data, which will not be repeated here.

Returning to the flow chart of FIG. 2, in step S204, the processor 14 will divide the input data and weight data into the main part and the secondary part, and write them into the input lines and memory units in batches to perform multiplication and addition operations to obtain multiple Operation result. In detail, the processor 14, for example, writes the weight data classified as the main part into the corresponding memory unit in the memory 12, and inputs the input data classified as the main part into the corresponding input line IL _i in the memory 12. , to be obtained by sensing the current I _j output from the output line OL _j by the sense amplifier SA connected to each output line OL _j , thereby accumulating the sensed value of the sense amplifier SA via a counter (counter) or an accumulator (accumulator) The operation result of the multiplication and addition operation of the input data and weight data. Similarly, the processor 14, for example, writes the weight data classified as the main part into the corresponding memory unit in the memory 12, and inputs the input data classified as the secondary part into the corresponding input line IL _i in the memory 12. , to obtain the operation result of the multiplication and addition operation; write the weight data that is divided into the secondary part into the corresponding memory unit in the memory 12, and input the input data that is divided into the main part into the corresponding input line in the memory 12 IL _i , to obtain the operation result of the multiplication and addition operation; The corresponding input line IL _i is used to obtain the operation result of the multiplication and addition operation.

In some embodiments, the memory 12 can also support operations such as Inverse, Logical And (AND), Logical Or (OR), Exclusive Or (XOR), Exclusive Negative Or (XNOR), etc., and are not limited to Multiply and add operations. In addition, the memory 12 is not limited to be implemented by using digital circuits, but can be implemented by using analog circuits, and this embodiment does not limit its implementation.

For example, in a digital circuit, the processor 14 can distinguish the input data into multi-bit MSB and multi-bit LSB (the number of bits is not limited), and perform different encoding (that is, pre-processing) methods After processing, it is sent to the memory 12 for inversion, logical and, logical or, mutual exclusive or, mutual exclusive inverse or multiplication and addition operations or a combination of the above operations, and finally after corresponding post-processing and filtering, you can get The final calculation result. In the analog circuit, the processor 14 can mask or filter out some bits in the input data (that is, pre-processing), and then send it to the memory 12 for inversion, logical and, logical or, exclusive or, Mutual exclusion or multiplication and addition operations or a combination of the above operations, and finally after corresponding post-processing and filtering, the final operation result can be obtained. The above is just an example, and the processor 14 can implement any kind of pre-processing and post-processing on the input data to obtain exclusive calculation results.

In step S206, the processor 14 filters out the calculation results according to the numerical value of each calculation result. In one embodiment, the in-memory computing device 10 includes, for example, a filter (not shown) for filtering out computation results whose values are not greater than a preset threshold. The processor 14 will sort the filtered calculation results, and select the top N calculation results for post-processing, and the N is, for example, 3, 5, 10, 20 or any positive integer. Set limits.

In step S208, the processor 14 performs post-processing on the filtered operation result according to the corresponding portion of the operation result to obtain output data. In one embodiment, when the processor 14 pre-processes the input data and the weight data, for example, encodes the input data and the weight data; As a result, a weighting operation corresponding to the encoding is performed.

In detail, the processor 14 multiplies the operation result by the first weight to obtain the first product in response to the operation result corresponding to the main part of the input data and the main part of the weight data; in response to the operation result corresponding to the input data The main part and the secondary part of the weight data, the processor 14 will multiply the operation result by the second weight to obtain the second product; in response to the operation result corresponding to the secondary part of the input data and the main part of the weight data, the processor 14 The operation result is multiplied by a third weight to obtain a third product; in response to the operation result corresponding to the sub-part of the input data and the sub-part of the weight data, the processor 14 multiplies the operation result by a fourth weight to obtain the first Four times. Finally, the processor 14 accumulates the first product, the second product, the third product and the fourth product obtained by weighting the above operation result, so as to output the accumulated result as output data.

For example, FIG. 4 is an example of data post-processing according to an embodiment of the disclosure. Referring to FIG. 4 , this embodiment illustrates the post-processing corresponding to the encoding method in FIG. 3 . Among them, it reflects that the operation result corresponds to the main part of the input data (ie, MSB) and the main part of the weight data, and the corresponding weight value is 16*16; it reflects that the operation result corresponds to the main part of the input data and the weight data The secondary part (ie, LSB), its corresponding weight value is 16*1; it reflects the main part of the input data and the main part of the weight data corresponding to the operation result, and its corresponding weight value is 1*16; it reflects the operation The result corresponds to the main part of the input data and the main part of the weight data, and the corresponding weight value is 1*1. By multiplying the operation results of the input data and weight data written in batches into the memory 12 by the corresponding weight value, the operation result of the multiplication and addition operation of the original input data and weight data can be restored.

After the multiplication and addition operation of each input data and weight data is completed and the operation result is obtained, the processor 14 will return to step S204, and continue to write the next input data and weight data into the memory 12 to perform the multiplication and addition operation until the completion The operation results of all input data and weight data are completed in memory.

In summary, the in-memory computing method and device of the embodiments of the present disclosure combine in-memory computing and a hierarchical filtering scheme to selectively The operations on the bits with a lower proportion of the data value (ie, LSB) are effectively deleted, and the operations on the bits with a higher proportion (ie, MSB) are prioritized, and by filtering the operation results, And select the operation result with higher value to carry out the corresponding data post-processing, and finally obtain the output data. In this way, the performance of the computing system can be improved without excessively affecting the value of the computing result.

Although the present disclosure has been disclosed by the above-mentioned embodiments, the embodiments are not intended to limit the present disclosure. It will be apparent to those having ordinary skill in the art that various modifications and changes can be made in the structures of the present disclosure without departing from the scope or spirit of the present disclosure. Therefore, the scope of protection of the present disclosure falls within the patent scope of the appended application.

10: computing device 12: memory 14: processor B7~B0, _B70 ~ _B77 , _B60 ~ _B63 : bit IL _i : input line OL _j : output line R _ij : resistance S202~S208: step SA : sense amplifier

FIG. 1 is a schematic diagram of an in-memory computing device according to an embodiment of the disclosure. FIG. 2 is a flowchart of an in-memory computing method according to an embodiment of the disclosure. FIG. 3 is an example of data encoding according to an embodiment of the disclosure. FIG. 4 is an example of data post-processing according to an embodiment of the disclosure.

S202~S208: steps

Claims (10)

An operation method in a memory is suitable for a processor to use a memory to perform a multiply-add (MAC) operation, wherein the memory includes a plurality of input lines and a plurality of output lines crossing each other, respectively configured on the input lines and the A plurality of memory units at the intersection points of the output lines, and a plurality of sense amplifiers respectively connected to the output lines, the method includes: input data and weights to be written into the input lines and the memory units The data are respectively pre-processed to be divided into main parts and secondary parts; the input data and the weight data that are divided into the main parts and the secondary parts are written into the input lines and the weight data in batches. The memory unit is used to perform the multiplication and addition operation to obtain multiple operation results, including writing the weight data of the main part into the corresponding memory unit and inputting the input data of the main part corresponding to the input line, write the weight data of the main part into the corresponding memory unit and input the input data of the secondary part into the corresponding input line, write the secondary Writing part of the weight data into the corresponding memory unit and inputting the input data of the main part into the corresponding input line, and writing the weight data of the secondary part into the corresponding the memory unit and input the input data of the secondary part into the corresponding input line, so as to obtain the operation result of the multiplication and addition operation of the input data and the weight data; according to each of the operation results filtering out the operation result; and performing post-processing on the filtered out operation result according to the part corresponding to the operation result to obtain output data. The operation method in memory as described in claim item 1, wherein the main The important part is the most significant bit (Most significant bit, MSB) of the processed data, and the secondary part is the least significant bit (Least significant bit, LSB) of the processed data. The in-memory operation method according to claim 1, wherein the step of filtering out the operation results according to the numerical value of each of the operation results includes: filtering out the operation results whose numerical values are not greater than a preset threshold; and sorting the filtered operation results, and selecting at least one operation result that is ranked first to perform the post-processing. The calculation method in memory as described in claim 1, further comprising: when performing the preprocessing on the input data and the weight data, encoding the input data and the weight data; and encoding the input data and the weight data; When the filtered operation result is subjected to the post-processing, a weighting operation corresponding to the encoding is performed on the operation result. The in-memory calculation method according to claim 4, wherein the step of performing the weighting calculation corresponding to the code on the calculation result comprises: responding to the fact that the calculation result corresponds to the main part of the input data and the main part of the weight data, multiplying the operation result by a first weight to obtain a first product; in response to the fact that the operation result corresponds to the main part of the input data and the weight data the secondary portion, multiplying the operation result by a second weight to obtain a second product; responsive to the operation result corresponding to the secondary portion of the input data and the primary portion of the weighted data , multiplying the operation result by a third weight to obtain a third product; In response to the operation result corresponding to the sub-portion of the input data and the sub-portion of the weight data, multiplying the operation result by a fourth weight to obtain a fourth product; The first product, the second product, the third product, and the fourth product obtained by performing the weighting operation on the operation result, and output the accumulation result as the output data. An in-memory computing device, comprising: a memory, including: a plurality of input lines and a plurality of output lines intersecting each other; a plurality of memory units respectively arranged at intersections of the input lines and the output lines; and a plurality of a sense amplifier, respectively connected to the output lines; a processor, coupled to the memory and configured to: perform pre-processing on input data and weight data to be written into the input lines and the memory unit, respectively , to be divided into a main part and a secondary part; the input data and the weight data that are divided into the main part and the secondary part are written into the input line and the memory unit in batches to performing the multiply-accumulate operation, and accumulating the sensing values of the sense amplifier to obtain multiple operation results, including writing the weight data of the main part into the corresponding memory unit and writing the main Input part of the input data into the corresponding input line, write the weight data of the main part into the corresponding memory unit and input the input data of the secondary part into the corresponding input line, write the weight data of the secondary part into the corresponding memory unit and input the input data of the main part into the corresponding input line, and write the secondary part Write the weight data of the sub-part into the corresponding memory unit and input the input data of the secondary part into the corresponding input line, so as to obtain the multiplication and addition of the input data and the weight data the operation result of the operation; filtering the operation result according to the numerical value of each operation result; and performing post-processing on the filtered operation result according to the corresponding part of the operation result to obtain output data. The in-memory computing device as claimed in claim 6, wherein the main part is the most significant bit of the multi-bits of the processed data, and the secondary part is the least significant bit of the multi-bits of the processed data . The in-memory computing device as described in claim 6 further includes: a filter for filtering out the operation results whose numerical values are not greater than a preset threshold value, wherein the processor includes sorting the filtered out operation results result, and select at least one operation result that is ranked first to perform the post-processing. The in-memory computing device according to claim 6, wherein the processor includes encoding the input data and the weight data when performing the preprocessing on the input data and the weight data, and When the post-processing is performed on the filtered operation result, a weighting operation corresponding to the encoding is performed on the operation result. The in-memory computing device according to claim 9, wherein the processor includes: responding to the operation result corresponding to the main part of the input data and the main part of the weight data, for the multiplying the operation result by a first weight to obtain a first product; in response to the fact that the operation result corresponds to the main part of the input data and multiplying the operation result by a second weight on the sub-portion of the weight data to obtain a second product; responsive to the operation result corresponding to the sub-portion of the input data and the weight data The main part of the operation result is multiplied by a third weight to obtain a third product; in response to the operation result corresponding to the sub part of the input data and the sub part of the weight data part, multiplying the operation result by a fourth weight to obtain a fourth product; and accumulating the first product, the second product, the third product and the weighted operation on the operation result the fourth product, and output the accumulation result as the output data.

Images