TWM614073U - Processing device for executing convolution neural network computation - Google Patents

Abstract

A processing device for executing convolution neural network computation is provided. The convolutional neural network computation include a plurality of convolutional layers. A processing device includes an internal memory and a calculation circuit. The calculation circuit executes convolution computation of each convolution layer and analyzes a physiological feature image sensed by a physiological feature sensing device. The internal memory is coupled to the calculation circuit, includes a plurality of memory cells, and is configured to store weight data of the convolutional layers. Each of the memory cells includes a control circuit and a capacitor. The control circuit has a leakage current path. Data retention time of Each of the memory cells is determined according to a leakage current on the leakage current path and a capacitance value of the capacitor.

Description

Processing device for performing convolutional neural network operations

The present invention relates to a computing device, and in particular to a processing device for performing convolutional neural network operations.

Artificial intelligence has developed rapidly in recent years, which has greatly affected people's lives. Based on artificial neural networks, especially convolutional neural networks (Convolutional Neural Network, CNN), the development of many applications is becoming more and more mature, for example, it is widely used in the field of computer vision. As the application of convolutional neural networks becomes more and more widespread, more and more chip design manufacturers begin to design processing chips for performing convolutional neural network operations. The processing chip that performs convolutional neural network operations requires complex operations and a huge amount of parameters to analyze the input data. For the processing chip used to perform convolutional neural network operations, in order to accelerate the processing speed and reduce the power consumption caused by repeated access to the external memory, the processing chip is generally equipped with internal memory (also known as the built-in chip). Memory (on-chip-memory)) to store temporary calculation results and weight data required for convolution operations. Generally speaking, this internal memory generally uses static random access memory (SRAM). However, when the data in the static random access memory is frequently read and written based on the characteristics of the convolutional neural network operation, the overall chip power consumption of the processing chip will increase.

In view of this, the present invention provides a processing device for performing convolutional neural network operations, which can reduce the power consumption and circuit area of the processing device for performing convolutional neural network operations.

The creative embodiment of the present invention proposes a processing device for performing convolutional neural network operations. This convolutional neural network operation includes multiple convolutional layers. The processing device includes internal memory and calculation circuits. The calculation circuit performs the convolution operation of each convolution layer. The internal memory is coupled to the calculation circuit and includes a plurality of memory cells, and is used to store the weight data of the convolutional layer. Each memory cell includes a control circuit and a capacitor. The control circuit has a leakage current path. The data retention time of each memory cell is determined according to the leakage current on the leakage current path and the capacitance value of the capacitor, and the data retention time is greater than the preset required time.

Based on the above, in the embodiment of the present invention, the data retention time of the memory cell of the internal memory is determined based on the leakage current on the leakage current path and the capacitance value of the capacitor, and the data retention time will be greater than the preset required time . In other words, under the condition of ensuring that the weight data in the internal memory is obtained by the calculation circuit, these weight data will only become invalid after being retained in the internal memory for a period of time. Based on this, under the condition that the weight data recorded in the internal memory can become invalid over time, the overall chip power consumption of the processing device including the internal memory can be reduced and the circuit area can be reduced.

In order to make the above-mentioned features and advantages of the new creation more obvious and understandable, the following specific examples are given in conjunction with the accompanying drawings to describe in detail as follows.

In order to make the content of the new creation easier to understand, the following specific examples are given as examples on which the new creation can indeed be implemented. In addition, wherever possible, elements/components/steps with the same reference numbers in the drawings and embodiments represent the same or similar components.

It should be understood that when an element is referred to as being "directly on" or "directly connected to" another element, there are no intervening elements. As used herein, "connected" can refer to physical and/or electrical connection. Furthermore, "electrically connected" or "coupled" may mean that there are other elements between two elements.

FIG. 1 is a schematic diagram of a computing system for performing convolutional neural network operations according to an embodiment of the new creation. Please refer to FIG. 1, the computing system 10 can analyze input data based on convolutional neural network operations to extract effective information. The computing system 10 can be installed in various electronic terminal devices to implement various application functions. For example, the computing system 10 can be installed in a smart phone, a tablet computer, a medical device or a robot device, and the present invention is not limited to this. In one embodiment, the computing system 10 may analyze the fingerprint image or palmprint image sensed by the fingerprint sensing device based on a convolutional neural network operation to obtain information related to the sensed fingerprint and palmprint.

The computing system 10 may include a processing device 110 and an external memory 120. The processing device 110 and the external memory 120 can communicate via the bus 130. In one embodiment, the processing device 110 may be implemented as a system chip. The processing device 110 can perform a convolutional neural network operation according to the received input data, where the convolutional neural network operation includes a plurality of convolutional layers. It should be noted that the new creation does not limit the neural network model corresponding to the convolutional neural network operation. It can be any neural network model that includes multiple convolutional layers, such as the GoogleNet model, the AlexNet model, Various convolutional neural network models such as VGGNet model, ResNet model and LeNet model.

The external memory 120 is coupled to the processing device 110 for recording various parameters required by the processing device 110 to perform convolutional neural network operations, such as weight data of each convolutional layer and so on. The external memory 120 may include dynamic random access memory (DRAM), flash memory, or other memory. The processing device 110 can read various parameters required for performing the convolutional neural network operation from the external memory 120 to perform the convolutional neural network operation on the input data.

Fig. 2 is a schematic diagram of a convolutional neural network model according to an embodiment of the new creation. Please refer to FIG. 2, the processing device 110 can input the input data d_i to the convolutional neural network-based model 20 to generate output data d_o. In one embodiment, the input data d_i can be a grayscale image or a color image. On the other hand, the input data d_i can be a fingerprint sensing image or a palmprint sensing image. The output data d_o can be a classification category that classifies the input data d_i, a segmented image that has undergone semantic segmentation, or image data that has undergone image processing (such as style conversion, image filling, or resolution optimization, etc.). This new creation There is no restriction on this.

The convolutional neural network model 20 may include multiple layers, and these layers may include multiple convolutional layers. In some embodiments, these layers may also include a pooling layer, an incentive layer, a fully connected layer, etc., which are not limited by the present invention. Each layer in the convolutional neural network model 20 can receive input data d_i or a feature map (feature map) generated by the previous layer to perform relative arithmetic processing to generate an output feature map or output data d_o. Here, the feature map is data used to express various features of the input data d_i, which can be in the form of a two-dimensional matrix or a three-dimensional matrix (also called a tensor).

For the convenience of description, FIG. 2 only shows the convolutional neural network model 20 including the convolutional layers L1 to L3 as an example for description. As shown in Figure 2, the feature maps FM1, FM2, FM3 generated by the convolutional layers L1 to L3 are in the form of a three-dimensional matrix. In this example, the feature maps FM1, FM2, and FM3 may have a width w (or called a row), a height h (or called a column), and a depth d (or called a channel number).

The convolution layer L1 can generate a feature map FM1 by performing a convolution operation on the input data d_i according to one or more convolution kernels. The convolution layer L2 may perform a convolution operation on the feature map FM1 according to one or more convolution kernels to generate the feature map FM2. The convolution layer L3 may perform a convolution operation on the feature map FM2 according to one or more convolution kernels to generate the feature map FM3. The convolution kernels used in the above convolution layers L1 to L3 may also be referred to as weight data, which may be in the form of a two-dimensional matrix or a three-dimensional matrix. For example, the convolutional layer L2 can perform a convolution operation on the feature map FM1 according to the convolution kernel WM. In some embodiments, the number of channels of the convolution kernel WM is the same as the depth of the feature map FM1. The convolution kernel WM slides in the feature map FM1 according to a fixed step size. Whenever the convolution kernel WM is shifted, each weight included in the convolution kernel WM will be multiplied by all the feature values of the overlapping region on the feature map FM1 and then added. Since the convolution layer L2 performs a convolution operation on the feature map FM1 according to the convolution kernel WM, the feature value corresponding to a channel in the feature map FM2 can be generated. FIG. 2 only takes a single convolution kernel WM as an example for illustration, but the convolution layer L2 can actually perform convolution operations on the feature map FM1 based on multiple convolution kernels to generate a feature map FM2 with multiple channels.

Fig. 3 is a schematic diagram of a convolution operation according to an embodiment of the new creation. Referring to FIG. 3, it is assumed that a certain convolutional layer performs a convolution operation on the feature map FM_i generated by the previous layer, and it is assumed that the convolutional layer has 5 convolution kernels WM_1 to WM_5. These convolution kernels WM_1 to WM_5 are the weight data of the convolution layer. The feature map FM_i has a height H1, a width W1, and M channels. The convolution kernels WM_1 to WM_5 have height H2, width W2, and M channels. The convolution layer uses the convolution kernel WM_1 and the feature map FM_i to perform a convolution operation to obtain the sub-feature map 31 belonging to the first channel in the feature map FM_(i+1). The convolution layer uses the convolution kernel WM_2 and the feature map FM_i to perform a convolution operation to obtain the sub-feature map 32 belonging to the second channel in the feature map FM_(i+1). So on and so forth. Based on this convolutional layer with 5 convolution kernels WM_1～WM_5, the sub-feature maps 31-35 corresponding to the convolution kernels WM_1～WM_5 can be generated, thereby generating a feature map FM_( with height H3, width W3 and 5 channels i+1).

Based on the description of FIG. 2 and FIG. 3, it can be seen that the processing device 110 for performing convolutional neural network operations needs to perform convolution operations based on weight data. In some embodiments, the weight data can be stored in the external memory 120 or other storage devices in advance. The external memory 120 can provide the weight data to the processing device 110. That is, the internal memory built into the processing device 110 can be used to store the weight data provided by the external memory 120.

Fig. 4 is a schematic diagram of a processing device according to an embodiment of the creation of the present invention. Please refer to FIG. 4, the processing device 110 may include an internal memory 111, a calculation circuit 112, and a controller 113. The internal memory 111 is also called on-chip memory. The internal memory 111 is coupled to the calculation circuit 112. In some embodiments, the storage capacity of the internal memory 111 is smaller than the storage capacity of the external memory 120. In one embodiment, the calculation circuit 112 is used to analyze the physiological characteristic image sensed by the physiological characteristic sensing device. In one embodiment, the calculation circuit 112 may analyze the facial image sensed by the facial sensing device based on a convolutional neural network operation. In one embodiment, the calculation circuit 112 may analyze the fingerprint image or palmprint image sensed by the fingerprint sensor device based on a convolutional neural network operation.

The calculation circuit 112 is used to perform layer operations of multiple layers in the convolutional neural network operation, and it may include arithmetic logic circuits for completing various layer operations. It can be understood that the calculation circuit 112 may include an arithmetic logic circuit such as a multiplier array, an accumulator array, etc., to complete a convolution operation. In addition, the calculation circuit 112 may include a weight buffer 41. The weight buffer is used to temporarily store the weight data provided by the internal memory 111 so that the arithmetic logic circuit in the calculation circuit 112 can efficiently perform convolution operations.

The controller 113 can be implemented by a central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), a digital signal processor (DSP), or other The calculation circuit is implemented, which can control the overall operation of the processing device 110. The controller 113 can manage the operation parameters required for the operation of the convolutional neural network, such as weight data, so that the processing device 110 can normally perform the operation of each layer in the operation of the convolutional neural network.

In some embodiments, the controller 113 can control the internal memory 111 to obtain the weight data of all convolutional layers from the external memory 120. In some embodiments, the controller 113 can control the internal memory 111 to obtain the weight data of different convolutional layers from the external memory 120 at different time points. For example, the controller 113 may control the internal memory 111 to obtain the weight data of the first convolutional layer from the external memory 120 at a first time point, and control the internal memory 111 to obtain the first convolutional layer weight data from the external memory 120 at a second time point. The weight data of the two convolutional layers, where the first time point is different from the second time point. At the second time point, the weight data of the first convolutional layer in the internal memory 111 will be updated to the weight data of the second convolutional layer. In some embodiments, the controller 113 can control the internal memory 111 to obtain different parts of the weight data of the same convolutional layer from the external memory 120 at different time points. For example, the controller 113 can control the internal memory 111 to obtain the first part of the weight data of the first convolutional layer from the external memory 120 at a first time point, and control the internal memory 11 to obtain the first part of the weight data of the first convolutional layer from the external memory 120 at the first time. The second part of the same weight data of the first convolutional layer is acquired at two time points, where the first time point is different from the second time point.

Based on the foregoing, all the weight data required for convolutional neural network operations can be written into the internal memory 111 together, and the weight data required for convolutional neural network operations can be divided into multiple parts and written at different time points. Internal memory 111. It can be seen that the data in the internal memory 111 used to store the weight data of the convolutional neural network operation and the intermediate operation result (for example, the feature map of each convolutional layer) is frequently updated. Based on this, in the inventive embodiment of the present invention, while ensuring that the weight data in the internal memory 111 can be obtained by the calculation circuit 112, the data recorded by the memory cell of the internal memory 111 can be allowed to elapse with time. That is, the weight data calculated by the convolutional neural network can be kept in the internal memory 111 for a period of time.

More specifically, in the embodiment of the present invention, each memory cell has a corresponding data retention time according to the circuit configuration and component characteristics of the memory cell of the internal memory 111. After the data is written into a certain memory cell of the internal memory 111, the written data can be retained in the memory cell until the data retention time expires. That is, the weight data recorded by the memory cell will become invalid when the data retention time expires. Examples will be listed below for clear description.

Fig. 5 is a schematic diagram of an internal storage device according to an embodiment of the invention. 5, the internal memory 111 may include a memory cell array 51, a column decoder 52, and a row decoder 53. In the memory cell array 51, a plurality of word lines WL and bit lines BL are mainly arranged alternately in an array manner, and each interlace point has a memory cell (Memory Cell) MC. That is, the memory cell array 51 includes a plurality of memory cells MC arranged in an array. These memory cells MC use the charge and discharge principle of capacitors to achieve the purpose of recording data. When the internal memory 111 receives the Access Row Address, it will be decoded by the column decoder 52 to enable the corresponding word line WL. Then, the charge of the capacitor in the memory cell MC connected to the enabled word line WL can flow to the corresponding bit line BL. The row decoder 53 can control the row selector according to the column address to read or write the data corresponding to the row address. It should be noted that, in some embodiments, the memory cell MC in the memory cell array 51 can be used to store the weight data of one or more convolutional layers. That is, the weight data of one or more convolutional layers can be written into the multiple memory cells MC in the memory cell array 51, and the weight data of one or more convolutional layers can be obtained from the multiple memory cells MC in the memory cell array 51. Is read out.

Fig. 6A is a schematic diagram of a memory cell according to an embodiment of the creation of the present invention. 6A, each memory cell MC in the memory cell array 51 may include a control circuit 61 and a capacitor C1. In some embodiments, the control circuit 61 may include a transistor M1. The control terminal of the transistor M1 is coupled to the word line WL of the internal memory 111, the first terminal of the transistor M1 is coupled to the bit line BL of the internal memory 111, and the second terminal of the transistor M1 is coupled to the capacitor C1. One end. However, in other embodiments, the control circuit 61 may also include other electronic components, which is not limited by the present invention. In some embodiments, the internal memory 111 uses the amount of charge stored in the capacitor C1 to represent a binary bit "1" or "0".

It is worth noting that even if the transistor M1 in the memory cell MC is turned off, the charge stored in the capacitor C1 will gradually disappear over time, causing data loss. That is, the capacitor C1 will have a leakage phenomenon, so that the recorded data will be lost. In more detail, the control circuit 61 may have a leakage current path, and the charge in the capacitor C1 may leak from the leakage current path of the control circuit 61. In the embodiment of the present invention, the data retention time of each memory cell MC is determined based on the leakage current on the leakage current path and the capacitance value of the capacitor C1, wherein the data retention time is greater than a predetermined required time. The preset required time is determined according to the calculation speed and calculation amount of the calculation circuit 112. The higher the calculation speed of the calculation circuit 112, the shorter the preset required time. The lower the calculation amount of the calculation circuit 112, the shorter the preset required time. It can be seen that when the preset demand time is shorter, the data retention time of the memory cell MC can also be shorter.

Fig. 6B is a schematic diagram of a memory cell according to an embodiment of the creation of the present invention. 6B, in some embodiments, each memory cell MC in the memory cell array 51 may include a capacitor C1, a switch SW1, a switch SW2, a sense amplifier circuit Amp1, and a write amplifier circuit Amp2. One end of the switch SW1 is coupled to one end of the capacitor C1, and the other end of the switch SW1 can be coupled to the bit line BL of the internal memory 111. One end of the switch SW2 is coupled to one end of the capacitor C1, and the other end of the switch SW2 is coupled to the input end of the sense amplifier circuit Amp1. The other end of the capacitor C1 can be coupled to the reference ground voltage. The output terminal of the sense amplifier circuit Amp1 can be coupled to the bit line BL of the internal memory 111. The output terminal of the write amplifier circuit Amp2 is coupled to one end of the switch SW2 and the input terminal of the sense amplifier circuit Amp1, and the input terminal of the write amplifier circuit Amp2 is coupled to the bit line BL of the internal memory 111. The control ends of the switch SW1 and the switch SW2 can be coupled to the word line WL of the internal memory 111. The internal memory 111 uses the amount of charge stored in the capacitor C1 to represent a "1" or "0" of a binary bit. When data is to be written into the capacitor C1, the switch SW1 or the switch SW2 can be turned on, so that the written data can be recorded in the capacitor C1 through the switch SW1 or the write amplifier circuit Amp2 and the switch SW2. When the data recorded by the capacitor C1 is to be read, the switch SW2 can be turned on, so that the data recorded by the capacitor C1 can be read through the sense amplifier circuit Amp1.

As shown in FIG. 6B, the capacitor C1 has a leakage phenomenon and generates a leakage current path L1 (represented by the leakage current source 65 here), so that the data recorded by the capacitor C1 is lost. In addition, even if the SW2 is not turned on, the switch SW2 will leak current and generate a leakage current path L2 (represented by the leakage current source 66 here), causing the data recorded by the capacitor C1 to be lost. Here, the leakage current levels of the leakage current source 65 and the leakage current source 66 depend on the element characteristics of the capacitor C1 and the switch SW2.

In some embodiments, after the calculation circuit 112 obtains the weight data of one or more convolutional layers from the internal memory 111, the weight data recorded by each memory cell MC becomes invalid when the data retention time expires. Here, the weight data of the convolutional layer may include part or all of the weight values in at least one convolution kernel. After the weight data is written into the memory cell MC, during the data retention time of the memory cell MC, the calculation circuit 112 will obtain the correct weight data from the memory cell MC, and temporarily store the weight data in the weight buffer 41 for subsequent volumes. Product operation is used. Moreover, after the data retention time of the memory cell MC has elapsed, too much charge leakage of the capacitor C1 in the memory cell MC causes the weight data recorded by the memory cell MC to become invalid.

In some embodiments, the data retention time of each memory cell MC is positively related to the capacitance value of the capacitor C1. That is, the smaller the capacitance value of the capacitor C1, the shorter the data retention time of the memory cell MC. Conversely, the larger the capacitance value of the capacitor C1, the longer the data retention time of the memory cell MC. Based on this, in the case of ensuring that the data retention time is greater than the preset required time, even the use of a capacitor C1 with a small capacitance value is allowable, thereby reducing the power consumption and circuit area of memory reading.

In some embodiments, the data retention time of each memory cell MC is negatively related to the current value of the leakage current. That is, the smaller the capacitance value of the leakage current on the leakage current path provided by the control circuit 61 is, the longer the data retention time of the memory cell MC is. Conversely, the greater the capacitance value of the leakage current on the leakage current path provided by the control circuit 61, the shorter the data retention time of the memory cell MC. Based on this, under the condition that the data retention time is greater than the preset required time, the circuit configuration and internal component design of the control circuit 61 with leakage current path can be more flexible.

It is worth mentioning that, compared with the traditional dynamic random access memory, the internal memory 111 does not need to enter a refresh mode to refresh the data of each memory cell MC. Therefore, the circuit area of the internal memory 111 can also be reduced in the absence of related circuits required for the refresh mode.

In addition, based on the foregoing knowledge, the internal memory 111 obtains the weight data of one or more convolutional layers from the external memory 120. To reduce the capacitance value of the capacitor C1 and thereby reduce the data retention time of the memory cell MC, it means that the update speed of the weight data in the internal memory 111 should be increased. Therefore, in some embodiments, the weight data required for the convolutional neural network operation can be sequentially written into the internal memory 111 of the processing device 110 in batches to speed up the update speed of the weight data. In this case, the data amount of the weight data of the convolutional layer obtained from the external memory 120 is positively correlated with the capacitance value of the capacitor C1.

For example, if a capacitor C1 with a small capacitance value is to be used to reduce reading power consumption, the internal memory 111 can first read the weight data of one of the multi-layer convolutional layers. The internal memory 111 can retain the weight data of one of the multi-layer convolutional layers until the data retention time expires, and the weight data recorded in the internal memory 111 will become invalid when the data retention time expires. After that, the internal memory 111 reads the weight data of another layer of the multi-layer convolutional layer. Similarly, the internal memory 111 can retain the weight data of the other layer of the multi-layer convolutional layer until the data retention time expires.

FIG. 7 is a schematic diagram of data retention time according to an embodiment of the creation of the present invention. Please refer to FIG. 7, at time t1, the weight data of the convolutional layer is written into the internal memory 111. For example, the weight value in one or more convolution kernels of one of the multiple convolution layers can be written to the internal memory 111 at time t1. Alternatively, part of the weight value in a convolution kernel of one of the multiple convolution layers can be written into the internal memory 111 at time t1. At time t2, the calculation circuit 112 reads the weight data of the convolutional layer from the internal memory 111. After the calculation circuit 112 obtains the weight data of the convolutional layer from the internal memory 111, at time t3, the weight data recorded by each memory cell MC becomes invalid when the data retention time ΔT expires. After the weight data recorded by the memory cell MC becomes invalid, at time t4, other weight data of the convolutional layer are written into the memory cell MC of the internal memory 111. At time t5, the calculation circuit 112 reads other weight data of the convolutional layer from the internal memory 111. At time t6, other weight data recorded by each memory cell MC becomes invalid when the data retention time ∆T expires.

FIG. 8 is a flowchart of a processing method for performing convolutional neural network operations according to an embodiment of the new creation. Please refer to FIG. 8, the method of this embodiment is applicable to the processing device 110 in the embodiment of FIG. 4, and the detailed steps of this embodiment are described below with various components in the processing device 110.

In step S801, the weight data of at least one convolutional layer is obtained from the external memory 120 through the internal memory 111, and the convolution operation of the convolutional layer is performed. In some embodiments, the processing device 110 may obtain the weight data of the first convolutional layer from the external memory 120 through the internal memory 111 at the first time point, and obtain the weight data of the first convolutional layer from the external memory 120 through the internal memory 111 at the second time point. Obtain the weight data of the second convolutional layer, where the first time point is different from the second time point. In some embodiments, the processing device 110 can obtain the first part of the weight data of the first convolutional layer from the external memory 120 through the internal memory 111 at a first time point, and from the external memory 120 through the internal memory 111 In the second part of obtaining the weight data of the first convolutional layer, the first time point is different from the second time point.

It should be noted that the weight data of at least one convolution layer recorded by each memory cell in the internal memory 111, such as all weight data or partial weight data of a certain convolution layer, will become invalid when the data retention time expires. Each memory cell in the internal memory 111 includes a control circuit and a capacitor. The control circuit has a leakage current path, and the data retention time of each memory cell is determined according to the leakage current on the leakage current path and the capacitance value of the capacitor.

In summary, in the creative embodiment of the present invention, the memory cell of the internal memory used to record the weight data of the convolutional layer has a data retention time. After the data retention time has elapsed, the weight data recorded by the memory cell will become invalid due to the leakage of the capacitor. The data retention time of the memory cell is determined by the leakage current and the capacitance value of the capacitor. Based on this, in the case of ensuring that the data retention time of the memory cell is greater than the preset required time, the memory cell can use a capacitor with a smaller capacitance value, thereby reducing the power consumption and circuit area of the memory read. Therefore, the circuit area and power consumption of the internal memory provided in the processing device can be reduced.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the new creation, not to limit it; although the new creation is described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand : It can still modify the technical solutions recorded in the foregoing embodiments, or equivalently replace some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the various embodiments of the invention The scope of the technical solution.

10: Computing system 110: processing device 120: External memory 130: bus d_i: input data d_o: output data 20: Convolutional Neural Network Model L1～L3: Convolutional layer FM1, FM2, FM3, FM_i, FM_(i+1): feature map WM, WM_1～WM_5: Convolution kernel 31～35: Sub-characteristic map 111: Internal memory 112: calculation circuit 113: Controller 41: weight buffer 51: Memory cell array 52: column decoder 53: Line decoder WL: Character line BL: bit line MC: memory cell 61: control circuit M1: Transistor C1: Capacitor SW1, SW2: switch L1, L2: leakage current path 65, 66: Leakage current source Amp1: sense amplifier circuit Amp2: Write amplifier circuit ∆T: data retention time t1～t6: time S801: Process steps

FIG. 1 is a schematic diagram of a computing system for performing convolutional neural network operations according to an embodiment of the new creation. Fig. 2 is a schematic diagram of a convolutional neural network model according to an embodiment of the new creation. Fig. 3 is a schematic diagram of a convolution operation according to an embodiment of the new creation. Fig. 4 is a schematic diagram of a processing device according to an embodiment of the creation of the present invention. Fig. 5 is a schematic diagram of an internal storage device according to an embodiment of the invention. Fig. 6A is a schematic diagram of a memory cell according to an embodiment of the creation of the present invention. Fig. 6B is a schematic diagram of a memory cell according to an embodiment of the creation of the present invention. FIG. 7 is a schematic diagram of data retention time according to an embodiment of the creation of the present invention. FIG. 8 is a flowchart of a processing method for performing convolutional neural network operations according to an embodiment of the new creation.

110: processing device

111: Internal memory

112: calculation circuit

113: Controller

120: External memory

130: bus

41: weight buffer

Claims (12)

A processing device for performing convolutional neural network operations. The convolutional neural network operations include multiple convolutional layers. The processing device includes: A calculation circuit for performing the convolution operation of each of the convolution layers, and the calculation circuit is used to analyze the physiological characteristic image sensed by the physiological characteristic sensing device; and The internal memory is coupled to the calculation circuit and includes a plurality of memory cells, and is used to store the weight data of the convolutional layer, Wherein, each of the memory cells includes a control circuit and a capacitor, the control circuit has a leakage current path, and the data retention time of each of the memory cells is determined according to the leakage current on the leakage current path and the capacitance value of the capacitor. The processing device for performing convolutional neural network operations according to claim 1, wherein after the calculation circuit obtains the weight data of the convolutional layer from the internal memory, the data recorded by each memory cell The weighted data becomes invalid when the said data retention time expires. The processing device for performing convolutional neural network operations according to claim 2, wherein after the weight data recorded by the memory cell becomes invalid, the other weight data of the convolutional layer is written into the internal memory的The memory cell. The processing device for performing convolutional neural network operations according to claim 1, wherein the data retention time is positively related to the capacitance value of the capacitor. The processing device for performing convolutional neural network operations according to claim 1, wherein the data retention time is negatively related to the current value of the leakage current. The processing device for performing convolutional neural network operations according to claim 1, wherein the internal memory obtains the weight data of the convolutional layer from an external memory. The processing device for performing convolutional neural network operations according to claim 6, wherein the data amount of the weight data of the convolutional layer obtained from an external memory is positively related to the capacitance value of the capacitor. The processing device for performing convolutional neural network operations according to claim 1, wherein the weight data of the convolution layer includes part or all of the weight values in at least one convolution kernel. The processing device for performing convolutional neural network operations according to claim 1, wherein the control circuit includes a transistor, the control end of the transistor is coupled to the word line of the internal memory, and The first end of the transistor is coupled to the bit line of the internal memory, and the second end of the transistor is coupled to one end of the capacitor. The processing device for performing convolutional neural network operations according to claim 1, wherein the calculation circuit includes a weight buffer, and the internal memory provides weight data of the convolution layer to the weight buffer . The processing device for performing convolutional neural network operations according to claim 1, wherein the physiological characteristic sensing device is a fingerprint sensing device, and the physiological characteristic image is a fingerprint image or a palm print image. The processing device for performing convolutional neural network operations according to claim 1, wherein the physiological characteristic sensing device is a face sensing device, and the physiological characteristic image is a face image.

Images