WO2020034079A1 - 基于脉动阵列的神经网络处理装置 - Google Patents

基于脉动阵列的神经网络处理装置 Download PDF

Info

Publication number
WO2020034079A1
WO2020034079A1 PCT/CN2018/100335 CN2018100335W WO2020034079A1 WO 2020034079 A1 WO2020034079 A1 WO 2020034079A1 CN 2018100335 W CN2018100335 W CN 2018100335W WO 2020034079 A1 WO2020034079 A1 WO 2020034079A1
Authority
WO
WIPO (PCT)
Prior art keywords
intermediate processing
processing result
calculation unit
result
control signal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/100335
Other languages
English (en)
French (fr)
Inventor
高明明
杨康
谷骞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
SZ DJI Technology Co Ltd
Original Assignee
SZ DJI Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by SZ DJI Technology Co Ltd filed Critical SZ DJI Technology Co Ltd
Priority to CN201880038645.0A priority Critical patent/CN110785778A/zh
Priority to PCT/CN2018/100335 priority patent/WO2020034079A1/zh
Publication of WO2020034079A1 publication Critical patent/WO2020034079A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • the present application relates to the field of neural networks, and more particularly, to a neural network processing device based on a pulsation array.
  • neural network technologies have problems such as occupying large amounts of resources, slow operating speeds, and high energy consumption in the actual application process.
  • the use of hardware accelerators to replace traditional software calculations has become an effective way to improve the efficiency of neural network calculations, such as neural network processors implemented using general-purpose graphics processors, special-purpose processor chips, and field programmable logic arrays.
  • an independent hardware device is designed for convolution operations in neural networks
  • an independent hardware device is designed for pooling operations
  • an independent set of independent hardware devices is designed for average pooling operations and maximum pooling operations, respectively.
  • Hardware device is designed for convolution operations in neural networks
  • an independent hardware device is designed for pooling operations
  • an independent set of independent hardware devices is designed for average pooling operations and maximum pooling operations, respectively.
  • the present application provides a neural network processing device based on a pulsation array, which can effectively reduce the hardware overhead and power consumption of the neural network.
  • a neural network processing device includes: a pulsation array including a plurality of computing units; a control unit for sending a control signal, the control signal indicating any of the following neural network operations: convolution Operation, average pooling operation, and maximization operation; each calculation unit in the pulsation array is used to receive the control signal, input feature data, weights, and the first from the upper-level calculation unit in the second dimension along the pulsation array.
  • An intermediate processing result performing a neural network-related operation on the input feature data, the weights, and the first intermediate processing result according to the control signal to obtain a second intermediate processing result; to the second dimension along the pulsation array
  • the next-level computing unit sends the second intermediate processing result; the input feature data is sent to the next-level computing unit along the first dimension of the pulsation array; the processing unit is connected to the computing unit at the output end of the pulsation array, and the processing The unit is used for receiving the control signal and the second medium output from the calculation unit at the output of the pulsation array.
  • a neural network-related operation is performed on the second intermediate processing result according to the control signal to obtain a neural network output result.
  • a pooling processing device in a second aspect, includes: a pulsation array including a plurality of calculation units; a control unit for sending a control signal, the control signal indicating an average pooling operation or a maximization operation; the pulsation array
  • Each calculation unit in is used to receive the control signal, input feature data, and a first intermediate processing result from a higher-level calculation unit along a second dimension along the pulsation array; according to the control signal, the input feature data and Pooling-related operations are performed on the first intermediate processing result to obtain a second intermediate processing result; sending the second intermediate processing result to a next-level computing unit of the second dimension along the pulsation array;
  • the next-level computing unit of the first dimension sends the input characteristic data;
  • the processing unit is connected to the computing unit at the output of the pulsation array, and the processing unit is configured to receive the control signal and the second output from the computing unit at the output of the pulsation array An intermediate processing result, and performing a pooling-related operation on
  • a chip is provided, on which the device provided in the first aspect or the device provided in the second aspect is integrated.
  • a control method is provided.
  • the control method is executed by a control unit in the device provided by the first aspect, and the method includes a related process performed by the control unit in the first aspect.
  • a control device includes a memory and a processor.
  • the memory is configured to store instructions.
  • the processor is configured to execute instructions stored in the memory. Execution of the instructions stored in the memory causes the processing.
  • the processor executes the method provided by the fourth aspect.
  • a computer-readable storage medium having stored thereon a computer program, which when executed by a computer causes the computer to implement the method provided by the fourth aspect.
  • a computer program product includes instructions that, when executed by a computer, cause the computer to implement the method provided in the fourth aspect.
  • FIG. 1 is a schematic block diagram of a pulsation array-based neural network processing device according to an embodiment of the present application.
  • FIG. 2 is a schematic flowchart of implementing a neural network operation by using a pulsation array-based neural network processing device according to an embodiment of the present application.
  • 3 to 6 are schematic diagrams of an internal structure of a computing unit in an embodiment of the present application.
  • FIG. 7 to FIG. 14 are timing diagrams of implementing a convolution operation or an average pooling operation by using the neural network processing device provided by the embodiment of the present application.
  • FIG. 15 to FIG. 21 are timing diagrams of implementing the maximum pooling operation by using the neural network processing device provided by the embodiment of the present application.
  • FIG. 22 is a schematic diagram of implementing a convolution operation for multiple convolution kernels by using a neural network processing device provided by an embodiment of the present application.
  • FIG. 23 is a schematic block diagram of a pulsation array-based pooling processing apparatus according to another embodiment of the present application.
  • FIG. 1 is a schematic block diagram of a neural network processing apparatus 100 according to an embodiment of the present application.
  • the device 100 includes a control unit 110, a pulsation array 120, and a processing unit 130.
  • the control unit 110 is configured to send a control signal to the pulsation array 120 and the processing unit 130, and the control signal indicates any one of the following neural network operations: a convolution operation, an average pooling operation, and a maximization operation.
  • the pulsation array 120 includes a plurality of calculation units 200.
  • the internal structure of each computing unit 200 in the pulsation array 120 is the same, which will be described in detail below.
  • Each calculation unit 200 in the pulsation array 120 is configured to receive a control signal, input feature data, weights, and a first intermediate processing result from a higher-level calculation unit along a second dimension along the pulsation array; according to the control signal, input characteristics
  • the data, weights, and first intermediate processing results are related to the neural network to obtain the second intermediate processing results.
  • the second intermediate processing results are sent to the next-level computing unit along the second dimension of the pulsation array.
  • the next-level computing unit of the first dimension sends the input feature data.
  • the neural network-related operations performed by the calculation unit 200 on the input feature data, weights, and the first intermediate processing result include multiplication operations and addition operations.
  • the control signal indicates the average pooling operation
  • the neural network-related operations performed by the calculation unit 200 on the input feature data, weights, and the first intermediate processing result include addition operations.
  • the control signal indicates the maximum pooling operation
  • the neural network related operations performed by the calculation unit 200 on the input feature data, weights, and the first intermediate processing result include comparison operations.
  • the first dimension corresponds to the rows of the pulsation array
  • the second dimension corresponds to the columns of the pulsation array.
  • the first dimension corresponds to the columns of the pulsation array
  • the second dimension corresponds to the rows of the pulsation array.
  • first dimension corresponds to the rows of the pulsating array 120
  • second dimension corresponds to the columns of the pulsation array 140.
  • left to right and top to bottom also appear in some embodiments below, where “left to right” means left to right along the rows of the pulsating array, and “top to bottom” Represents the columns along the pulsation array from top to bottom.
  • the calculation unit may receive input feature values from a feature data storage 140 other than the pulsation array 120.
  • the remaining calculation units all receive input feature values from the upper-level calculation unit along the first dimension.
  • each calculation unit in the pulsation array 120 may receive weights from a weight memory 150 outside the pulsation array 120.
  • the main processor determines that the input feature data and weights to be loaded into the pulsation array 120 are then loaded from the feature data storage 140 into the pulsation array 120 and the weights are loaded from the weight storage 150 Into the pulsation array 120.
  • the main processor may be the control unit 110, or may be another processor that understands the calculation law of the pulsation array in the neural network system.
  • the neural network processing apparatus 100 includes a feature data storage 140 and a weight storage 150.
  • the neural network processing device 100 does not include the feature data storage 140 and the weight storage 150, but includes a communication interface for communicating with the feature data storage 140 and the weight storage 150.
  • the neural network processing apparatus 100 may further include a cache unit, which is configured to cache data transmitted from the communication interface. That is, the data in the feature data storage 140 and the weight storage 150 pass through the buffer unit and enters the pulsation array 120. It should be understood that using the buffer unit to cache the data transmitted from the communication interface helps to ensure the correctness of the timing of the data entering the pulse array 120.
  • the processing unit 130 is connected to the calculation unit 200 at the output of the pulsation array 120.
  • the processing unit 130 is configured to receive a control signal and a second intermediate processing result output from the calculation unit at the output of the pulsation array, and perform a neural network on the second intermediate processing result according to the control signal. Related operations to obtain the output of the neural network.
  • a unit described herein is connected to a unit, including but not limited to a wired communication connection and a wireless communication connection.
  • the calculation unit at the output of the pulsation array 120 refers to a calculation unit that finally outputs the intermediate processing result in the pulsation array 120.
  • it refers to the calculation unit of the last row of the pulsation array 120.
  • the sending interface of the computing unit at the output of the pulsation array 120 is communicatively connected to the receiving interface of the processing unit 130.
  • the processing unit 130 When the control signal indicates a convolution operation, the processing unit 130 performs an accumulation operation on the second intermediate processing result. When the control signal indicates an average pooling operation, the processing unit 130 performs an averaging operation on the second intermediate processing result. When the control signal indicates the maximum pooling operation, the processing unit 130 performs a comparison operation on the second intermediate processing result.
  • control unit 110 is used to control the types of neural network related operations performed by the pulsation array 120 and the processing unit 130, that is, the convolution operation, the average pooling operation, or the maximum pooling operation, so that the device 100 provided in this application can both implement Convolution operations can also achieve pooling operations.
  • the solution provided in this application can increase the hardware utilization rate and save costs by reusing a set of hardware devices for convolution operation and pooling operation.
  • the elements of the pulsation (forwarded between the calculation units of the pulsation array) in the pulsation array include the input feature data and the intermediate processing results calculated by the calculation unit, excluding weights.
  • the weights are loaded into the computing unit, they are no longer forwarded.
  • first intermediate processing result and the second intermediate processing result mentioned in this article are only for distinguishing the intermediate processing result received and sent by the same computing unit, and other descriptions can also be used to distinguish the intermediate receiving and sending of the same computing unit.
  • the processing result is not limited in this application.
  • calculation unit X is a calculation unit along the input end of the pulsation array along the first dimension
  • calculation unit X is from an external memory.
  • the calculation unit Y does not have a previous-stage calculation unit along the second dimension, that is, the calculation unit Y is used to receive the previous stage.
  • the input of the port of the first intermediate processing result of the computing unit is empty or zero.
  • a calculation unit Z is a calculation unit along the output end of the pulsation array along the second dimension, there is no subsequent calculation unit along the second dimension in the calculation unit Z.
  • the calculation unit Z is connected to the processing unit 130 for The obtained second intermediate processing result is sent to the processing unit 130.
  • the weight loaded into the pulsation array 120 is a convolution kernel.
  • the distribution manner of the weights in the pulsation array 120 is consistent with or mirrored the distribution manner of the weights in the convolution kernel.
  • the two-dimensional matrix corresponding to the convolution kernel is Then the distribution of the weights loaded in the pulsation array 120 in the pulsation array 120 can be Can also be
  • the main processor (such as the control unit 110 or other processors in the neural network system) reads the corresponding weights from the weight memory and loads them into the pulsation array 120, where the addressing of the weights and the weights in the pulsation array 120
  • the distribution manners can be determined according to actual requirements, which is not limited in this application, as long as a convolution operation or a pooling operation can be implemented.
  • the pooling operation is only for the input feature data and does not involve weights. Therefore, in order to enable the computing unit 200 to be compatible with the convolution operation and the pooling operation, when the control signal indicates the maximum pooling operation or the average pooling operation, The value is configured as 1.
  • the apparatus 100 further includes the following units.
  • the feature data storage (such as the feature data storage 140 shown in FIG. 1) is used to store input feature data.
  • a first weight memory (such as weight memory 150 shown in FIG. 1) and a second weight memory (not shown in FIG. 1).
  • the first weight memory is used to store the convolution kernel of the current convolution operation
  • the second weight memory is used to store the convolution kernel of the next convolution operation in advance.
  • the control unit 110 is configured to load the weights in the first weight memory into the calculation unit in the pulsation array 120, that is, load the calculation units in the input end along the first dimension of the pulsation array 120.
  • the apparatus provided in this embodiment when performing a convolution operation, stores the convolution kernel of the next convolution operation of the current convolution operation in advance, so that after the related operation of the current convolution operation is completed using the pulsation array, it can start immediately
  • the operation of the next convolution operation can improve the efficiency of the convolution operation.
  • the process of processing a neural network by using the pulsation array-based device 100 shown in FIG. 1 is as follows.
  • the control unit 110 determines a neural network operation type, and the neural network operation type is any one of the following operations: a convolution operation, an average pooling operation, and a maximum pooling operation.
  • the control unit 110 addresses the storage unit (such as the feature memory and the weight memory shown in FIG. 1) according to the operation type of the neural network, reads and parses the instructions to be executed in the next step.
  • the storage unit such as the feature memory and the weight memory shown in FIG. 1
  • the control unit 110 obtains a storage address according to the analysis instruction, obtains input data (input feature data and weights) from the storage unit, and loads the input feature data and weights into the pulsation array, respectively.
  • the control unit 110 sends a control signal to each calculation unit 200 in the pulsation array 120, where the control signal indicates a type of operation of the neural network.
  • Each calculation unit 200 in the pulsation array 120 performs a correlation operation on the received input feature data and weights based on the control signal to obtain a second intermediate processing result, and sends the first intermediate processing unit along the second dimension to the first calculation unit. Second intermediate processing results. Step S250 will be described in detail below.
  • the control unit 120 sends a control signal to the processing unit 130, where the control signal indicates a neural network operation type.
  • Step S270 The processing unit 130 obtains a second intermediate processing result from the computing unit 200 at the output end of the pulsation array 120, and performs a correlation operation on the second intermediate processing result based on the control signal to obtain an output result of the neural network. Step S270 will be described in detail below.
  • the output result of the neural network output by the processing unit 130 may be stored in an output storage unit of the neural network system.
  • the above step S250 includes the following content.
  • the calculation unit 200 multiplies the input feature data and the weight to obtain a product, and accumulates the product with the first intermediate processing result to obtain an accumulation result.
  • the accumulation result is The second intermediate processing result.
  • the calculation unit 200 compares the input feature data with the first intermediate processing result to obtain the maximum value of the two, which is the second intermediate processing result.
  • the calculation unit 200 multiplies the input feature data and the weight to obtain a product, and compares the product with the first intermediate processing result to obtain the maximum value of the two, which is the second intermediate processing result.
  • the weights loaded into the pulsation array are all 1.
  • step S250 is a calculation unit along the second dimension output end in the pulsation array
  • the second intermediate processing result is sent to the processing unit 130.
  • the processing unit 130 receives the second intermediate processing result as the intermediate result of the convolution operation corresponding to a certain convolution kernel as an example, or the processing unit 130 receives the second intermediate processing result as the pool corresponding to a certain pooling window
  • the intermediate result of the change operation is taken as an example, and step S270 specifically includes the following content.
  • the processing unit 130 When the control signal indicates a convolution operation, the processing unit 130 accumulates the plurality of second intermediate processing results received to obtain a calculation result of a convolution kernel.
  • the processing unit 130 averages a plurality of second intermediate processing results received based on the size (k * k) of the pooling window to obtain an average pooling calculation result of one pooling window .
  • the processing unit 130 compares the plurality of second intermediate processing results received, obtains the maximum value among them, and obtains the maximum pooling calculation result of a pooling window.
  • each calculation unit in the pulsation array adopts the same internal structure.
  • the internal structure of the computing unit can be implemented in various ways.
  • 3 and 4 are schematic diagrams of an internal structure of the computing unit 200. 3 and 4 show two working modes of the computing unit 200 based on the same internal structure.
  • 5 and 6 are schematic diagrams of another internal structure of the computing unit 200. 5 and 6 show two working modes of the computing unit 200 based on the same internal structure.
  • FIG. 3 As an implementation manner, the internal structure of the computing unit 200 is shown in FIG. 3 (or FIG. 4).
  • the feature register 201 is used to store the input feature data and is also used to forward the input feature data to a subsequent-stage computing unit along the first dimension of the pulsation array.
  • the weight register 202 is used for storing weights.
  • the multiplier 203 is connected to the feature register 201 and the weight register 202.
  • the multiplier 203 is used to output a product of weights and input feature data.
  • the adder 204 is connected to the multiplier 203.
  • the comparator 205 is connected to the multiplier 203.
  • the calculation unit 200 has two operating modes.
  • the computing unit 200 receives the control signal, and when the control signal obtained by the analysis indicates the convolution operation or the average pooling operation, the work mode one is turned on, as shown in FIG. 3.
  • the adder 204 is configured to receive the product of the multiplier 203 and the first intermediate processing result from the previous-stage calculation unit along the second dimension, and accumulate the product and the first intermediate processing result to obtain a second intermediate processing result. , Sending the second intermediate processing result to the subsequent-level computing unit along the second dimension.
  • working mode 1 the circuit where the comparator 205 is located is invalid (such as the circuit marked by the dashed line in FIG. 3).
  • the computing unit 200 receives the control signal, and when the parsed control signal indicates the maximum pooling operation, it starts the working mode two, as shown in FIG. 4.
  • the comparator 205 is used to receive the product of the multiplier 203 and the first intermediate processing result from the previous-stage calculation unit along the second dimension, and output the maximum value of the product and the first intermediate result, and to move along the second
  • the next-level calculation unit of the dimension sends the second intermediate processing result, and the second intermediate processing result is the maximum value.
  • the circuit where the adder 204 is located is invalid (such as the circuit marked by the dotted line in FIG. 4).
  • the computing unit 200 further includes a control sub-unit that switches a working mode of the computing unit 200 according to a control signal.
  • FIG. 5 As another implementation manner, the internal structure of the computing unit 200 is shown in FIG. 5 (or FIG. 6).
  • the feature register 401 is used to store the input feature data and is also used to forward the input feature data to a subsequent-stage computing unit along the first dimension of the pulsation array.
  • the weight register 402 is used for storing weights.
  • the multiplier 403 is connected to the feature register 401 and the weight register 402.
  • the multiplier 403 is used to output a product of weights and input feature data.
  • the adder 404 is connected to the multiplier 403.
  • the selector 405 is connected to the multiplier 403 and the adder 404.
  • the calculation unit 200 has two operating modes.
  • the calculation unit 200 receives the control signal, and when the parsed and obtained control signal indicates the convolution operation or the average pooling operation, the work mode one is turned on, as shown in FIG. 5.
  • the adder 404 is configured to receive the product of the multiplier 403 and the first intermediate processing result from the previous-stage calculation unit along the second dimension, and accumulate the product and the first intermediate processing result to obtain a second intermediate processing result. , Sending the second intermediate processing result to the subsequent-level computing unit along the second dimension.
  • the circuit where the selector 405 is located is invalid (such as the circuit marked by the dotted line in FIG. 5).
  • the computing unit 200 receives the control signal, and when the parsed control signal indicates the maximum pooling operation, it starts the working mode two, as shown in FIG. 6.
  • the adder 404 is used for receiving the product of the multiplier 403 and the first intermediate processing result from the previous-stage calculation unit along the second dimension, and outputting the subtraction result of the product and the first intermediate processing result;
  • the selector 405 is used for After receiving the product output from the multiplier 403, the subtraction result output from the adder 404, and the first intermediate processing result, a maximum value is selected from the product and the first intermediate processing result according to the subtraction result, and is directed along the second dimension.
  • the computing unit at the subsequent stage sends the second intermediate processing result, and the second intermediate processing result is the maximum value.
  • some circuits where the adder's 404 is located are invalid (such as the circuit marked by the dotted line in FIG. 6).
  • the computing unit 200 further includes a control sub-unit, which is configured to switch the working mode of the computing unit 200 according to a control signal.
  • multiplier mentioned in this article can be implemented by a multiplication circuit
  • adder can be implemented by an adder circuit
  • comparator can be implemented by a comparison circuit
  • selector can be implemented by a comparison circuit
  • FIGS. 3 to 6 are merely examples and are not limiting.
  • the internal results of the computing unit 200 may also be implemented in other implementation manners, which are not limited in this application, as long as the computing unit 200 can switch the working mode according to the type of neural network operation indicated by the control signal.
  • the processing unit 130 can switch between a plurality of working modes according to the operation type of the neural network indicated by the control signal.
  • the processing unit 130 is configured to accumulate the received second intermediate processing result to obtain an output result of the convolution operation.
  • the processing unit 130 is configured to average the received second intermediate processing result based on the size of the pooling window to obtain an output result of the average pooling operation.
  • the processing unit 130 is configured to compare the received second intermediate processing result to obtain an output result of the maximum pooling operation.
  • the internal results of the processing unit 130 include: an accumulator, a divider (or multiplier), and a comparator.
  • the accumulator inside the processing unit 130 works.
  • the accumulator is configured to output the accumulation results of the second intermediate processing results of the multiple computing units, and obtain the output results of the convolution operation.
  • the accumulator and divider inside the processing unit 130 work. Specifically, the accumulator is used to output the accumulation results of the second intermediate processing results of multiple calculation units, and the divider is used to average the accumulation results based on the size of the pooling window, for example, the size of the pooling window is k * k , The divider divides the accumulation result by k * k to obtain the output of the average pooling operation. It should be understood that averaging can also be implemented by a multiplier. For example, if the size of the pooling window is k * k, the multiplier multiplies the accumulation result by 1 / (k * k) to obtain the output of the average pooling operation. .
  • the comparator inside the processing unit 130 works. Specifically, the comparator is configured to output the maximum value of the second intermediate processing results of the multiple computing units to obtain the output result of the maximum pooling operation.
  • the solution provided in this application can increase the hardware utilization rate and save costs by reusing a set of hardware devices for convolution operation and pooling operation.
  • the neural network processing device of the pulsation array performs a process of a convolution operation, an average pooling operation, and a maximum pooling operation.
  • each computing unit may be as shown in FIG. 3 and FIG. 4, or as shown in FIG. 5 or FIG. 6.
  • a 3 * 3 weight matrix is used as With 3 * 3 input feature matrix as Describe as an example.
  • control signal indicates a convolution operation.
  • a 3 * 3 weight matrix Is a convolution kernel.
  • a process of performing input feature data and weight loading and convolution calculation is as follows.
  • the weights are set in advance Load into pulsation array.
  • the left-to-right flow described in the following embodiments corresponds to the first dimension (row) flow along the pulsation array described in the above embodiments.
  • the top-to-bottom flow described in the following embodiments corresponds to the second dimension (column) flow along the pulsation array described in the above embodiments.
  • the input feature data a11 enters the calculation unit C00, where the input feature data a11 is loaded from the left side of the calculation unit C00 and flows from left to right.
  • the calculation result of the calculation unit C00 is a11 * W11.
  • the calculation result a11 * W11 of the calculation unit C00 flows from the top to the bottom.
  • the input characteristic data a11 flows right into the calculation unit C01, and the calculation result a11 * W11 flows downward into the calculation unit C10; at the same time, the input characteristic data a21 is loaded into the calculation unit C00, the input characteristic data a21 is loaded into the calculation unit C10.
  • the calculation result of the calculation unit C00 is a12 * W11
  • the calculation result of the calculation unit C01 is a11 * W12
  • the calculation result of the calculation unit C10 is a11 * W11 + a21 * W21.
  • the calculation result of each calculation unit flows from the top to the bottom.
  • the input characteristic data a11 flows right into the calculation unit C02, a12 flows right into the calculation unit C01, and a21 flows right into the calculation unit C11, and the calculation result a12 * W11
  • the downward flow flows into the calculation unit C10, and the calculation result a12 * W12 flows downwards into the calculation unit C11, and the calculation result a11 * W11 + a21 * W21 flows downwards into the calculation unit C20.
  • a13 is loaded into the calculation unit C00
  • a22 is loaded into the calculation unit C10
  • a31 is loaded into the calculation unit C20.
  • the calculation result of calculation unit C00 is a13 * W11
  • the calculation result of calculation unit C01 is a12 * W12
  • the calculation result of calculation unit C02 is a11 * W13
  • the calculation result of calculation unit C10 is a12 * W11 + a22 * W21
  • the calculation result of the calculation unit C11 is a11 * W12 + a21 * W22
  • the calculation result of the calculation unit C20 is a11 * W11 + a21 * W21 + a31 * W31.
  • the input characteristic data a12 flows right into the calculation unit C02, a13 flows right into the calculation unit C01, a21 flows right into the calculation unit C12, and a22 flows right into Computing units C11, a31 flow right into computing unit C21.
  • the calculation results of the calculation units C00, C01, C02, C10, and C11 flow down into their respective lower-level calculation units.
  • a23 is loaded into the calculation unit C10, and a32 is loaded into the calculation unit C20.
  • the calculation result of calculation unit C01 is a13 * W12
  • the calculation result of calculation unit C02 is a12 * W13
  • the calculation result of calculation unit C10 is a13 * W11 + a23 * W21
  • the calculation of calculation unit C11 The result is a12 * W12 + a22 * W22
  • the calculation result of calculation unit C12 is a11 * W13 + a21 * W23
  • the calculation result of calculation unit C20 is a12 * W11 + a22 * W21 + a32 * W31
  • the calculation result of calculation unit C21 For a11 * W12 + a21 * W22 + a31 * W32, the calculation results of each calculation unit flow from top to bottom.
  • a13 flows right into the calculation unit C02
  • a22 flows right into the calculation unit C12
  • a23 flows right into the calculation unit C11
  • a31 flows right into the calculation unit C22
  • A32 flows right into computing unit C21.
  • the calculation results of the calculation units C01, C02, C10, C11, and C12 flow down into their respective lower-level calculation units.
  • a33 is loaded into the calculation unit C20.
  • the calculation result of calculation unit C02 is a13 * W13
  • the calculation result of calculation unit C11 is a13 * W12 + a23 * W22
  • the calculation result of calculation unit C12 is a12 * W13 + a22 * W23.
  • the calculation result of unit C20 is a13 * W11 + a23 * W21 + a33 * W31
  • the calculation result of calculation unit C21 is a12 * W12 + a22 * W22 + a32 * W32
  • the calculation result of calculation unit C22 is a11 * W13 + a21 * W23 + a31 * W33
  • a23 flows right into the calculation unit C12
  • a32 flows right into the calculation unit C22
  • a33 flows right into the calculation unit C21.
  • the calculation results of the calculation units C02, C11, and C12 flow down into their respective lower-level calculation units.
  • the calculation result of calculation unit C12 is a13 * W13 + a23 * W23
  • the calculation result of calculation unit C21 is a13 * W12 + a23 * W22 + a33 * W32
  • the calculation result of calculation unit C22 is a12 * W13 + a22 * W23 + a32 * W33
  • the calculation results of each calculation unit flow from top to bottom.
  • a33 flows to the right to enter the calculation unit C22, and the calculation result a13 * W13 + a23 * W23 flows down to enter the calculation unit C22.
  • the calculation result of the calculation unit C22 is a13 * W13 + a23 * W23 + a33 * W33, and the calculation results of each calculation unit flow from the top to the bottom.
  • the processing unit 130 is configured to receive intermediate calculation results output by the calculation units C20, C21, and C22, and calculate the calculation result of the calculation unit C20 at the end of the third cycle, the calculation result of the calculation unit C21 at the end of the fifth cycle, and The calculation result of the calculation unit C22 is accumulated at the end of each cycle to obtain a data block. And weights The calculation result of the convolution operation.
  • control signal indicates an average pooling operation.
  • a 3 * 3 weight matrix Can indicate that the size of the pooling window is 3 * 3, the so-called weight matrix here
  • the average pooling operation should be calculated as: (a11 + a12 + a13 + a21 + a22 + a23 + a31 + a32 + a33) / 9.
  • the process of performing input feature data and weight loading and average pooling calculations is similar to the process of performing convolution operations described above, except that the weight matrix
  • the value of And the processing unit 130 calculates the calculation result of the calculation unit C20 at the end of the third cycle, the calculation result of the calculation unit C21 at the end of the fifth cycle, and the calculation unit C22 at the end of the seventh cycle based on the 3 * 3 pooling window.
  • the calculation results are averaged to get the data block
  • the calculation result of the average pooling operation (a11 + a12 + a13 + a21 + a22 + a23 + a31 + a32 + a33) / 9.
  • control signal indicates the maximum pooling operation.
  • the process of performing input feature data and weight loading and maximum pooling calculation is as follows.
  • Weighting (Value ) Is loaded into the pulsation array (not shown in Figures 15 to 21).
  • the input feature data a11 enters the calculation unit C00, where the input feature data a11 is loaded from the left side of the calculation unit C00 and flows from left to right.
  • the calculation result of the calculation unit C00 is Max (a11), that is, a11.
  • the calculation result Max (a11) of the calculation unit C00 flows from the top to the bottom.
  • the calculation result of the calculation unit C00 is Max (a12)
  • the calculation result of the calculation unit C01 is Max (a11)
  • the calculation result of the calculation unit C10 is Max (a11, a21).
  • the calculation result of each calculation unit flows from the top to the bottom.
  • the calculation result of calculation unit C00 is Max (a13)
  • the calculation result of calculation unit C01 is Max (a12)
  • the calculation result of calculation unit C02 is Max (a11)
  • the calculation result of calculation unit C10 Is Max (a12, a22)
  • the calculation result of the calculation unit C11 is Max (a11, a21)
  • the calculation result of the calculation unit C20 is Max (Max (a11, a21), a31).
  • the calculation result of each calculation unit flows from the top to the bottom.
  • the calculation result of calculation unit C01 is Max (a13)
  • the calculation result of calculation unit C02 is Max (a12)
  • the calculation result of calculation unit C10 is Max (a13, a23)
  • the calculation unit of C11 The calculation result is Max (a12, a22)
  • the calculation result of calculation unit C12 is Max (a11, a21)
  • the calculation result of calculation unit C20 is Max (Max (a12, a22), a32)
  • the calculation result of calculation unit C21 is Max (Max (a11, a21), a31)
  • the calculation result of each calculation unit flows from the top to the bottom.
  • the calculation result of calculation unit C02 is Max (a13)
  • the calculation result of calculation unit C11 is Max (a13, a23)
  • the calculation result of calculation unit C12 is Max (a12, a22)
  • the calculation unit The calculation result of C20 is Max (Max (a13, a23), a33)
  • the calculation result of calculation unit C21 is Max (Max (a12, a22), a32)
  • the calculation result of calculation unit C22 is Max (Max (a11, a21) ), A31), the calculation results of each calculation unit flow from the top to the bottom.
  • the calculation result of the calculation unit C12 is Max (a13, a23)
  • the calculation result of the calculation unit C21 is Max (Max (a13, a23), a33)
  • the calculation result of the calculation unit C22 is Max ( Max (a12, a22), a32)
  • the seventh cycle see FIG. 21.
  • the loading and flow of input feature data and the flow of calculation results are similar to those in FIG. 14.
  • the calculation result of the calculation unit C22 is Max (Max (a13, a23), a33), and the calculation results of each calculation unit flow from the top to the bottom.
  • the processing unit 130 is configured to receive intermediate calculation results output by the calculation units C20, C21, and C22, and calculate the calculation result of the calculation unit C20 at the end of the third cycle, the calculation result of the calculation unit C21 at the end of the fifth cycle, and the seventh
  • the calculation result of the calculation unit C22 is accumulated at the end of each cycle to obtain a data block.
  • the calculation result of the maximum pooling operation is configured to receive intermediate calculation results output by the calculation units C20, C21, and C22, and calculate the calculation result of the calculation unit C20 at the end of the third cycle, the calculation result of the calculation unit C21 at the end of the fifth cycle, and the seventh. The calculation result of the calculation unit C22 is accumulated at the end of each cycle to obtain a data block.
  • the calculation result of the maximum pooling operation is configured to receive intermediate calculation results output by the calculation units C20, C21, and C22, and calculate the calculation result of the calculation unit C20 at the end of the third cycle, the calculation result of the calculation unit C21 at the end
  • the pulsation array-based neural network processing device can implement both convolution operations and pooling operations (average pooling and maximum pooling). Specifically, the convolution operation and the pooling operation can share a set of data channels. Through the indication of the control signal and / or modifying the weight value, the same pulsation array and processing unit can realize the convolution operation and the pooling. operating. Therefore, in this application, the pooling operation and the convolution operation reuse a set of hardware devices, thereby improving hardware utilization and saving costs.
  • the size of the convolution kernel or pooling window is not necessarily 3 * 3; the dimensions of the input feature matrix (that is, the input feature map) are not necessarily two-dimensional, and may be three-dimensional; the dimensions of the convolution kernel May also be 3D.
  • calculations may be performed for multiple 2-dimensional convolution kernels, or this may be calculated for multiple 3-dimensional convolution kernels.
  • the neural network processing device provided in this application can also implement the neural network operation in the above scenario.
  • the neural network processing device provided by the present application is used to implement a convolution operation for nine 3 * 3 convolution kernels.
  • the weight mapping method is: tile mapping.
  • a pulsation array with a size of 9 * 9 can map exactly to the weight matrix of 9 3 * 3 filters, and each weight matrix is directly mapped without the need to flip.
  • the method for loading the input feature data in the input feature map into the pulsation array is: each row of the input feature data in the input feature map is sequentially input to each row of the pulsation array. Each cycle, one row of input feature data is input to the leftmost column of calculation units (MAC) in the pulsation array, and each calculation unit propagates the input feature data that has been saved to the right to adjacent calculation units. So in the column direction, the input feature data received by each computing unit comes from the computing unit to its left. In the row direction, two adjacent rows, the lower row is always delayed by one cycle from the upper row.
  • MAC calculation units
  • an embodiment of the present application further provides a pooling processing device 2300 based on a pulsation array.
  • the device 2300 is used to perform average pooling or maximum pooling.
  • the device 2300 includes a control unit 2310, a pulsation array 2320, and a processing unit 2330.
  • the control unit 2310 is configured to send a control signal to the pulsation array 2320 and the processing unit 2330, where the control signal indicates an average pooling operation or a maximum operation.
  • the pulsation array 2320 includes a plurality of calculation units 3000.
  • the internal structure of each calculation unit 3000 in the pulsation array 2320 is the same.
  • Each calculation unit 3000 in the pulsation array 2320 is configured to receive a control signal, input characteristic data, and a first intermediate processing result from a higher-level calculation unit along the second dimension of the pulsation array 2320; And the first intermediate processing result performs pooling-related operations to obtain a second intermediate processing result; sending the second intermediate processing result to the next-level computing unit along the second dimension of the pulsating array 2320; The next-level computing unit of one dimension sends the input feature data.
  • the calculation unit along the input end in the first dimension in the pulsation array 2320 it may receive the input feature value from the feature data memory 2340 outside the pulsation array 2320.
  • the remaining calculation units all receive input feature values from the upper-level calculation unit along the first dimension.
  • the main processor determines that the input feature data in the pulsation array 2320 needs to be loaded, and then loads the input feature data from the feature data storage 2340 into the pulsation array 2320.
  • the main processor may be the control unit 2310, or may be another processor in the neural network system.
  • the neural network processing device 2300 provided in this application includes a feature data storage 2340.
  • the neural network processing device 2300 provided in the present application does not include the feature data storage 2340, but includes a feature data storage 2340.
  • the processing unit 2330 is connected to the calculation unit 3000 at the output end of the pulsation array 2320.
  • the processing unit 2230 is configured to receive the control signal and the second intermediate processing result output by the calculation unit at the output of the pulsation array. Operation to get the pooled output.
  • the calculation unit at the output of the pulsation array 2320 refers to the calculation unit that finally outputs the intermediate processing result in the pulsation array 2320.
  • the calculation unit of the last row of the pulsation array 2320 it refers to the calculation unit of the last row of the pulsation array 2320.
  • the sending interface of the computing unit at the output of the pulsation array 2320 is communicatively connected to the receiving interface of the processing unit 2330.
  • the processing unit 2330 When the control signal indicates an average pooling operation, the processing unit 2330 performs an averaging operation on the second intermediate processing result. When the control signal indicates the maximum pooling operation, the processing unit 2330 performs a comparison operation on the second intermediate processing result.
  • control unit 2310 is configured to control the pulsation array 2320 and the processing unit 2330 to perform an average pooling operation or a maximum pooling operation, which can improve the hardware utilization rate to a certain extent.
  • the pooling processing device provided in the present application can reuse a set of hardware devices by making the average pooling operation and the maximum pooling operation, thereby improving hardware utilization and saving costs, and can also effectively reduce the process of pooling operations. The number of times data was read.
  • the internal structure of the computing unit 3000 can be implemented in various ways.
  • each computing unit 3000 is as follows.
  • the feature register is used to store the input feature data and is also used to forward the input feature data to a subsequent-stage computing unit along the first dimension of the pulsation array 2320.
  • the adder is connected to the characteristic register.
  • the comparator is connected to the characteristic register.
  • the adder is used to receive the input feature data and the first intermediate processing result from the previous-stage computing unit along the second dimension, and accumulate the input feature data and the first intermediate processing result. To obtain a second intermediate processing result, and send the second intermediate processing result to a subsequent-level computing unit along the second dimension.
  • the comparator is configured to receive the input feature data and the first intermediate result, and output the maximum value of the input feature data and the first intermediate result to the next-level calculation unit along the second dimension.
  • the second intermediate processing result is sent, and the second intermediate processing result is the maximum value.
  • each computing unit 3000 is as follows.
  • the feature register is used to store the input feature data and is also used to forward the input feature data to a subsequent-stage computing unit along the first dimension of the pulsation array 2320.
  • the adder is connected to the characteristic register.
  • the selector is connected to the characteristic register and the adder.
  • the adder is used to receive the input feature data and the first intermediate processing result from the previous-stage computing unit along the second dimension, and accumulate the input feature data and the first intermediate processing result. To obtain a second intermediate processing result, and send the second intermediate processing result to a subsequent-level computing unit along the second dimension.
  • the adder is used for receiving the input feature data and the first intermediate processing result, and outputting the subtraction result of the product and the first intermediate processing result;
  • the selector is used for receiving the input feature data, the subtraction result, and The first intermediate processing result, the maximum value is selected from the input feature data and the first intermediate processing result according to the subtraction result, and the second intermediate processing result is sent to the next-level calculation unit along the second dimension, and the second intermediate processing The result is the maximum.
  • the processing unit 2330 is configured to: when the control signal indicates the average pooling operation, based on the size of the pooling window, average the received second intermediate processing results to obtain an average pooling operation Or when the control signal indicates the maximum pooling operation, comparing the received second intermediate processing result to obtain the output result of the maximum pooling operation.
  • the neural network processing device 2300 provided in the present application can reuse a set of hardware devices by making the average pooling operation and the maximum pooling operation, thereby improving hardware utilization, saving costs, and effectively reducing the pooling operation process. The number of times data was read.
  • the neural network processing device provided in the embodiment of the present application can be implemented on a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC).
  • FPGA field-programmable gate array
  • ASIC application-specific integrated circuit
  • An embodiment of the present invention also provides a method for controlling a computing unit.
  • the computing unit may be, for example, the computing unit 200 described in any of the above embodiments.
  • This control method is executed by the control unit 110 described in the above embodiment.
  • the control method may include a processing flow performed by the control unit 110 as described above. To avoid repetition, details are not described herein.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be from a website site, computer, server, or data center Transmission by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) to another website site, computer, server, or data center.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, and the like that includes one or more available medium integration.
  • the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a digital video disc (DVD)), or a semiconductor medium (for example, a solid state disk (SSD)), etc. .
  • a magnetic medium for example, a floppy disk, a hard disk, a magnetic tape
  • an optical medium for example, a digital video disc (DVD)
  • DVD digital video disc
  • SSD solid state disk
  • the disclosed systems, devices, and methods may be implemented in other ways.
  • the device embodiments described above are only schematic.
  • the division of the unit is only a logical function division.
  • multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, which may be electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objective of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one computing unit, or each unit may exist separately physically, or two or more units may be integrated into one unit.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Neurology (AREA)
  • Complex Calculations (AREA)

Abstract

提供一种神经网络处理装置，该装置包括：脉动阵列；控制单元，用于发送指示卷积操作、平均池化操作或最大化操作的控制信号；脉动阵列中各个计算单元用于，接收控制信号、输入特征数据、权重以及来自沿脉动阵列的第二维度的上一级计算单元的第一中间处理结果；根据控制信号，对输入特征数据、权重以及第一中间处理结果进行相关操作，获得第二中间处理结果；向沿着第二维度的下一级计算单元发送第二中间处理结果；向沿着第一维度的下一级计算单元发送输入特征数据；处理单元，用于接收控制信号以及脉动阵列输出端的计算单元输出的第二中间处理结果，根据控制信号对第二中间处理结果进行相关操作，获得神经网络输出结果。可以降低硬件开销。

Description

基于脉动阵列的神经网络处理装置
版权申明
本专利文件披露的内容包含受版权保护的材料。该版权为版权所有人所有。版权所有人不反对任何人复制专利与商标局的官方记录和档案中所存在的该专利文件或者该专利披露。
技术领域
本申请涉及神经网络领域，并且更为具体地，涉及一种基于脉动阵列的神经网络处理装置。
背景技术
随着神经网络复杂度的不断提高，神经网络技术在实际应用过程中存在占用资源多、运算速度慢、能量消耗大等问题。采用硬件加速器替代传统软件计算的方法成为提高神经网络计算效率的行之有效的方式，例如利用通用图形处理器、专用处理器芯片和现场可编程逻辑阵列实现的神经网络处理器。
当前技术，针对神经网络中的卷积操作设计一套独立的硬件装置，针对池化操作设计一套独立的硬件装置，或者，分别针对平均池化操作与最大池化操作，设计一套独立的硬件装置。
现有方案，在一定程度上增加了神经网络的硬件开销与功耗。
发明内容
本申请提供一种基于脉动阵列的神经网络处理装置，可以有效降低神经网络的硬件开销与功耗。
第一方面，提供一种神经网络处理装置，该装置包括：脉动阵列，包括多个计算单元；控制单元，用于发送控制信号，该控制信号指示如下神经网络操作中的任一种：卷积操作、平均池化操作以及最大化操作；该脉动阵列中的各个计算单元用于，接收该控制信号、输入特征数据、权重以及来自沿该脉动阵列的第二维度的上一级计算单元的第一中间处理结果；根据该控制信号，对该输入特征数据、该权重以及该第一中间处理结果进行神经网络相关操作，获得第二中间处理结果；向沿着该脉动阵列的该第二维度的下一级计算单元发送该第二中间处理结果；向沿着该脉动阵列的第一维度的下一级 计算单元发送该输入特征数据；处理单元，与该脉动阵列输出端的计算单元连接，该处理单元用于接收该控制信号以及该脉动阵列输出端的计算单元输出的第二中间处理结果，根据该控制信号对该第二中间处理结果进行神经网络相关操作，获得神经网络输出结果。
第二方面，提供一种池化处理装置，该装置包括：脉动阵列，包括多个计算单元；控制单元，用于发送控制信号，该控制信号指示平均池化操作或最大化操作；该脉动阵列中的各个计算单元用于，接收该控制信号、输入特征数据以及来自沿该脉动阵列的第二维度的上一级计算单元的第一中间处理结果；根据该控制信号，对该输入特征数据以及该第一中间处理结果进行池化相关操作，获得第二中间处理结果；向沿着该脉动阵列的该第二维度的下一级计算单元发送该第二中间处理结果；向沿着该脉动阵列的第一维度的下一级计算单元发送该输入特征数据；处理单元，与该脉动阵列输出端的计算单元连接，该处理单元用于接收该控制信号以及该脉动阵列输出端的计算单元输出的第二中间处理结果，根据该控制信号对该第二中间处理结果进行池化相关操作，获得池化输出结果。
第三方面，提供一种芯片，该芯片上集成第一方面提供的装置或集成第二方面提供的装置。
第四方面，提供一种控制方法，该控制方法由第一方面提供的装置中的控制单元执行，该方法包括第一方面中的控制单元执行的相关流程。
第五方面，提供一种控制装置，该控制装置包括存储器和处理器，该存储器用于存储指令，该处理器用于执行该存储器存储的指令，并且对该存储器中存储的指令的执行使得该处理器执行第四方面提供的方法。
第六方面，提供一种计算机可读存储介质，其上存储有计算机程序，该计算机程序被计算机执行时使得该计算机实现第四方面提供的方法。
第七方面，提供一种包含指令的计算机程序产品，该指令被计算机执行时使得该计算机实现第四方面提供的方法。
附图说明
图1为根据本申请实施例的基于脉动阵列的神经网络处理装置的示意性框图。
图2为采用本申请实施例提供的基于脉动阵列的神经网络处理装置实现 神经网络操作的示意性流程图。
图3至图6为本申请实施例中的计算单元的内部结构的示意图。
图7至图14为采用本申请实施例提供的神经网络处理装置实现卷积操作或平均池化操作的时序示意图。
图15至图21为采用本申请实施例提供的神经网络处理装置实现最大池化操作的时序示意图。
图22为采用本申请实施例提供的神经网络处理装置实现针对多个卷积核的卷积操作的示意图。
图23为根据本申请另一实施例提供的基于脉动阵列的池化处理装置的示意性框图。
具体实施方式
下面将结合附图，对本申请实施例中的技术方案进行描述。
除非另有定义，本文所使用的所有的技术和科学术语与属于本申请的技术领域的技术人员通常理解的含义相同。本文中在本申请的说明书中所使用的术语只是为了描述具体的实施例的目的，不是旨在于限制本申请。
图1为根据本申请实施例的神经网络处理装置100的示意性框图。该装置100包括控制单元110、脉动阵列120、处理单元130。
控制单元110，用于向脉动阵列120和处理单元130发送控制信号，控制信号指示如下神经网络操作中的任一种：卷积操作、平均池化操作以及最大化操作。
脉动阵列120，包括多个计算单元200。脉动阵列120中的每个计算单元200的内部结构相同，下文将详细进行描述。
脉动阵列120中的各个计算单元200用于，接收控制信号、输入特征数据、权重以及来自沿脉动阵列的第二维度的上一级计算单元的第一中间处理结果；根据控制信号，对输入特征数据、权重以及第一中间处理结果进行神经网络相关操作，获得第二中间处理结果；向沿着脉动阵列的第二维度的下一级计算单元发送第二中间处理结果；向沿着脉动阵列的第一维度的下一级计算单元发送输入特征数据。
当控制信号指示卷积操作时，计算单元200对输入特征数据、权重以及第一中间处理结果进行的神经网络相关操作包括乘法运算和加法运算。当控 制信号指示平均池化操作时，计算单元200对输入特征数据、权重以及第一中间处理结果进行的神经网络相关操作包括加法运算。当控制信号指示最大池化操作时，计算单元200对输入特征数据、权重以及第一中间处理结果进行的神经网络相关操作包括比较运算。
第一维度对应于脉动阵列的行，第二维度对应于脉动阵列的列。或者第一维度对应于脉动阵列的列，第二维度对应于脉动阵列的行。
为了便于理解与描述，下文某些实施例中以第一维度对应于脉动阵列的行，第二维度对应于脉动阵列的列为例进行描述。例如，在图1所示实施例中，第一维度对应于脉动阵列120的行，第二维度对应于脉动阵列140的列。
下文某些实施例中也会出现“从左到右”与“从上到下”的描述，其中“从左到右”表示沿着脉动阵列的行从左到右，“从上到下”表示沿着脉动阵列的列从上到下。
如图1所示，对于脉动阵列120中沿着第一维度上的输入端的计算单元来说，其可以从脉动阵列120之外的特征数据存储器140中接收输入特征值。其余计算单元均是从沿着第一维度的上一级计算单元接收输入特征值。
如图1所示，脉动阵列120中各个计算单元均可以从脉动阵列120之外的权重存储器150中接收权重。
可选地，由主处理器确定需要载入脉动阵列120中的输入特征数据和权重，然后将该输入特征数据从特征数据存储器140载入到脉动阵列120中，将该权重从权重存储器150载入到脉动阵列120中。其中，该主处理器可以为控制单元110，也可以为神经网络系统中的了解脉动阵列的计算规律的其它处理器。
可选地，本申请提供的神经网络处理装置100包括特征数据存储器140和权重存储器150。
可选地，本申请提供的神经网络处理装置100不包括特征数据存储器140和权重存储器150，但是包括用于与特征数据存储器140和权重存储器150进行通信的通信接口。
在本实施例中，神经网络处理装置100还可以包括一个缓存单元，该缓存单元用于对从通信接口传递过来的数据进行缓存。也就是说，特征数据存储器140和权重存储器150中的数据通过该缓存单元，进入脉动阵列120中。应理解，利用缓存单元缓存从通信接口传递过来的数据，有助于保证进入脉 动阵列120的数据的时序的正确性。
处理单元130，与脉动阵列120输出端的计算单元200连接，处理单元130用于接收控制信号以及脉动阵列输出端的计算单元输出的第二中间处理结果，根据控制信号对第二中间处理结果进行神经网络相关操作，获得神经网络输出结果。
本文中描述的某个单元与某个单元连接，包括但不限于有线通信连接和无线通信连接。
脉动阵列120输出端的计算单元指的是，脉动阵列120中最终输出中间处理结果的计算单元，例如，在图1中，指的是，脉动阵列120的最后一行的计算单元。
例如，脉动阵列120输出端的计算单元的发送接口与处理单元130的接收接口通信连接。
当控制信号指示卷积操作时，处理单元130对第二中间处理结果进行累加运算。当控制信号指示平均池化操作时，处理单元130对第二中间处理结果进行求平均运算。当控制信号指示最大池化操作时，处理单元130对第二中间处理结果进行比较运算。
上述可知，控制单元110用于控制脉动阵列120和处理单元130进行神经网络相关操作的类型，即卷积操作、平均池化操作或最大池化操作，从而使得本申请提供的装置100既可以实现卷积操作，又可以实现池化操作。
因此，本申请提供的方案，通过使得卷积运算和池化运算复用一套硬件装置，从而可以提高硬件利用率，节省成本。
需要说明的是，在本申请中，在脉动阵列中脉动(在脉动阵列的各个计算单元之间转发)的元素包括输入特征数据与计算单元计算得到的中间处理结果，不包括权重。换言之，对于一次神经网络操作，权重被载入计算单元后，不再被转发。
应理解，本文中提及的第一中间处理结果与第二中间处理结果仅为了区分同一个计算单元接收与发送的中间处理结果，还可以采用其它描述来区分同一个计算单元接收与发送的中间处理结果，本申请对此不做限定。
还应理解，当一个计算单元X为脉动阵列沿着第一维度的输入端的计算单元时，该计算单元X不存在沿着第一维度的前一级计算单元，即该计算单元X从外部存储器中获取输入特征数据。当一个计算单元Y为脉动阵列沿 着第二维度的输入端的计算单元时，该计算单元Y不存在沿着第二维度的前一级计算单元，即该计算单元Y的用于接收上一级计算单元的第一中间处理结果的端口的输入为空或者为零。当一个计算单元Z为脉动阵列沿着第二维度的输出端的计算单元时，该计算单元Z不存在沿着第二维度的后一级计算单元，该计算单元Z与处理单元130连接，用于将获得的第二中间处理结果发送到处理单元130。
应理解，在采用本申请提供的装置100实现卷积操作时，载入到脉动阵列120中的权重就是卷积核。
可选地，在控制信号指示卷积操作时，权重在脉动阵列120中的分布方式与权重在卷积核中的分布方式一致或呈镜像。
例如，卷积核对应的二维矩阵为
Figure PCTCN2018100335-appb-000001
则载入到脉动阵列120中的权重在脉动阵列120中的分布方式可以为
Figure PCTCN2018100335-appb-000002
也可以为
Figure PCTCN2018100335-appb-000003
由主处理器(例如控制单元110或神经网络系统中的其他处理器)从权重存储器中读取相应的权重载入到脉动阵列120中，其中，权重的寻址以及权重在脉动阵列120中的分布方式均可以根据实际需求来确定，本申请对此不做限定，只要能够实现卷积操作或池化操作即可。
池化操作仅针对输入特征数据，并不涉及权重，因此，为了能够使计算单元200可以兼容卷积运算和池化运算，在控制信号指示最大池化操作或平均池化操作时，将权重的值配置为1。
可选地，在一些实施例中，装置100还包括如下单元。
特征数据存储器(如图1中所示的特征数据存储器140)，用于存储输入特征数据。
第一权重存储器(如图1中所示的权重存储器150)和第二权重存储器(图1中未示出)。当控制信号指示卷积操作时，第一权重存储器用于存储当前卷积操作的卷积核，第二权重存储器用于预先存储下一次卷积操作的卷 积核。
控制单元110用于将第一权重存储器中的权重载入脉动阵列120中的计算单元，即载入沿着脉动阵列120的第一维度的输入端的计算单元中。
本实施例提供的装置，在执行卷积操作时，通过预先存储当前卷积操作的下一次卷积操作的卷积核，从而在利用脉动阵列完成当前卷积操作的相关运算后，可以立即开始下一次卷积操作的运算，从而可以提高卷积操作的效率。
作为示例，如图2所示，采用如图1所示的基于脉动阵列的装置100处理神经网络的过程如下。
S210，控制单元110确定神经网络操作类型，神经网络操作类型为下列操作中的任一种：卷积操作、平均池化操作以及最大池化操作。
S220，控制单元110根据神经网络操作类型，对存储单元(如图1中所示的特征存储器与权重存储器)寻址，读取并解析下一步需要执行的指令。
S230，控制单元110根据解析指令得到存储地址，从存储单元中获取输入数据(输入特征数据和权重)，将输入特征数据和权重分别载入至脉动阵列中。
S240，控制单元110向脉动阵列120中的各个计算单元200发送控制信号，该控制信号指示神经网络操作类型。
S250，脉动阵列120中的各个计算单元200基于控制信号，对接收的输入特征数据与权重执行相关运算，获得第二中间处理结果，并向沿着第二维度的下一级计算单元发送该第二中间处理结果。下文将详细描述步骤S250。
S260，控制单元120向处理单元130发送控制信号，该控制信号指示神经网络操作类型。
S270，处理单元130从脉动阵列120输出端的计算单元200获取第二中间处理结果，并基于控制信号，对该第二中间处理结果执行相关运算，获得神经网络的输出结果。下文将详细描述步骤S270。
可选地，处理单元130输出的神经网络的输出结果可以被存储到神经网络系统的输出存储单元中。
以脉动阵列中的某一个计算单元200为例，上述步骤S250包括如下内容。
当控制信号指示卷积操作或平均池化操作时，计算单元200对输入特征 数据与权重进行相乘，获得乘积，对该乘积与第一中间处理结果进行累加，获得累加结果，该累加结果为第二中间处理结果。
当控制信号指示最大池化操作时，计算单元200对输入特征数据与第一中间处理结果进行比较，获得二者中的最大值，该最大值为第二中间处理结果。或者，计算单元200对输入特征数据与权重进行相乘，获得乘积，对该乘积与第一中间处理结果进行比较，获得二者中的最大值，该最大值为第二中间处理结果。
需要说明的是，当控制信号指示池化操作(平均池化或最大池化)时，载入脉动阵列中的权重的值均为1。
应理解，当执行步骤S250的计算单元为脉动阵列中沿着第二维度输出端的计算单元，则向处理单元130发送第二中间处理结果。
以处理单元130接收到第二中间处理结果为某一个卷积核对应的卷积操作的中间结果为例，或者，以处理单元130接收到第二中间处理结果为某一个池化窗对应的池化操作的中间结果为例，步骤S270具体包括如下内容。
当控制信号指示卷积操作时，处理单元130对接收的多个第二中间处理结果进行累加，获得一个卷积核的计算结果。
当控制信号指示平均池化操作时，处理单元130基于池化窗的大小(k*k)，对接收的多个第二中间处理结果进行求平均，获得一个池化窗的平均池化计算结果。
当控制信号指示最大池化操作时，处理单元130对接收的多个第二中间处理结果进行比较，获得其中的最大值，则获得了一个池化窗的最大池化计算结果。
在本申请实施例中，脉动阵列中的每个计算单元采用相同的内部结构。本申请中，计算单元的内部结构可以有多种实现方式。
图3和图4是计算单元200的一种内部结构的示意图。图3和图4分别是计算单元200基于同一种内部结构的两种工作模式。
图5和图6是计算单元200的另一种内部结构的示意图。图5和图6分别是计算单元200基于同一种内部结构的两种工作模式。
作为一种实现方式，计算单元200的内部结构如图3(或图4)所示。
特征寄存器201用于存储输入特征数据，还用于向沿着脉动阵列的第一维度的后一级计算单元转发输入特征数据。
权重寄存器202用于存储权重。
乘法器203与特征寄存器201和权重寄存器202连接，乘法器203用于输出权重与输入特征数据的乘积。
加法器204与乘法器203连接。
比较器205，与乘法器203连接。
基于这种内部结果，计算单元200具有两种工作模式。
具体地，计算单元200接收控制信号，当解析获得控制信号指示卷积操作或平均池化操作时，开启工作模式一，如图3中所示。加法器204用于，接收乘法器203输出的乘积与来自沿着第二维度的前一级计算单元的第一中间处理结果，对乘积与第一中间处理结果进行累加，获得第二中间处理结果，向沿着第二维度的后一级计算单元发送第二中间处理结果。在工作模式一，比较器205所在的电路是无效的(如图3中虚线标注的电路)。
计算单元200接收控制信号，当解析获得控制信号指示最大池化操作时，开启工作模式二，如图4所示。比较器205用于接收乘法器203输出的乘积与来自沿着第二维度的前一级计算单元的第一中间处理结果，并输出乘积与第一中间结果中的最大值，向沿着第二维度的后一级计算单元发送第二中间处理结果，第二中间处理结果为最大值。在工作模式二，加法器204所在的电路是无效的(如图4中虚线标注的电路)。
计算单元200还包括一个控制子单元，该控制子单元用根据控制信号切换计算单元200的工作模式。
作为另一种实现方式，计算单元200的内部结构如图5(或图6)所示。
特征寄存器401用于存储输入特征数据，还用于向沿着脉动阵列的第一维度的后一级计算单元转发输入特征数据。
权重寄存器402用于存储权重。
乘法器403与特征寄存器401和权重寄存器402连接，乘法器403用于输出权重与输入特征数据的乘积。
加法器404与乘法器403连接。
选择器405与乘法器403和加法器404连接。
基于这种内部结果，计算单元200具有两种工作模式。
具体地，计算单元200接收控制信号，当解析获得控制信号指示卷积操作或平均池化操作时，开启工作模式一，如图5中所示。加法器404用于， 接收乘法器403输出的乘积与来自沿着第二维度的前一级计算单元的第一中间处理结果，对乘积与第一中间处理结果进行累加，获得第二中间处理结果，向沿着第二维度的后一级计算单元发送第二中间处理结果。在工作模式一，选择器405所在的电路是无效的(如图5中虚线标注的电路)。
计算单元200接收控制信号，当解析获得控制信号指示最大池化操作时，开启工作模式二，如图6所示。加法器404用于，接收乘法器403输出的乘积与来自沿着第二维度的前一级计算单元的第一中间处理结果，输出乘积与第一中间处理结果的相减结果；选择器405用于接收乘法器403输出的乘积、加法器404输出的相减结果、以及第一中间处理结果，根据相减结果从乘积与第一中间处理结果中选择出最大值，并向沿着第二维度的后一级计算单元发送第二中间处理结果，第二中间处理结果为最大值。在工作模式二，加法器的404所在的某些电路是无效的(如图6中虚线标注的电路)。
例如，计算单元200还包括一个控制子单元，该控制子单元用根据控制信号切换计算单元200的工作模式。
应理解，本文中提及的乘法器可以采用乘法电路实现，加法器可以采用加法电路实现，比较器可以采用比较电路实现，选择器可以采用比较电路实现。
应理解，图3至图6仅为示例而非限定。计算单元200的内部结果还可以采用其他实现方式，本申请对比不做限定，只要能够使得计算单元200根据控制信号所指示的神经网络操作类型切换工作模式即可。
处理单元130根据控制信号所指示的神经网络操作类型，可以在多种工作模式之间切换。
在控制信号指示卷积操作时，处理单元130用于对接收的第二中间处理结果进行累加，获得卷积操作的输出结果。
在控制信号指示平均池化操作时，处理单元130用于基于池化窗的大小，对接收的第二中间处理结果进行求平均，获得平均池化操作的输出结果。
在控制信号指示最大池化操作时，处理单元130用于对接收的第二中间处理结果进行比较，获得最大池化操作的输出结果。
作为一个示例，处理单元130的内部结果包括：累加器、除法器(或乘法器)、比较器。
在控制信号指示卷积操作时，处理单元130内部的累加器工作。具体地， 该累加器用于输出多个计算单元的第二中间处理结果的累加结果，获得卷积操作的输出结果。
在控制信号指示平均池化操作时，处理单元130内部的累加器与除法器工作。具体地，该累加器用于输出多个计算单元的第二中间处理结果的累加结果，除法器用于基于池化窗口的大小，对该累加结果求平均，例如，池化窗口的大小为k*k，则除法器将该累加结果除以k*k，获得平均池化操作的输出结果。应理解，求平均也可以采用乘法器来实现，例如，池化窗口的大小为k*k，则乘法器将该累加结果乘以1/(k*k)，获得平均池化操作的输出结果。
在控制信号指示最大池化操作时，处理单元130内部的比较器工作。具体地，该比较器用于输出多个计算单元的第二中间处理结果中的最大值，获得最大池化操作的输出结果。
因此，本申请提供的方案，通过使得卷积运算和池化运算复用一套硬件装置，从而可以提高硬件利用率，节省成本。
为了更好地理解本申请提供的方案，下文结合图7至图21，以3*3的脉动阵列、3*3的权重矩阵、3*3的输入特征矩阵为例，描述本申请提供的基于脉动阵列的神经网络处理装置执行卷积操作、平均池化操作以及最大池化操作的过程。
如图7所示，假设图1中所示的脉动阵列120由计算单元C00、C01、C02、C10、C11、C12、C20、C21与C22组成，处理单元130与计算单元C20、C21与C22连接，用于根据其输出的中间计算结果获得最终的计算结果。各个计算单元的内部结构可以为图3与图4所示，或者为图5或图6所示。
本例中，以3*3的权重矩阵为
Figure PCTCN2018100335-appb-000004
与3*3的输入特征矩阵为
Figure PCTCN2018100335-appb-000005
为例进行描述。
第一种情形，控制信号指示卷积操作。
这种情形下，3*3的权重矩阵
Figure PCTCN2018100335-appb-000006
为卷积核。
对于数据块
Figure PCTCN2018100335-appb-000007
和权重
Figure PCTCN2018100335-appb-000008
的卷积操作，其计算结果应为：
a11*W11+a12*W12+a13*W13+a21*W21+a22*W22+a23*W23+a31*W31+a32*W32+a33*W33。
如图7至图14所示，采用本申请的神经网络处理装置，执行输入特征数据与权重的载入和卷积计算的过程如下。
参见图7所示，预先将权重
Figure PCTCN2018100335-appb-000009
加载到脉动阵列中。
下面实施例中描述的从左到右流动对应于上述实施例中描述的沿着脉动阵列的第一维度(行)流动。下面实施例中描述的从上到下流动对应于上述实施例中描述的沿着脉动阵列的第二维度(列)流动。
在第一个周期(T1)，参见图8所示，输入特征数据a11进入计算单元C00，其中，输入特征数据a11从计算单元C00的左边载入并从左向右流动。在第一个周期结束时，计算单元C00的计算结果为a11*W11。计算单元C00的计算结果a11*W11从上向下流动。
在第二个周期(T2)，参见图9所示，输入特征数据a11向右流动进入计算单元C01，计算结果a11*W11向下流动进入计算单元C10；同时输入特征数据a21载入至计算单元C00，输入特征数据a21载入至计算单元C10。在第二个周期结束时，计算单元C00的计算结果为a12*W11，计算单元C01的计算结果为a11*W12，计算单元C10的计算结果为a11*W11+a21*W21。各个计算单元的计算结果从上向下流动。
在第三个周期(T3)，参见图10所示，输入特征数据a11向右流动进入 计算单元C02，a12向右流动进入计算单元C01，a21向右流动进入计算单元C11，计算结果a12*W11向下流动进入计算单元C10，计算结果a12*W12向下流动进入计算单元C11，计算结果a11*W11+a21*W21向下流动进入计算单元C20。同时，a13载入计算单元C00，a22载入计算单元C10，a31载入计算单元C20。在第三个周期结束时，计算单元C00的计算结果为a13*W11，计算单元C01的计算结果为a12*W12，计算单元C02的计算结果为a11*W13，计算单元C10的计算结果为a12*W11+a22*W21，计算单元C11的计算结果为a11*W12+a21*W22，计算单元C20的计算结果为a11*W11+a21*W21+a31*W31。各个计算单元的计算结果从上向下流动。
在第四个周期(T4)，参见图11所示，输入特征数据a12向右流动进入计算单元C02，a13向右流动进入计算单元C01，a21向右流动进入计算单元C12，a22向右流动进入计算单元C11，a31向右流动进入计算单元C21。第三个周期结束后，计算单元C00、C01、C02、C10、C11的计算结果向下流动进入各自的下一级计算单元中。同时，a23载入计算单元C10，a32载入计算单元C20。在第四个周期结束时，计算单元C01的计算结果为a13*W12，计算单元C02的计算结果为a12*W13，计算单元C10的计算结果为a13*W11+a23*W21，计算单元C11的计算结果为a12*W12+a22*W22，计算单元C12的计算结果为a11*W13+a21*W23，计算单元C20的计算结果为a12*W11+a22*W21+a32*W31，计算单元C21的计算结果为a11*W12+a21*W22+a31*W32，各个计算单元的计算结果从上向下流动。
在第五个周期(T5)，参见图12所示，a13向右流动进入计算单元C02，a22向右流动进入计算单元C12，a23向右流动进入计算单元C11，a31向右流动进入计算单元C22，a32向右流动进入计算单元C21。第四个周期结束后，计算单元C01、C02、C10、C11、C12的计算结果向下流动进入各自的下一级计算单元中。同时，a33载入计算单元C20。在第五个周期结束时，计算单元C02的计算结果为a13*W13，计算单元C11的计算结果为 a13*W12+a23*W22，计算单元C12的计算结果为a12*W13+a22*W23，计算单元C20的计算结果为a13*W11+a23*W21+a33*W31，计算单元C21的计算结果为a12*W12+a22*W22+a32*W32，计算单元C22的计算结果为a11*W13+a21*W23+a31*W33，各个计算单元的计算结果从上向下流动。
在第六个周期(T6)，参见图13所示，a23向右流动进入计算单元C12，a32向右流动进入计算单元C22，a33向右流动进入计算单元C21。第五个周期结束后计算单元C02、C11、C12的计算结果向下流动进入各自的下一级计算单元中。在第六个周期结束时，计算单元C12的计算结果为a13*W13+a23*W23，计算单元C21的计算结果为a13*W12+a23*W22+a33*W32，计算单元C22的计算结果为a12*W13+a22*W23+a32*W33，各个计算单元的计算结果从上向下流动。
在第七个周期(T7)，参见图14所示，a33向右流动进入计算单元C22，计算结果a13*W13+a23*W23向下流动进入计算单元C22。在第七个周期结束时，计算单元C22的计算结果为a13*W13+a23*W23+a33*W33，各个计算单元的计算结果从上向下流动。
可知，第三个周期结束时计算单元C20的计算结果a11*W11+a21*W21+a31*W31、第五个周期结束时计算单元C21的计算结果a12*W12+a22*W22+a32*W32、以及第七个周期结束时计算单元C22的计算结果a13*W13+a23*W23+a33*W33的累加值为数据块
Figure PCTCN2018100335-appb-000010
和权重
Figure PCTCN2018100335-appb-000011
的卷积操作的计算结果。
处理单元130用于接收计算单元C20、C21与C22输出的中间计算结果，并对第三个周期结束时计算单元C20的计算结果、第五个周期结束时计算单元C21的计算结果、以及第七个周期结束时计算单元C22的计算结果作累 加，得到数据块
Figure PCTCN2018100335-appb-000012
和权重
Figure PCTCN2018100335-appb-000013
的卷积操作的计算结果。
第二种情形，控制信号指示平均池化操作。
这种情形下，3*3的权重矩阵
Figure PCTCN2018100335-appb-000014
可以表示池化窗的大小为3*3，这里所谓的权重矩阵
Figure PCTCN2018100335-appb-000015
的值为
Figure PCTCN2018100335-appb-000016
对于数据块
Figure PCTCN2018100335-appb-000017
的平均池化操作，其计算结果应为：(a11+a12+a13+a21+a22+a23+a31+a32+a33)/9。
采用本申请的神经网络处理装置，执行输入特征数据与权重的载入和平均池化计算的过程与上述的执行卷积运算的过程类似，不同的是，权重矩阵
Figure PCTCN2018100335-appb-000018
的值为
Figure PCTCN2018100335-appb-000019
以及处理单元130基于3*3的池化窗，对第三个周期结束时计算单元C20的计算结果、第五个周期结束时计算单元C21的计算结果、以及第七个周期结束时计算单元C22的计算结果求平均，得到数据块
Figure PCTCN2018100335-appb-000020
的平均池化操作的计算结果(a11+a12+a13+a21+a22+a23+a31+a32+a33)/9。
第三种情形，控制信号指示最大池化操作。
这种情形下，3*3的权重矩阵
Figure PCTCN2018100335-appb-000021
表示化窗的大小为3*3，权 重矩阵
Figure PCTCN2018100335-appb-000022
的值为
Figure PCTCN2018100335-appb-000023
对于数据块
Figure PCTCN2018100335-appb-000024
的最大池化，其计算结果应为：Max(a11，a12，a13，a21，a22，a23，a31，a32，a33)，Max()表示求最大值。
如图15至图21所示，采用本申请的神经网络处理装置，执行输入特征数据与权重的载入和最大池化计算的过程如下。
预先将权重
Figure PCTCN2018100335-appb-000025
(值为
Figure PCTCN2018100335-appb-000026
)加载到脉动阵列中(图15至图21中未示出)。
在第一个周期(T1)，参见图15所示，输入特征数据a11进入计算单元C00，其中，输入特征数据a11从计算单元C00的左边载入并从左向右流动。在第一个周期结束时，计算单元C00的计算结果为Max(a11)，即a11。计算单元C00的计算结果Max(a11)从上向下流动。
在第二个周期(T2)，参见图16所示。关于输入特征数据的载入与流动，以及计算结果的流动与图9类似，具体描述请参见上文结合图9的描述，为了简洁，这里不再赘述。在第二个周期结束时，计算单元C00的计算结果为Max(a12)，计算单元C01的计算结果为Max(a11)，计算单元C10的计算结果为Max(a11，a21)。各个计算单元的计算结果从上向下流动。
在第三个周期(T3)，参见图17所示。关于输入特征数据的载入与流动，以及计算结果的流动与图10类似，具体描述请参见上文结合图10的描述，为了简洁，这里不再赘述。在第三个周期结束时，计算单元C00的计算结果为Max(a13)，计算单元C01的计算结果为Max(a12)，计算单元C02的计算结果为Max(a11)，计算单元C10的计算结果为Max(a12，a22)，计算单元C11的计算结果为Max(a11，a21)，计算单元C20的计算结果为 Max(Max(a11，a21)，a31)。各个计算单元的计算结果从上向下流动。
在第四个周期(T4)，参见图18所示。关于输入特征数据的载入与流动，以及计算结果的流动与图11类似，具体描述请参见上文结合图11的描述，为了简洁，这里不再赘述。在第四个周期结束时，计算单元C01的计算结果为Max(a13)，计算单元C02的计算结果为Max(a12)，计算单元C10的计算结果为Max(a13，a23)，计算单元C11的计算结果为Max(a12，a22)，计算单元C12的计算结果为Max(a11，a21)，计算单元C20的计算结果为Max(Max(a12，a22)，a32)，计算单元C21的计算结果为Max(Max(a11，a21)，a31)，各个计算单元的计算结果从上向下流动。
在第五个周期(T5)，参见图19所示。关于输入特征数据的载入与流动，以及计算结果的流动与图12类似，具体描述请参见上文结合图12的描述，为了简洁，这里不再赘述。在第五个周期结束时，计算单元C02的计算结果为Max(a13)，计算单元C11的计算结果为Max(a13，a23)，计算单元C12的计算结果为Max(a12，a22)，计算单元C20的计算结果为Max(Max(a13，a23)，a33)，计算单元C21的计算结果为Max(Max(a12，a22)，a32)，计算单元C22的计算结果为Max(Max(a11，a21)，a31)，各个计算单元的计算结果从上向下流动。
在第六个周期(T6)，参见图20所示。关于输入特征数据的载入与流动，以及计算结果的流动与图13类似，具体描述请参见上文结合图13的描述，为了简洁，这里不再赘述。在第六个周期结束时，计算单元C12的计算结果为Max(a13，a23)，计算单元C21的计算结果为Max(Max(a13,a23)，a33)，计算单元C22的计算结果为Max(Max(a12，a22)，a32)，各个计算单元的计算结果从上向下流动。
在第七个周期(T7)，参见图21所示。关于输入特征数据的载入与流动，以及计算结果的流动与图14类似，具体描述请参见上文结合图14的描述，为了简洁，这里不再赘述。在第七个周期结束时，计算单元C22的计算结果 为Max(Max(a13，a23)，a33)，各个计算单元的计算结果从上向下流动。
可知，第三个周期结束时计算单元C20的计算结果Max(Max(a11，a21)，a31)、第五个周期结束时计算单元C21的计算结果Max(Max(a12，a22)，a32)、以及第七个周期结束时计算单元C22的计算结果Max(Max(a13，a23)，a33)的最大值为
Figure PCTCN2018100335-appb-000027
的最大池化操作的计算结果。
处理单元130用于接收计算单元C20、C21与C22输出的中间计算结果，并对第三个周期结束时计算单元C20的计算结果、第五个周期结束时计算单元C21的计算结果、以及第七个周期结束时计算单元C22的计算结果作累加，得到数据块
Figure PCTCN2018100335-appb-000028
的最大池化操作的计算结果。
上述可知，本申请提供的基于脉动阵列的神经网络处理装置既可以实现卷积操作，也可以实现池化操作(平均池化、最大池化)。具体地，卷积操作与池化操作可以共用一套数据通道，通过控制信号的指示和/或修改权重值，就可以使得同一脉动阵列和处理单元即可以实现卷积操作，又可以实现池化操作。因此，在本申请中，池化操作与卷积操作复用一套硬件装置，从而提高硬件利用率，节省成本。
应理解，上述图7至图21仅为示例而非限定。实际应用中，卷积核或者池化窗的大小不一定是3*3；输入特征矩阵(即输入特征图)的维度不一定是二维的，还可以能是三维的；卷积核的维度也可能是3维的。此外，实际应用中，在一次卷积运算中，可能针对多个2维的卷积核进行计算，或者，这都能对多个3维的卷积核进行计算。本申请提供的神经网络处理装置也可实现上述场景下的神经网络操作。
例如，如图22所示，采用本申请提供的神经网络处理装置实现针对9个3*3的卷积核的卷积操作。
权重映射方式为：平铺映射。
如图22所示，一个大小是9*9的脉动阵列可以刚好映射满9个3*3的滤波器(filter)的权重矩阵，并且每个权重矩阵直接映射，不需要翻转。
输入特征图中的输入特征数据载入脉动阵列的方式为：输入特征图中的每行输入特征数据按照顺序输入到脉动阵列的每行。每个周期，每行输入一个输入特征数据到脉动阵列的最左侧一列计算单元(MAC)中，同时每个计算单元把已经保存的输入特征数据向右传播到相邻的计算单元中。所以在列方向，每个计算单元接收到的输入特征数据来自于它左侧的计算单元。在行方向，相邻两行，下面一行总是延迟于上面一行一个周期。
计算时序可参考上文描述的图7至图14为了简洁，这里不再赘述。
如图23所示，本申请实施例还提供一种基于脉动阵列的池化处理装置2300。该装置2300用于执行平均池化或最大池化。该装置2300包括控制单元2310、脉动阵列2320、处理单元2330。
控制单元2310，用于向脉动阵列2320和处理单元2330发送控制信号，控制信号指示平均池化操作或最大化操作。
脉动阵列2320，包括多个计算单元3000。脉动阵列2320中的每个计算单元3000的内部结构相同。
脉动阵列2320中的各个计算单元3000用于，接收控制信号、输入特征数据以及来自沿脉动阵列2320的第二维度的上一级计算单元的第一中间处理结果；根据控制信号，对输入特征数据以及第一中间处理结果进行池化相关操作，获得第二中间处理结果；向沿着脉动阵列2320的第二维度的下一级计算单元发送第二中间处理结果；向沿着脉动阵列2320的第一维度的下一级计算单元发送输入特征数据。
如图23所示，对于脉动阵列2320中沿着第一维度上的输入端的计算单元来说，其可以从脉动阵列2320之外的特征数据存储器2340中接收输入特征值。其余计算单元均是从沿着第一维度的上一级计算单元接收输入特征值。
可选地，由主处理器确定需要载入脉动阵列2320中的输入特征数据，然后将该输入特征数据从特征数据存储器2340载入到脉动阵列2320中。其中，该主处理器可以为控制单元2310，也可以为神经网络系统中的其他处理器。
可选地，本申请提供的神经网络处理装置2300包括特征数据存储器2340。
可选地，本申请提供的神经网络处理装置2300不包括特征数据存储器2340，但是包括用于与特征数据存储器2340。
处理单元2330与脉动阵列2320输出端的计算单元3000连接，处理单元2230用于接收控制信号以及脉动阵列输出端的计算单元输出的第二中间处理结果，根据控制信号对第二中间处理结果进行池化相关操作，获得池化输出结果。
脉动阵列2320输出端的计算单元指的是，脉动阵列2320中最终输出中间处理结果的计算单元，例如，在图2300中，指的是，脉动阵列2320的最后一行的计算单元。
例如，脉动阵列2320输出端的计算单元的发送接口与处理单元2330的接收接口通信连接。
当控制信号指示平均池化操作时，处理单元2330对第二中间处理结果进行求平均运算。当控制信号指示最大池化操作时，处理单元2330对第二中间处理结果进行比较运算。
上述可知，控制单元2310用于控制脉动阵列2320和处理单元2330进行平均池化操作或最大池化操作，可以在一定程度上提高硬件利用率。
此外，利用脉动阵列实现池化运算，可以有效降低池化操作过程中读数据的次数。
因此，本申请提供的池化处理装置，可以通过使得平均池化操作和最大池化操作复用一套硬件装置，从而可以提高硬件利用率，节省成本，同时也可以有效降低池化操作过程中读数据的次数。
计算单元3000的内部结构可以有多种实现方式。
可选地，作为一种实现方式，各个计算单元3000的内部结构如下。
特征寄存器，用于存储输入特征数据，还用于向沿着脉动阵列2320的第一维度的后一级计算单元转发输入特征数据。
加法器，与特征寄存器连接。
比较器，与特征寄存器连接。
在控制信号指示平均池化操作时，加法器用于，接收输入特征数据与来自沿着第二维度的前一级计算单元的第一中间处理结果，对输入特征数据与第一中间处理结果进行累加，获得第二中间处理结果，向沿着第二维度的后一级计算单元发送第二中间处理结果。
在控制信号指示最大池化操作时，比较器用于接收输入特征数据与第一中间结果，并输出输入特征数据与第一中间结果中的最大值，向沿着第二维 度的后一级计算单元发送第二中间处理结果，第二中间处理结果为最大值。
可选地，作为另一种实现方式，各个计算单元3000的内部结构如下。
特征寄存器，用于存储输入特征数据，还用于向沿着脉动阵列2320的第一维度的后一级计算单元转发输入特征数据。
加法器，与特征寄存器连接。
选择器，与特征寄存器和加法器连接。
在控制信号指示平均池化操作时，加法器用于，接收输入特征数据与来自沿着第二维度的前一级计算单元的第一中间处理结果，对输入特征数据与第一中间处理结果进行累加，获得第二中间处理结果，向沿着第二维度的后一级计算单元发送第二中间处理结果。
在控制信号指示最大池化操作时，加法器用于，接收输入特征数据与第一中间处理结果，输出乘积与第一中间处理结果的相减结果；选择器用于接收输入特征数据、相减结果以及第一中间处理结果，根据相减结果从输入特征数据与第一中间处理结果中选择出最大值，并向沿着第二维度的后一级计算单元发送第二中间处理结果，第二中间处理结果为最大值。
可选地，在本实施例中，处理单元2330用于：在控制信号指示平均池化操作时，基于池化窗的大小，对接收的第二中间处理结果进行求平均，获得平均池化操作的输出结果；或在控制信号指示最大池化操作时，对接收的第二中间处理结果进行比较，获得最大池化操作的输出结果。
因此，本申请提供的神经网络处理装置2300，可以通过使得平均池化操作和最大池化操作复用一套硬件装置，从而可以提高硬件利用率，节省成本，同时也可以有效降低池化操作过程中读数据的次数。
需要说明的是，虽然上文按照特定顺序描述了各个步骤，但是并不意味着必须按照上述特定顺序来执行各个步骤，实际上，这些步骤中的一些可以并发执行，甚至改变顺序，只要能够实现所需要的功能即可，本申请对各个步骤的执行顺序不做严格限定。
本申请实施例提供的神经网络处理装置可以在现场可编程门阵列(field－programmable gate array，FPGA)或专门应用的集成电路(application specific integrated circuit，ASIC)上实现。
本发明实施例还提供了一种计算单元的控制方法。该计算单元例如可以是上文任一实施例描述的计算单元200。该控制方法由上述实施例描述的控 制单元110执行。该控制方法可以包括如上文中描述的由控制单元110执行的处理流程，为避免重复，此处不再详述。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其他任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本发明实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质(例如，软盘、硬盘、磁带)、光介质(例如数字视频光盘(digital video disc，DVD))、或者半导体介质(例如固态硬盘(solid state disk，SSD))等。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方， 或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个计算单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (17)

  1. 一种神经网络处理装置，其特征在于，包括：
    脉动阵列，包括多个计算单元；
    控制单元，用于发送控制信号，所述控制信号指示如下神经网络操作中的任一种：卷积操作、平均池化操作以及最大化操作；
    所述脉动阵列中的各个计算单元用于，接收所述控制信号、输入特征数据、权重以及来自沿所述脉动阵列的第二维度的上一级计算单元的第一中间处理结果；根据所述控制信号，对所述输入特征数据、所述权重以及所述第一中间处理结果进行神经网络相关操作，获得第二中间处理结果；向沿着所述脉动阵列的所述第二维度的下一级计算单元发送所述第二中间处理结果；向沿着所述脉动阵列的第一维度的下一级计算单元发送所述输入特征数据；
    处理单元，所述处理单元与所述脉动阵列输出端的计算单元连接，所述处理单元用于接收所述控制信号以及所述脉动阵列输出端的计算单元输出的第二中间处理结果，根据所述控制信号对所述第二中间处理结果进行神经网络相关操作，获得神经网络输出结果。
  2. 根据权利要求1所述的装置，其特征在于，在所述控制信号指示最大池化操作或平均池化操作时，所述权重的值为1。
  3. 根据权利要求1所述的装置，其特征在于，在所述控制信号指示卷积操作时，所述权重在所述脉动阵列中的分布方式与所述权重在卷积核中的分布方式一致或呈镜像。
  4. 根据权利要求1至3中任一项所述的装置，其特征在于，所述装置包括：
    特征数据存储器，用于存储输入特征数据；
    权重存储器，用于存储权重；
    所述控制单元用于，将所述特征数据存储器中的输入特征数据载入所述脉动阵列，将所述权重存储器中的权重载入所述脉动阵列。
  5. 根据权利要求4所述的装置，其特征在于，所述装置还包括：
    另一权重存储器，用于在所述控制信号指示卷积操作时，预存下一次卷积操作的权重。
  6. 根据权利要求1至5中任一项所述的装置，其特征在于，所述各个计算单元包括：
    权重寄存器，用于存储权重；
    特征寄存器，用于存储输入特征数据，还用于向沿着所述脉动阵列的所述第一维度的后一级计算单元转发所述输入特征数据；
    乘法器，所述乘法器与所述权重寄存器和所述特征寄存器连接，所述乘法器用于输出所述权重与所述输入特征数据的乘积；
    加法器，与所述乘法器里连接；
    在所述控制信号指示卷积操作或平均池化操作时，所述加法器用于，接收所述乘积与来自沿着所述第二维度的前一级计算单元的第一中间处理结果，对所述乘积与所述第一中间处理结果进行累加，获得第二中间处理结果，向沿着所述第二维度的后一级计算单元发送所述第二中间处理结果。
  7. 根据权利要求6所述的装置，其特征在于，
    在所述控制信号指示最大池化操作时，所述加法器用于，接收所述乘积与所述第一中间处理结果，输出所述乘积与所述第一中间处理结果的相减结果；
    所述计算单元还包括：
    选择器，与所述加法器和所述乘法器连接，在所述控制信号指示最大池化操作时，所述选择器用于接收所述乘积、所述相减结果、以及所述第一中间处理结果，根据所述相减结果从所述乘积与所述第一中间处理结果中选择出最大值，并向沿着所述第二维度的后一级计算单元发送第二中间处理结果，所述第二中间处理结果为所述最大值。
  8. 根据权利要求6所述的装置，其特征在于，所述计算单元还包括：
    比较器，与所述乘法器连接；
    在所述控制信号指示最大池化操作时，所述比较器用于接收所述乘积与所述第一中间结果，并输出所述乘积与所述第一中间结果中的最大值，向沿着所述第二维度的后一级计算单元发送第二中间处理结果，所述第二中间处理结果为所述最大值。
  9. 根据权利要求1至8中任一项所述的装置，其特征在于，所述处理单元用于：
    在所述控制信号指示卷积操作时，对接收的第二中间处理结果进行累加，获得所述卷积操作的输出结果；或
    在所述控制信号指示平均池化操作时，基于池化窗的大小，对接收的第 二中间处理结果进行求平均，获得所述平均池化操作的输出结果；或
    在所述控制信号指示最大池化操作时，对接收的第二中间处理结果进行比较，获得所述最大池化操作的输出结果。
  10. 根据权利要求1至9中任一项所述的装置，其特征在于，所述第一维度对应于所述脉动阵列的行，所述第二维度对应于所述脉动阵列的列。
  11. 一种池化处理装置，其特征在于，包括：
    脉动阵列，包括多个计算单元；
    控制单元，用于发送控制信号，所述控制信号指示平均池化操作或最大化操作；
    所述脉动阵列中的各个计算单元用于，接收所述控制信号、输入特征数据以及来自沿所述脉动阵列的第二维度的上一级计算单元的第一中间处理结果；根据所述控制信号，对所述输入特征数据以及所述第一中间处理结果进行池化相关操作，获得第二中间处理结果；向沿着所述脉动阵列的所述第二维度的下一级计算单元发送所述第二中间处理结果；向沿着所述脉动阵列的第一维度的下一级计算单元发送所述输入特征数据；
    处理单元，与所述脉动阵列输出端的计算单元连接，所述处理单元用于接收所述控制信号以及所述脉动阵列输出端的计算单元输出的第二中间处理结果，根据所述控制信号对所述第二中间处理结果进行池化相关操作，获得池化输出结果。
  12. 根据权利要求11所述的装置，其特征在于，所述装置还包括：
    特征数据存储器，用于存储输入特征数据；
    所述控制单元用于，将所述特征数据存储器中的输入特征数据载入所述脉动阵列。
  13. 根据权利要求11或12所述的装置，其特征在于，所述各个计算单元包括：
    特征寄存器，用于存储输入特征数据，还用于向沿着所述脉动阵列的所述第一维度的后一级计算单元转发所述输入特征数据；
    加法器，与所述特征寄存器连接；
    在所述控制信号指示平均池化操作时，所述加法器用于，接收所述输入特征数据与来自沿着所述第二维度的前一级计算单元的第一中间处理结果，对所述输入特征数据与所述第一中间处理结果进行累加，获得第二中间处理 结果，向沿着所述第二维度的后一级计算单元发送所述第二中间处理结果。
  14. 根据权利要求13所述的装置，其特征在于，
    在所述控制信号指示最大池化操作时，所述加法器用于，接收所述输入特征数据与所述第一中间处理结果，输出所述乘积与所述第一中间处理结果的相减结果；
    所述计算单元还包括：
    选择器，与所述特征寄存器和所述加法器连接；
    在所述控制信号指示最大池化操作时，所述选择器用于接收所述输入特征数据、所述相减结果以及所述第一中间处理结果，根据所述相减结果从所述输入特征数据与所述第一中间处理结果中选择出最大值，并向沿着所述第二维度的后一级计算单元发送第二中间处理结果，所述第二中间处理结果为所述最大值。
  15. 根据权利要求13所述的装置，其特征在于，所述计算单元还包括：
    比较器，与所述特征寄存器连接；
    在所述控制信号指示最大池化操作时，所述比较器用于接收所述输入特征数据与所述第一中间结果，并输出所述输入特征数据与所述第一中间结果中的最大值，向沿着所述第二维度的后一级计算单元发送第二中间处理结果，所述第二中间处理结果为所述最大值。
  16. 根据权利要求11至15中任一项所述的装置，其特征在于，所述处理单元用于：
    在所述控制信号指示平均池化操作时，基于池化窗的大小，对接收的第二中间处理结果进行求平均，获得所述平均池化操作的输出结果；或
    在所述控制信号指示最大池化操作时，对接收的第二中间处理结果进行比较，获得所述最大池化操作的输出结果。
  17. 根据权利要求11至16中任一项所述的装置，其特征在于，所述第一维度对应于所述脉动阵列的行，所述第二维度对应于所述脉动阵列的列。
PCT/CN2018/100335 2018-08-14 2018-08-14 基于脉动阵列的神经网络处理装置 Ceased WO2020034079A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201880038645.0A CN110785778A (zh) 2018-08-14 2018-08-14 基于脉动阵列的神经网络处理装置
PCT/CN2018/100335 WO2020034079A1 (zh) 2018-08-14 2018-08-14 基于脉动阵列的神经网络处理装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/100335 WO2020034079A1 (zh) 2018-08-14 2018-08-14 基于脉动阵列的神经网络处理装置

Publications (1)

Publication Number Publication Date
WO2020034079A1 true WO2020034079A1 (zh) 2020-02-20

Family

ID=69383057

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/100335 Ceased WO2020034079A1 (zh) 2018-08-14 2018-08-14 基于脉动阵列的神经网络处理装置

Country Status (2)

Country Link
CN (1) CN110785778A (zh)
WO (1) WO2020034079A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113792867A (zh) * 2021-09-10 2021-12-14 中科寒武纪科技股份有限公司 运算电路、芯片和板卡
CN115423085A (zh) * 2022-09-30 2022-12-02 上海励驰半导体有限公司 脉动阵列、脉动阵列系统及其运算方法、装置、存储介质
CN115423084A (zh) * 2022-09-30 2022-12-02 上海励驰半导体有限公司 脉动阵列、脉动阵列系统及其运算方法、装置、存储介质
CN116108902A (zh) * 2023-02-22 2023-05-12 成都登临科技有限公司 采样操作实现系统、方法、电子设备及存储介质

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112639836A (zh) * 2020-02-25 2021-04-09 深圳市大疆创新科技有限公司 数据处理装置、电子设备和数据处理方法
CN112100118B (zh) * 2020-08-05 2021-09-10 中科驭数（北京）科技有限公司 神经网络计算方法、装置和存储介质
CN112016522B (zh) * 2020-09-25 2022-06-07 苏州浪潮智能科技有限公司 一种视频数据处理方法、系统及相关组件
WO2022061867A1 (zh) * 2020-09-28 2022-03-31 深圳市大疆创新科技有限公司 数据处理方法、装置及计算机可读存储介质
CN112712456A (zh) * 2021-02-23 2021-04-27 中天恒星(上海)科技有限公司 Gpu处理电路结构
US11544213B2 (en) 2021-03-04 2023-01-03 Samsung Electronics Co., Ltd. Neural processor
CN116185339A (zh) * 2021-11-25 2023-05-30 Oppo广东移动通信有限公司 基于神经网络的数据处理方法、计算装置及存储介质
CN116959532A (zh) * 2023-03-09 2023-10-27 腾讯科技（深圳）有限公司 数据读取方法、装置、设备及可读存储介质
CN118709729B (zh) * 2024-08-28 2024-11-22 上海励驰半导体有限公司 针对神经网络的数据处理方法、加速器及电子设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107341544A (zh) * 2017-06-30 2017-11-10 清华大学 一种基于可分割阵列的可重构加速器及其实现方法
CN107578098A (zh) * 2017-09-01 2018-01-12 中国科学院计算技术研究所 基于脉动阵列的神经网络处理器
CN107578095A (zh) * 2017-09-01 2018-01-12 中国科学院计算技术研究所 神经网络计算装置及包含该计算装置的处理器
CN108182471A (zh) * 2018-01-24 2018-06-19 上海岳芯电子科技有限公司 一种卷积神经网络推理加速器及方法

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107169560B (zh) * 2017-04-19 2020-10-16 清华大学 一种自适应可重构的深度卷积神经网络计算方法和装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107341544A (zh) * 2017-06-30 2017-11-10 清华大学 一种基于可分割阵列的可重构加速器及其实现方法
CN107578098A (zh) * 2017-09-01 2018-01-12 中国科学院计算技术研究所 基于脉动阵列的神经网络处理器
CN107578095A (zh) * 2017-09-01 2018-01-12 中国科学院计算技术研究所 神经网络计算装置及包含该计算装置的处理器
CN108182471A (zh) * 2018-01-24 2018-06-19 上海岳芯电子科技有限公司 一种卷积神经网络推理加速器及方法

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113792867A (zh) * 2021-09-10 2021-12-14 中科寒武纪科技股份有限公司 运算电路、芯片和板卡
CN113792867B (zh) * 2021-09-10 2024-05-10 中科寒武纪科技股份有限公司 运算电路、芯片和板卡
CN115423085A (zh) * 2022-09-30 2022-12-02 上海励驰半导体有限公司 脉动阵列、脉动阵列系统及其运算方法、装置、存储介质
CN115423084A (zh) * 2022-09-30 2022-12-02 上海励驰半导体有限公司 脉动阵列、脉动阵列系统及其运算方法、装置、存储介质
CN116108902A (zh) * 2023-02-22 2023-05-12 成都登临科技有限公司 采样操作实现系统、方法、电子设备及存储介质
CN116108902B (zh) * 2023-02-22 2024-01-05 成都登临科技有限公司 采样操作实现系统、方法、电子设备及存储介质

Also Published As

Publication number Publication date
CN110785778A (zh) 2020-02-11

Similar Documents

Publication Publication Date Title
CN110785778A (zh) 基于脉动阵列的神经网络处理装置
CN111142938B (zh) 一种异构芯片的任务处理方法、任务处理装置及电子设备
CN107957976B (zh) 一种计算方法及相关产品
EP4071619B1 (en) Address generation method, related device and storage medium
CN110119807B (zh) 运算方法、装置、计算机设备和存储介质
CN103970719B (zh) 一种拟合方法及拟合装置
CN114780481B (zh) 用于深度学习的可重构处理单元
CN110598844A (zh) 一种基于fpga的并行卷积神经网络加速器及加速方法
CN116227599A (zh) 一种推理模型的优化方法、装置、电子设备及存储介质
CN110209627A (zh) 一种面向智能终端的ssd硬件加速方法
US10127040B2 (en) Processor and method for executing memory access and computing instructions for host matrix operations
CN116648694A (zh) 芯片内的数据处理方法及芯片
CN108108189B (zh) 一种计算方法及相关产品
CN111026697A (zh) 核间通信方法、系统、电子器件以及电子设备
CN113128673B (zh) 数据处理方法、存储介质、神经网络处理器及电子设备
CN111382856B (zh) 数据处理装置、方法、芯片及电子设备
CN112148453A (zh) 用于隐私计算的计算芯片及网络计算系统
US20230267002A1 (en) Multi-Instruction Engine-Based Instruction Processing Method and Processor
CN111966401A (zh) 指令处理方法、装置及相关产品
CN117474055A (zh) 神经网络算法的处理系统、数据处理方法和电子设备
CN104678815A (zh) Fpga芯片的接口结构及配置方法
CN111258641B (zh) 运算方法、装置及相关产品
CN111026440B (zh) 运算方法、装置、计算机设备和存储介质
CN111260070B (zh) 运算方法、装置及相关产品
WO2022141321A1 (zh) Dsp处理器及其并行计算方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18930450

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18930450

Country of ref document: EP

Kind code of ref document: A1