KR20180034853A - Apparatus and method test operating of convolutional neural network - Google Patents

Abstract

According to the present invention, a convolutional neural network (CNN) apparatus includes an information processing unit including a register for storing weight data, at least one channel set for performing the operations of a feature extraction layer and a classification layer based on input data and the weight data, and a CNN control unit functionally connected to the information processing unit and the channel set. The CNN control unit can control the channel set to perform the operation of the feature extraction layer and to perform the operation of the classification layer when the operation of the feature extraction layer is completed. Accordingly, the present invention can perform a test operation with high speed and low power.

Description

[0001] APPARATUS AND METHOD TEST OPERATING OF CONVOLUTIONAL NEURAL NETWORK [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus and method for computing a composite-object neural network.

Deep learning algorithms, which are in the limelight in recent years, are simple calculations (addition, multiplication) and many operations. They analyze big data and find hidden features in a data set. It can be used to help. Deep learning is a combination of multiple nonlinear transformation techniques that allows for a high level of abstraction (machine learning) that attempts to summarize key content or functions in large amounts of data or complex data, Can be defined as a set of algorithms.

The deep-running algorithm can be divided into training and test operations. Learning can be a series of processes that learn features from an input data set. The training process can learn what is called a weight, a feature or a kernel, and a test can be conducted using learned parameters. In the deep learning algorithm, the test is performed by inputting data in a format different from the input data set for training (for example, in the case of an image, a frontal view of the same type of object (e.g., a cat, a car, etc.) A task such as sorting can be performed.

Deep learning algorithms are Deep Neural Network (DNN) Such as the Convolutional Neural Network (CNN), the Recurrent Neural Network (RNN), and the Deep Belief Network (DBN).

When implementing a deep running algorithm, a simple computation in a sequential calculation method of a computer may be performed in a large number of repetitive ways, resulting in a large power consumption and a long execution time. Using a processor optimized for parallel computing (eg, a graphics processor unit (GPU)) may be good for performance, but it can be very power intensive, and every time the algorithm changes, coding can be difficult. The method of executing the deep learning algorithm through a CPU or a GPU is slower in operation speed and consumes more power than a hardware configuration.

Various embodiments of the present invention propose an apparatus and method that can perform a deep running algorithm.

Various embodiments of the present invention propose an apparatus and method for performing a test operation of a CNN algorithm for convoluting input data with training weight data.

Various embodiments of the present invention propose an apparatus and method for storing and storing input data for a convolution operation in an orderly manner using an address map in a convolution operation to speed up the operation speed.

In various embodiments of the present invention, a CNN computing device and method includes a preprocessed address map to increase the parallelism of the convolution and pulling operations and reduce the latency, and when setting input data for computation, To thereby determine input data.

A CNN computing apparatus and method according to various embodiments of the present invention proposes an apparatus and method for passing a computation operation when an operand is 0 to reduce the execution time of a convolution operation and a pooling operation.

The CNN computing apparatus and method according to various embodiments of the present invention perform a convolution operation by pre-fetching weight data used in a convolution operation to perform a convolution operation by a memory load latency An apparatus and method for reducing the multiplication delay are proposed.

A CNN computing apparatus and method according to various embodiments of the present invention proposes an apparatus and method for rapidly performing a convolution operation and a pulling operation based on a data processing method of processor elements of a convolution block.

A CNN computing apparatus and method according to various embodiments of the present invention proposes an apparatus and method for performing a dropout operation when executing a classification layer.

 A CNN apparatus according to embodiments of the present invention includes an information processing unit including a register for storing weight data, at least one channel set for performing a feature extraction layer and classification layer operation, and an information processing unit for functionally connecting Lt; / RTI > The CNN control unit may perform the operation of the feature extraction layer in the channel set, and perform the class-action layer operation when the operation of the feature extraction layer is completed.

An operation method of a CNN (convolutional neural network) apparatus according to embodiments of the present invention includes an information processing unit including a register for storing weight data, and at least one channel set including a convolution block and a pulling block, The method comprising the steps of: performing operations of a feature extraction layer by activating a convolution block and a pulling block of a channel set; and performing a class-action layer operation by activating a convolution block of the channel set when the operation of the feature extraction layer is completed .

Various embodiments of the present invention may implement a CNN (Convolutional Neural Network Algorithm) as a hardware (CNN architecture) to perform test operations at high speed and low power. The CNN accelerator according to various embodiments of the present invention can operate at low power and can perform a deep learning algorithm even in a portable device. The CNN architecture according to an embodiment of the present invention can operate a CNN (Convolution Neural Network) algorithm of various sizes through configuration.

The CNN architecture according to an embodiment of the present invention performs a zero pass function to easily perform computation in a computation block of a convolution block (PE, ADD) unlike a general computation operation. Therefore, the deep learning algorithm (deep learning algorithm) can be performed quickly. The CNN architecture according to an embodiment of the present invention can reduce the time to load weight parameters from memory by pre-fetching the weight parameters in the convolution block, learning algorithm can be performed quickly.

1 is a diagram showing a structure of a CNN feedforward algorithm of a CNN algorithm.
2 is a diagram for explaining a feedforward path flow of CNN.
3 is a diagram illustrating an operational procedure of a CNN apparatus according to various embodiments of the present invention.
4A and 4B are block diagrams of a CNN apparatus according to various embodiments of the present invention.
5 is a diagram showing a configuration of a CNN system according to various embodiments of the present invention.
6 is a diagram showing a structure of an input data set (CI reg input data set) stored in the CI reg of the input unit 530
7 is a diagram showing a configuration of a convolution block according to various embodiments of the present invention.
8A and 8B are diagrams showing a configuration of a process element of a convolution block according to various embodiments of the present invention.
Figs. 9A to 9C are diagrams showing the structures of an input data set, a weight data set, and an output data set of a convolution block.
10 is a diagram showing an input data structure of a pulling operation in a CNN apparatus according to various embodiments of the present invention.
11 is a diagram showing the structure of output data and activated data of a pulling block in a CNN apparatus according to various embodiments of the present invention.
12 is a diagram illustrating a structure of input and output data of a convolution block in a classification layer in a CNN apparatus according to various embodiments of the present invention.
13 is a diagram showing a structure of layer output data in a CNN apparatus according to various embodiments of the present invention.
14 is a diagram showing another configuration of a CNN system according to various embodiments of the present invention.
15A-15C are flow charts illustrating operation of a CNN device in accordance with various embodiments of the present invention.
16A and 16B are flow charts illustrating the operation of the convolution layer operation of a CNN device in accordance with various embodiments of the present invention.
17A and 17B are flow charts illustrating the operation of a pulling layer operation of a CNN device in accordance with various embodiments of the present invention.
18 is a flow chart illustrating the output operation of the classification layer of the CNN device according to various embodiments of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, only parts necessary for understanding the operation according to the present invention will be described, and the description of other parts will be omitted so as not to disturb the gist of the present invention.

Deep learning algorithms, which are in the limelight in recent years, are simple calculations (addition, multiplication) and many operations. They analyze big data and find hidden features in a data set. It can be used to help. Deep learning algorithms can be divided into training and test operations. Learning can be a series of processes that learn features in an input dataset. The deep learning algorithm learns what is called a weight, a feature, or a kernel in the learning process and can use learned parameters in the testing process. The test operation of the deep learning algorithm is based on input data sets for learning and other input data sets (for example, different types of the same type (for example, cat, car, etc.) It is possible to perform a task such as appropriately classifying a plurality of objects.

Deep learning algorithms can be implemented in a way that simple computation is repeated by a sequential computation method of a computer, which consumes much power and can take a long time. Using a graphics processing unit (GPU) optimized for parallel computation is good for performance but can still be very power intensive and software coding for parallel computations can be difficult every time the algorithm changes. Implementing the deep-running algorithm in hardware can result in faster processing speed and lower power consumption than running a deep-running algorithm through a CPU or GPU. Various embodiments of the present invention propose a hardware architecture capable of processing deep running algorithms.

Various embodiments of the present invention may provide a configuration and operation of a CNN hardware architecture capable of convolving input data with a set of weights of the learned algorithm to execute a puncturing algorithm.

According to one embodiment, CNN hardware architecture can input input data using pre-computed memory map without sequentially inputting input data. The CNN hardware architecture improves the parallelism of the convolution and pooling operations by inputting the memory data in a non-sequential manner with reference to the memory map rather than sequentially fetching the input data. ) Can be reduced.

According to one embodiment, the CNN hardware architecture may perform zero convolution operation and reduce the execution time of the convolution operation and the pulling operation when the operator (e.g., input data or weight data) is 0.

According to one embodiment, the CNN hardware architecture does not load the weight parameter data used in the deep learning algorithm every time in the memory, but pre-fetches the weight data to be used from the start to the end of the layer in the block performing the convolution operation. (pre-fetch) to reduce the convolution operation time (reduce the multiplication delay due to memory load latency in the convolution operation).

According to one embodiment, the CNN architecture may use a dropout method, a regularization method, to improve the performance of an algorithm in a fully connected layer.

Hereinafter, embodiments of the present invention will be described in more detail with reference to the drawings.

1 is a diagram showing a structure of a CNN feedforward algorithm of a CNN algorithm.

Referring to FIG. 1, the CNN may include a feature extraction part 110 and a classification part 120. The feature extraction unit 110 may have a configuration in which a convolution layer and a pooling layer are repeated. The feature extraction part 110 may include a pair of convolutional layers and output data processed in the pooling layer in the next pair of convolution layers < RTI ID = 0.0 > The external input data 100 may be input data of the first convolution layer of the measurement extraction part 110 and may be processed in the convolution layer and output The output data processed in the first pulling layer may be the input data of the second convolution layer in the next pair. The convolution operation and the pulling operation are performed by the number of times set in the host (for example, N times), and the pooling layer output of the last pair may be performed during the classification part 12 0) input data.

The classification part 120 may include at least one fully conned layer. The classification part 120 can perform a predetermined number of connection layer operations, and the output of the classification part 120 is completed. And may be training data 130.

2 is a diagram for explaining a feedforward path flow of CNN. Figure 2 shows an example of data processed in a pair of convolutional and pulling layers to illustrate the CNN's feedforward algorithm.

Referring to FIG. 2, the leftmost input data (Tensor) 210 may be composed of a plurality of channels, and the input data 210 of FIG. 2 is composed of three channels . The input data 210 may be represented by a width, a height, and a depth. Reference numerals 210 to 240 in FIG. 2 denote the data processed by the convolution operation and the pulling operation of one channel input data, and data of other channels can be processed in the same manner. The input data 200 may be zero padded to fit the size of the convolution window. Zero padding may mean inserting a zeros into the border of the input data 210.

The convolution layer may perform two convolution operations with the input data 210 and two weight sets 220 having the size of CxCxD. The convolution operation may be an operation of calculating data having the same input depth and weight depth of input data. For example, the convolution may be an operation in which the weight elements at the same position as the elements corresponding to the positions of the convolution window 211 of the input data are multiplied and added together. The convolution operation may be performed for all depths equally, and adding the sum of the three products yields Z (0,0,0) corresponding to reference numeral 231 in the output () 220 of the convolution layer . The mathematical expression related to this may be a formula for obtaining Fig. The weight set may be referred to as a filter and the output data depth may be determined by the number of filters as well as the number of filters. ). Reference numeral 239 denotes an example of expressing a convolution operation as an equation.

When the convolution operation is performed, output data such as 230 in FIG. 2 may be generated. When the convolution is terminated, the output data may be computed as an activation value via one of the activation functions. Reference numeral 249 denotes an example of expressing the activation function as an expression. The activation output value may be the input data of the pooling layer. The pulling operation may be performed by obtaining a maximum value or an average value among the values of the pulling layer input data in the pooling window, such as 241 or 242 in FIG. 2, and compressing the value to a single value such as 251 or 252 . Reference numeral 259 shows an example in which the pulling operation is expressed by an equation.

The convolution operation and the pulling operation can be performed for other channel data of the same input data as shown in FIG. For each channel and layer, the weight value or the number of filters may vary. The output data of the pulling operation may be input data of the next layer (next convolution layer). At this time, the number of weight sets (eight se), that is, the number of filters, may be the number of channels of the next layer input data.

The CNN apparatus according to various embodiments of the present invention can provide a new apparatus and method for performing CNN's feed forward algorithm as shown in FIGS.

3 is a diagram illustrating an operational procedure of a CNN apparatus according to various embodiments of the present invention.

Referring to FIG. 3, the CNN device may receive and store information for executing the CNN algorithm in the initialization operation 210. The received information may be weight data. The CNN device according to various embodiments of the present invention may also include layer configuration information, an address map for convolution input, an activation function lookup table (LUT) After performing the initialization operation, the CNN device may perform a feature extraction operation 320. The feature extraction operation 320 may be a convolution operation and a pulling operation. Operation and pooling operations may be performed by methods and procedures as illustrated in Figures 1 and 2. Upon completion of the feature extraction operation of the input data, the CNN device may perform the classification operation 330. [ The classification operation 330 may include at least one fully connected layer as shown in Figure 1. Upon completion of the classification operation, In the medical operation unit 340 may output the output data to the external system.

4A is a block diagram of a CNN apparatus according to various embodiments of the present invention.

Referring to FIG. 4A, the CNN apparatus may include a CNN control unit 400, an information processing unit 410, and a channel set 420. The channel set 420 may be implemented in one or more than two.

The CNN control unit 400 can perform the operation of executing the feature extraction layer and the classification layer and reconstructing the input data when executing the corresponding layer. The CNN control unit 400 may include an arbiter and an arbiter / MMU (MMU).

The information processing unit 410 may include at least one register and at least one counter. For example, the information processing unit 410 may include a weight register that stores weight data that can be used in the convolution operation. For example, the information processing unit 410 may include counters for updating the number of operations calculated according to the operation of the CNN apparatus. The information processing unit 410 may store the information transmitted from the host in the corresponding registers when the system is initialized, and may update the count value according to the operation of the CNN apparatus.

The channel set 420 may be composed of a plurality of channels. Each set of channels 420 may perform a feature extraction layer operation and a classification operation. The feature extraction layer operation of the input data of the corresponding channel can be performed and the classification layer operation can be performed upon completion of the feature extraction layer operation.

Each set of channels 420 may include an input 430, a convolution block 440, a pulling block 450, and an output 460.

The input unit 430 may be an input register for storing input data to be reconfigured in the CNN control unit 400. The input unit 430 includes a CI register (convolution in put register, CI-reg) for storing convolution input data of the feature extraction layer, and a fully connected input register (FCI register) FCI-reg).

The convolution block 440 may perform convolutional computation of the input data stored in the input unit 430 and the weight data of the information processing unit 410. The convolution block 540 may perform a convolution operation on the weight data having the same ID as the input data.

The pulling block 450 may pool the data having the same ID in the convoluted data at a specific extraction layer. The pooling block 550 may not operate at the connection layer.

The output unit 560 may store data output from the pulling block 550 in the feature extraction layer and may store data output from the convolution block 540 in the connection layer. The output unit 560 may include an LO register (LO-reg) and an activation block.

In the CNN apparatus having the above configuration, the CNN control unit 400 may store the data received from the external system in the input unit 430 when the feature extraction layer is first executed. The CNN control unit 400 reconstructs the output data of the pulling block 450 stored in the output unit 460 and stores the reconstructed output data in the input unit 430 as the convolution input data. When the feature extraction operation is completed, the CNN control unit 400 can execute the classification layer. The CNN control unit 400 reconstructs the output data calculated in the convolution layer in the classification layer and stores the reconstructed output data in the input unit 430 as input data of the connection layer. The CNN control unit 400 can output the result data to the external system upon completion of execution of the recovery layer.

The CNN apparatus according to various embodiments of the present invention includes an information processing unit 410 including a register for storing weight data, at least one channel set 420 for performing a feature extraction layer and classification layer operation, The CNN control unit 410 controls the operation of the feature extraction layer to perform the operation, and performs the class-action layer operation when the operation of the feature extraction layer is completed. Each channel set 420 includes an input unit 430 for storing input data, a convolution block 440 for convoluting input data and weight data, a pulling block 450 for pulling up convolution computed data, Or an output unit 460 for storing pooling operation data. The CNN device according to various embodiments of the present invention may drive the convolution block 440 and the pulling block 450 of the channel set 420 in the feature extraction layer operation to perform the convolution layer and the pulling layer operations. The CNN apparatus according to various embodiments of the present invention may perform a classification layer operation upon completion of the feature extraction layer operation. In the classification operation, the CNN apparatus drives the convolution block of the channel set 420 ) Are not driven) to perform the link layer operation.

4B is a block diagram of another configuration of a CNN apparatus according to various embodiments of the present invention.

Referring to FIG. 4B, the CNN apparatus may include a CNN control unit 405, an information processing unit 415, a feature extraction layer block 425, and a classification block 427. The feature extraction layer block 425 may be implemented as one or more than two.

The CNN control unit 405 can control the operations of the feature extraction layer block 425 and the classification layer block 427 and perform the operation of reconstructing the input data when executing the corresponding layer. The CNN control unit 405 controls the operation of the feature extraction layer block 425 and controls the operation of the classification layer block 427 when the operation of the feature extraction layer block 425 is completed. The CNN control unit 405 may include an arbiter and an arbiter / MMU (MMU).

The information processing unit 410 may include at least one register and at least one counter. For example, the information processing unit 410 may include a weight register that stores weight data that can be used in the convolution operation of the feature extraction layer block 425 and the classification block 427. For example, the information processing unit 410 may include counters for updating the number of operations calculated according to the operation of the CNN apparatus. The information processing unit 410 may store the information transmitted from the host in the corresponding registers when the system is initialized, and may update the count value according to the operation of the CNN apparatus.

The feature extraction layer block 425 may be composed of a plurality of features. Each of the feature extraction layer blocks 425 can perform feature extraction layer operations of the ambiguous channel data. Each feature extraction layer block 425 may include an input 435, a convolution block 445, a pulling block 455 and an output 465.

The classification layer block 427 may have a plurality of blocks in one classification layer. The classification layer block 427 can perform classification layer operation of the data on which the feature extraction layer operation is completed. The extraction layer block 427 may include an input section 437, a convolution block 447, and an output section 467.

In the CNN apparatus having the above configuration, the CNN control unit 405 may store the data received from the external system in the input unit 435 of the feature extraction layer block 425 when the feature extraction layer is first executed. The CNN control unit 405 reconstructs the output data of the pulling block 455 stored in the output unit 465 and stores the reconstructed output data in the input unit 435 as the convolution input data when the second and subsequent feature extraction layers are executed. The CNN control unit 400 can execute the classification layer block 427 when the feature extraction operation is completed. The classification layer block 427 reconstructs the output data calculated by the convolution block 447 and stores it as input data of the connection layer in the input unit 437. The CNN control unit 405 can output the result data at the completion of the execution of the recovery layer to the external system.

5 is a diagram showing a configuration of a CNN system according to various embodiments of the present invention.

5, the CNN system includes a CNN control unit 500, an information processing unit 510, one or at least two channel sets 520, a memory 590, a memory control unit 580, a wrapper 570, . ≪ / RTI > Each channel set 520 may include an input 530, a convolution block 540, a pulling block 550, and an output 560. In the following description, the CNN apparatus may be a device including a CNN control unit 500, an information processing unit 510, an input unit 530, a convolution block 540, a pulling block 550 and an output unit 560, The system may be meant to include a CNN device and an external device (e.g., memory and memory controller). 5, the CNN control unit 500, the information processing unit 510 and one or at least two channel sets 520 are the same as the CNN control unit 400, the information processing unit 410, and the channel set 420 of FIG. And the input 530, convolution block 540, pulling block 550 and output 560 of the channel set 520 of FIG. 5 may correspond to the input 430 of the channel set 420 of FIG. 4A, The block 440, the pulling block 450, and the output 460.

The memory 590 may store data to perform the deep-running algorithm. The memory control unit 580 can control the read and write operations of the memory 590 and can provide input data to the CNN apparatus. The memory control unit 580 can perform data communication with an external system (e.g., a host). The wrapper 570 can perform a data interface operation between the CNN apparatus and the memory control unit 580. For example, when the system is initialized, the memory control unit 580 stores information (e.g., weight data, address map, layer configuration information, lookup table, and / or dropout information) for executing the deep learning algorithm And stored in a corresponding register of the information processing unit 510 through the wrapper 570. [ In addition, the CNN control unit 500 can receive information (for example, an address map) stored in the information processing unit 510 through the wrapper 570.

The CNN control unit 500 may include an arbiter and an MMU. The information processing unit 510 may include a plurality of registers and counters. The information processing unit 510 includes an address map register (CI re-map address map) 511, a weight register 512, a layer configuration register (L_cfgreg) A lookup table 513 and an activation function look-up table 514 and a drop out generator 515. The address map register 511 of the information processing unit, The layer configuration register 513, the activation lookup table 514, and the drop-out generator 515 may receive and store corresponding data through an external system (e.g., a host) at system initialization.

The input unit 530 may be an input register for storing input data reconfigured in the CNN control unit 400. [ The input unit 430 may include a CI register (CI-reg) for storing the convolution input data of the feature extraction layer and / or an FCI register (FCI-reg) for storing connection layer input data. The convolution block 540 may include a PE controller 543, a plurality of process elements 545, a plurality of adders 547 and an accumulator 549 have. The pulling block 550 may include a pulling operation unit 553 and a pulling input register 555. Output portion 560 may include a layer output register (LO-reg) and an activation block 560. The CNN device may have a plurality of channel hardware sets 520 and each channel set 520 may include an input 530, a convolution block 540, a pulling block 550, and an output 560. [ Lt; / RTI >

Abbreviations used in various embodiments of the present invention are shown in Table 1 below.

abbreviation stand for CNN accel convolutional neural network accelerator CH set channel hardware set reg register ID identification CI 전형 L cfg 층 구성 CI id convolution input ID CO id convolution output ID PI id pooling input ID PO id pooling output ID L id layer ID PE 공정 요소 PE ctrl 공정 요소 제어기 K id kernel DI PI reg pooling 입력 레지스터

The information processing unit 510 may include a register and a counter for storing information used in each configuration of the CH set. The information processing unit 510 includes an address map register constituting an input register, a register for trained weight, registers for storing configuration information of each layer, and counters to be used in each operation can do. First, the registers of the information processing unit 510 according to various embodiments of the present invention will be described, and then the counters will be described.

First, we explain the registers.

The weight register 512 may be a register for storing trained weight data sets. Learning can be a series of processes for learning features in an input data set. In learning, input data and weight data can be convoluted to learn features. The convolution block 540 may perform a convolution operation on the weight set and the input data stored in the weight register 512 and may calculate the input data and the weight data having the same depth and weight depth have.

The CI re-map address map block 511 may receive and store an address map for the convolution input transmitted from the host in an initialization state. The address map may exist for each layer. The address map may be a map that maps addresses of the LO reg and the CI reg. The arbiter of the CNN control unit 500 outputs the input addr corresponding to the CI reg addr of the input unit 530 through the interface wrapper 57 and refers to the address map to obtain the data of the LO reg corresponding to the input addr It can be stored in the CI reg pointed to by CI reg addr. If there is no input addr for the CI reg addr, the CNN control unit 500 can set a zero flag (ZF) to ZF = 1, for example. If the zero flag is the set address, the CNN control unit 500 can store the zero padded input data in the CI reg.

The host can calculate the address map of the address map register 511 by using the following equations (1) and (2). Equation (1) may be a mathematical expression for calculating an input address to a zero pad input address map. The input address can be from 0 to W * H * D-1. Equation (2) may be a formula for calculating a zero padded input address and a zero padded input address to convolutional input data address map. From the CI reg address, a zero padding register address (ZP reg addresses) can be calculated.

The L_cfg reg 513 may be a register for storing configuration information of a corresponding layer. The CNN control unit 500 can confirm the information contained in the L_cfg reg of the corresponding layer and configure the CI reg. The Arbiter can store the CO id, PO id, and LO id of the input data in the CI reg from L cfg reg. The information of the L-cfg reg can be the same information as the layer configuration table of Table 2 below.

name description Input data width / height / depth / channel W / H / D / CH Input data zero pad size Zx, Zy Zero padded width / height / size W + 2 * Zx / H + 2 * Zy / (W + 2 * Zx) * (H + 2 * Zy) Conv layer width / height

/

/

Conv layer zero pad size

/

Conv window size / stride size Cx / Cy /

/

Conv layer kernel K Conv output matrix size ((W-Cx + 2Zx) / Sx) +1 / ((W-Cy + 2Zy) / Sy) +1 Pool layer width / height

/

Pool layer zero pad size

/

Pool window size / stride size Px / Py /

/

Input dataset layer number L num Number of input data per channel W * H * D CI data count max Cx * Cy * D PI data count max Px * Py LO data count max

The output value of the activation lookup table 514 may be a register that stores data for performing an activation function. The data of the LO reg can be stored by performing an activation function. The activation function can be calculated by referring to the lookup table or performing a direct function.

The dropout generator 515 may generate a dropout signal. The process elements 545 of the convolution block 540 may be determined through a parameter value of the dropout generator 515. [

The information processing unit 510 may include the following counters in addition to the above registers.

The max of the channel count 501 may be the number of channels of the input data set. The channel count 501 may be a counter that is incremented by one each time one channel of input data is input as one CH set. If the number of input data channels is greater than CH set, the classification layer may not be started until the channel count value reaches max. For example, if the number of input data channels is 5 and the CH set is 3, the max of the channel count can be 5. In addition, if three CH sets can extract the characteristics of the input data of three channels in parallel, and if the characteristics are extracted from the input data of three channels, the channel count can be 3, and the three CH sets Two CH sets can be extracted from the characteristics of the input data of the two channels. In this case, the CNN device may not perform the classification layer until the count value of the channel count 501 becomes 5.

The start layer count / end layer count 502 may be a counter for checking the start and end states of the layer. The max of the start layer count can be (number of layers of the input data set - 1). The start layer count can be incremented by 1 each time the operation of the current layer is completed, and the CNN device can know the state of the current layer by checking the value of the start layer count. The max of the end layer count can be the number of layers in the input data set. The end layer count can be incremented by 1 each time a new layer starts, and can indicate the current state of the layer.

The max of CI data count / LI data count 503 may be Cx * Cy * D. The CI data count value may be incremented by one each time data corresponding to the same CO id is added in the ADD block 547 of the convolution block 540. [ This count value can be a value that can be checked to see if all the convolution input elements have been added. The ACC block 549 can input the data set to the last ADD block layer of the adder 547 if the CI data count value corresponding to the CO id of the current data is not max. In the connection layer (FC layer), the role of CO id can be performed by K_id.

The max of the PI data count 506 may be Px * Py. Pooling block 550 may check the PO id value of the data set and increment the corresponding PI data count value by one. When the PI data count value reaches the max value, the pooling block 550 may output the data set to the LO reg.

The max of the LO data count 504 may be determined by Equation (3) below. The LO data count 504 can increase the corresponding LO data count by 1 if the L_id of the data set coming in the LO reg is confirmed. The LO data count 504 may indicate whether all outputs of the layer have been calculated.

The max of the kernel count 507 can be K. The Kernel count 507 can increase the Kernel count value by 1 when the K_id of the data set input to the LO reg is confirmed. The kernel count 507 may indicate whether all outputs of the layer are calculated.

The information processing unit 510 may store information used in the channel sets. Table 3 below shows the configurations using the information of the information processing unit 510.

CNN control section Convolution block
polling
LO reg arbitor MMU PE controller PE ADD, ACC CI re-map
Address map ○ L cfg reg ○ ○ ○ ○ ○ ○ ○ Weight reg ○ ○ ○ Channel cnt ○ ○ Start layer cnt
End layer cnt ○ CI data cnt ○ ○ LI data cnt ○ LO data cnt ○ Drop out gene ○ K id cnt ○ ○ ○ PI data cnt ○ ○ ○ Kenter cnt ○

The CNN apparatus having the configuration as shown in FIG. 5 may be a CNN accelerator, and may perform both a feature learning part (feature extraction part) and a classification part. The channel sets (CH sets) 521-52N can be appropriately designed according to the number of input data channels to be used. For example, if the input data is RGB image data, the number of channels can be three, and the number of channel sets can be three.

Each of the channel sets 521 to 52N receives the input data of the corresponding channel, and performs the feature extraction operation and the classification operation. The channel sets 521-52N may include an input 530, a convolution block 540, a pulling block 550 and an output 560. The channel sets 521-52N may first perform a feature extraction operation based on the operation of the input unit 530, the convolution block 540, and the output unit 560. [ Second, after completing the feature extraction operation, the set of channels selected from the channel sets 521-52N may perform a classification operation. The selected channel cent may perform a classification operation based on operations of the input unit 530, the convolution block 540, and the output unit 560.

The channel sets 521-52N may be responsible for the computation of the CNN algorithm. The operations of the CNN algorithm may be convolution, pooling, addition (pooling element wise multiply, addition). The deep learning accelerator controller (Top Ctrl) 510 may store information used in the operation blocks of the channel sets 521-52N. The information processing unit 510 includes an address map register 511 for constructing input data in the input unit 530 and a weight register 512 for storing the learned weight for trained weight, An L-cfg register 513 for storing configuration information of layers, and the like, and may include counters that can be used in each operation.

The input data of channel sets 521-52N may be 16 or 36 point data and may also have a variable or fixed data format (16 or 32 floating or fixed point format). The configuration of the convolution block 540 of the channel sets 521-52N may be changed according to the format of the input data. The number of processor elements 545 of the convolution block 540 and the water resource of the adder 547 and the input data. If the input data is stored in the input unit 530, the convolution block 540 can immediately start the operation.

The CNN device can perform the operation of the feature extraction layer and can perform the operation of the classification layer after completing the operation of the feature extraction layer.

First, the operation of the feature extraction layer will be described.

The CNN control unit 500 can perform the function of constructing the input data to efficiently perform calculation of each layer using the limited processor elements of the convolution block 540. [ That is, the CNN control unit 500 may function as a CI reg (convolution input data register) of the input unit 530. The CNN control unit 500 can confirm the information stored in the Lcfg reg 513 of the corresponding layer and configure the CI reg of the input unit 530. In the feature extraction layer, the CNN control unit 500 may allocate one channel set 520 per channel of input data and distribute the input data to the assigned channel set 520. If the number of channels of the input data is greater than the number of the channel sets 520, the channel set 520 for the channel data of the input data may be allocated and calculated after the calculation of the channel set 520 is completed. For example, if the number of channel sets 520 is 3 (N = 3) and the number of channels of input data is 5, the CNN control unit 500 assigns input data of three channels to three channel sets first When the calculation of the input data of the three channels is completed, the input data can be calculated by assigning the input data to the two channel sets out of the three channel sets.

The CI reg can store new input data input through an external interface, and can also store output data of a previous layer as input data. The CNN control unit 500 may store data input from the external system through the wrapper 570 in the first feature extraction layer as input data in the CI reg. And, in the last feature extraction layer in the second feature extraction layer, the CNN control unit 500 can store the data that is calculated and outputted in the previous feature extraction layer as input data. 6 is a diagram showing the structure of an input data set (CI reg input data set) stored in the CI reg of the input unit 530. As shown in FIG. The input data set may include input data (i.data), zero flag (ZF), CO id, PO id, L id, and the like.

The arbiter of the CNN control unit 500 may set a zero flag ZF when the input data value is 0 when the input data is stored in the CI reg, for example, by zeroing the zero flag bit to 1 . When the zero flag bit of the input data is set, the convolution block 540 and the pulling block 550 can quickly perform the arithmetic operation based on the zero flag bit. For example, in the case of a multiplication operation, 0 is output without a calculation operation, and in the case of addition, another value (operand value) is directly output to reduce an operation latency.

When the calculation of the feature extraction layer is completed, the CNN control unit 500 can select one channel set among the channel sets 520 and input the entire input data to the selected channel set, thereby performing a calculation operation. The operation of the classification layer may be a convolution operation, and the pulling operation may be omitted.

The CNN control unit 500 may consider the current layer state when storing the input data in the dml CI reg of the input unit 530. [ The CNN control unit 500 reconfigures the input data and stores it in the CI reg. After confirming the layer configuration of the corresponding layer in the L cfg reg 513, the CNN control unit 500 generates an address map register (convolutional input re- ) 511 to determine the LO reg address of the output unit 560 and store the LO data at the corresponding address position in the CI reg of the input unit 530 as input data. The address map register 511 and the L cfg reg 513 store the CI re-map address map information and the layer configuration information transmitted from the host in an initialization state into a wrapper 570 The address map register 511 may store address maps (CI re-map addr maps) existing for each layer. The layer configuration information of the L cfg reg 513 is stored in the < Table 3> can be a layer configuration table.

The arbiter of the CNN control unit 500 can store the input data having the structure as shown in FIG. 6 up to the last address of the CI reg, and then continue to store the CI reg area already processed from the beginning. The size of the CI reg can be determined considering the external interface bandwidth of the external interface and the bandwidth of the CNN device (bandwidth of the deep learning accelerator hardware). The arbiter reads the CO id, PO id, and LO id from the L-cfg reg to construct the input data of the structure shown in FIG. Also, the arbiter may reduce the latency by providing the zero flag ZF of the corresponding input data when the process element PEs of the convolution block 540 calculate the input data.

The CNN apparatus can perform the pulling operation using the convolution computed data in the feature extraction layer. When performing the pulling operation using the convolution output, a method of waiting for the entire pooling input (convolution output) and performing the pulling operation may take a lot of time. If a CNN device (channel set) has two or more pooling inputs (a convolution output), performing a pooling operation in advance with two values can complete the entire feature layer calculation more quickly. A CNN device (channel set) according to various embodiments of the present invention may perform a pulling operation regardless of whether or not the convolution operation is completed, if at least two pulling inputs are present. To this end, a CNN device (channel set) according to various embodiments of the present invention may first calculate input data corresponding to an onvolution input operand involved in one pulling input.

The arbiter of the CNN control unit 500 refers to the CI re-map addr map of the address map register 511 of the information processing unit 510 and outputs necessary input data to the LO unit 560 of the output unit 560 You can read from reg and save it as input data in CI_reg. The CI re-map addr map of the address map register 511 can receive and store the values calculated from the host in the initial state as described above, and the arbitration map can exist for each layer. The address map may be constructed by referring to Equation (1) and Equation (2).

The arbiter of the CNN control unit 500 outputs the input address of the CI reg to the address map register through the wrapper 570 and stores the value read from the address map register 511 in the CI reg indicated by the address of the corresponding CI reg have. The CNN control unit 500 can set the zero flag ZF to 1 if there is no input address corresponding to the address of CIreg. An address where the zero flag ZF is set to one may correspond to zero padding (zero padding of CNN's data pre-processing). For example,

For example, if the input data, the convolution parameter, and the full parameter have the following conditions, they can be represented by a map as shown in Table 4 below.

input data; W = 3, H = 3, D = 3, CH = 3.

First layer parameter convolution: Cx = Cy = D = 3, Zx = Zy = Sx = Sy = 1, K = 2

first layer parameter pool; Px = Py = 2, Sx = Sy = 1

CI reg addr ZP reg addr input addr 0 0 One One 2 2 3 5 4 6 0 5 7 6 10 7 11 3 8 12 4 9 25 10 26 11 27 12 30 13 31 9 14 32 10 15 35 16 36 12 17 37 13 18 50 -
-
- -
-
- -
-
-

When the input data is stored in the CI reg of the input unit 530, the convolution block 540 may perform a convolution operation. The convolution block 540 includes a PE control unit 543, a plurality of process element blocks (PE units) 545, a plurality of adders (ADD blocks) 547, and an accumulator (ACC) (547). The convolution block 540 may receive the input data stored in the CI reg and the weight data stored in the weight reg 512 and may output the convolution computed output data to the pulling block 550.

7 is a diagram showing a configuration of a convolution block according to various embodiments of the present invention. 8A and 8B are diagrams showing a configuration of a process element of a convolution block according to various embodiments of the present invention. 9a? 9C is a diagram showing a structure of an input data set, a weight data set, and an output data set of a convolution block.

7, a convolution block 540 includes a PE control unit 543, a PE unit 545 constituting a plurality of process elements, an addition unit 547 including a plurality of adders, 549 &lt; / RTI &gt;

Each of the PEs of the PE unit 545 may perform an operation of multiplying input data and weight data, which are input, respectively. The PE control unit 543 can input the input data and the weight data respectively to the PE 545 from the CI reg and the weight reg 512 so that the PE unit 545 can perform the multiply operation of the convolution. May save time to read weight data by pre-fetching weight data in a weight reg 512. The PE control unit 543 of the CNN apparatus according to various embodiments of the present invention may start For example, if the start layer counter is 1 and the end layer counter is 3, the PE control unit 543 may prefetch the weight data in the weight reg 512 by referring to the layer cnt / end layer cnt 502. For example, May prefetch the weight data set of layer 1 - layer 3 in the weight reg 512. The PE control unit 543 transmits weight data corresponding to the L_id of the input data among the prefetched weight data sets to the PE unit 545 .

The PE control unit 543 can input the input data of the CI reg and the weight data of the weight reg 512 of the corresponding layer to the PE unit 545 which is ready for calculation. The multiply operations of each PE unit 545 that inputs input data and weight data may have different operation times. If there is more than one input data in the CI reg and there is a PE that can be calculated by the PE unit 545, the PE control unit 543 can input the corresponding input data and weight data to the prepared PE in the PE unit 545 have. The PE control unit 543 inputs the next input data and weight data to the corresponding PE and outputs the CI reg and the weight reg 512 Can be increased by one. If the weight data is 0, the PE control unit 543 can set the zero flag ZF to 1.

The PE control unit 543 can input the input data having the structure of the CI reg of FIG. 6 to the PE for which the multiplication operation has been completed, and the convolution block 543 processes the operation operation of the input data having the CO id . That is, the output of the convolution block 540 may be the sum of the multiplication result of the input data having the same CO id. The PE control unit 543 can check the layer configuration information of the L cfg reg 513 and input the generated weight data to the PE unit 545. [ The PE control unit 543 confirms the value of the start to end layer count 502 and the layer configuration information of the corresponding L cfg reg 513 and then prefetches the corresponding weight data set in the weight reg 512 The lead time of the weight data can be reduced. K_id represents a weight set number, and each channel set 520 can be used to determine whether to perform a layer operation of the next layer.

The PE unit 545 can multiply input data and weight data input by the PE control unit 543. The addition unit 547 may add the result of the convolution operation output from the PE unit 545. [ The accumulator 549 may accumulate the output of the adder 547 added so that the results of all convolutional multiplications are added. The accumulator 549 may output the accumulation result to the pooling block 550 when all the convolution multiplication results are accumulated.

Referring to FIG. 8A, convolution block 540 may perform convolution layer operations by inputting a set of input data such as 911 of FIG. 9A and a weight data set such as 913. The input data set may be a data set comprising input data, a zero flag ZF, a COID (convolution output ID), a PI id (pooling input ID), and a PO id (pooling output id) L id (layer). The weight data set may be a data set comprising weight data, zero flags ZF and Kid (kernel id). The multiplier 810 may multiply the input data of 911 of FIG. 9A and the weight data of 913 of FIG. 9A. The OR gate 820 can perform the logical sum operation on the zero flag included in the input data set having the structure like 911 of FIG. 9A and the zero flag included in the weight data set having the structure like 913 of FIG. 9A. The selector 830 inputs a first input of an output of the multiplier 810 and a second input of a ground (zero) signal. The selector 830 selects and outputs the first input or the second input by the output of the gate 820. Each PE of the PE portion 545 may include a bead generator 810 corresponding to the number of points of the input data. For example, a PE can be a 16bit or 32bit / floating or fixed point multiplier. Each PE can input the input data and the weight data as shown in FIG. 9A under the control of the PE control unit 543, and the multiplier 810 can perform convolutional multiplication of the two input data. The output of the multiplier 810 may be the first input of the selector 830. The gate 820 can perform the logical sum operation on the input data such as 911 in Fig. 9A and the zero flag ZF included in the weight data as 913 in Fig. 9A. Therefore, the selector 830 can selectively output the second input if any one of the zero flag signals included in the input data or the weight data is set to the zero flag ZF. That is, the selector 830 can select the output of the multiplier 810 if the zero flag ZF of the input data and weight data is not all set. The output data set structure of the PE outputting the selector 830 may have the structure as shown in 915 of FIG. 9A. The PE's result data set may be a data set consisting of the result data, the zero flag ZF, CO id, PO id, L id, K id.

Referring to FIG. 8B, each PE of the PE unit 545 can output zero data by a dropout signal in a fully connected layer operation of a classification layer. The multiplier 810 may multiply the input data (input data and weight data) by convolution. The gate 840 may perform an OR operation on the input data set such as 911 of FIG. 9A and the zero flag ZF included in the weight data set as 913 of FIG. 9A and the dropout signal. The dropout signal may be a signal for dropping the operation of the PEs set in the classification operation, and may refer to the output of the dropout generator 515 of the information processing unit 510. Therefore, the selector 830 can selectively output the second input if either the zero flag ZF is set in the input data or the zero flag signal included in the weight data, or the dropout signal is set. That is, the selector 830 can select the output of the multiplier 810 when neither the zero flag ZF of the input data and the weight data is set, and the dropout signal is not set.

The CNN apparatus according to various embodiments of the present invention may set the ZF of the result data packet to 0 and pass it to the next step if the ZF flag ZF of the input data or weight data is 1. Since the CNN neuron's operand data is often zero, the convolution block 540 may only check ZF and not perform calculations, thereby improving the speed of convolution calculation. Also, the convolution block 540 may not perform the convolution operation if the dropout signal is set.

Each PE of the PE unit 545 may insert the K id of the weight data as 913 of FIG. 9A into the K id of the result data packet having the structure of 915 of FIG. 9A. The output data calculated at each of the PEs of the PE unit 545 may be transmitted to the adding unit 547. [ At least two PE result values may be sent to one adder of the adder 547. [ The necessary values among the feature values of the operand data sets may be stored as result data sets and transmitted to the adder 547. [

The adder 547 may be composed of a plurality of adders (for example, 16 bits or 32 bits / floating or fixed point adders). The addition unit 547 can perform an operation of inputting and adding two or more multiplication result values calculated by the PEs of the PE unit 545. For example, each adder of the adder 547 adds the first result data set, such as 931 of FIG. 9B and the second result data sets, such as 933, output from the PE unit 545, Can be output. The structure of the first and second result data sets, such as 931 and 933 of FIG. 9B, may be the same as the data set structure 915 of FIG. 9A. At this time, if the zero flag set (ZF = 1) state of the adder input data (PE result data, for example, PE result data A 931) The same PE result value B) can be directly output as an addition result value. 9B and 933, the adder stores feature information (feature bit values) in a result data set such as 935, and the next block (ADD or ACC ).

Each adder of the adder 547 can add data corresponding to the same CO id. That is, the data (PE output data) input to the adders of the adder 547 may have the same structure as 931 and 933, and the adders can input and add two data having the same CO id among the PE output data have. At this time, if the CO id of the two data operands is not the same, the adders can pass the two data as they are without performing the add operation.

The convolution block 540 increments the CI data count 503 corresponding to the CO id of the information processing unit 510 by one every time the same CO id operation is performed. CI data count can be increased to Cx * Cy * D. (The information on each count of the information processing unit is shown in Table 2, and the maximum value of each counter can be stored in L-cfg reg. Each adder of the adder 547 adds two inputs ADD block layers. The adders of the first stage receive and add the two data output from the PE 545, and the adders after the second stage are shifted Therefore, the maximum number of ADD operands that can be received increases with the passage of the ADD block layer, and the adders have the same CO id You can add them to each other and pass all the others to the next ADD block layer.

The accumulator 549 may accumulate the output of the last adder layer of the adder 547, such as 951-955 in Figure 9c. The accumulator 549 checks the CO id of the data output from the adder 547. When the value of the CI data count 503 corresponding to CO id reaches max (Cx * Cy * D of the corresponding layer) , The corresponding data such as 961 in FIG. 9C may be stored in the PI reg 555 of the pulling block 550. If the value of the corresponding CI data count 503 is not the max value, the accumulator 549 can input the corresponding data as 963 in FIG. 9C to the last Add layer of the adder 547 again. The last Add layer can confirm that the data has been input from the accumulator 549 and perform the add operation. The fact that the CI data count has not reached max may mean that the input data with the corresponding CO id is still being calculated or not transmitted in the CI_reg of the input unit 530. When the result data is passed from the accumulator 549 to the pooling block 550, the CO id is no longer used, so that CO id can be removed as shown at 961 in FIG. 9C.

10 is a diagram showing an input data structure of a pulling operation in a CNN apparatus according to various embodiments of the present invention. The input data of the pulling operation may be the output data of the convolution block such as 961 of FIG. 9C. 11 is a diagram showing the structure of output data and activated data of a pulling block in a CNN apparatus according to various embodiments of the present invention.

The convolution result data as shown in FIG. 10 may be sequentially stored in a pooling input register block (hereinafter referred to as PI reg) of the pulling block 550. When two or more pieces of data (convoluted data) are stored in the PI reg 555, the pulling execution unit 553 can perform a pooling operation of data having the same PO id by checking the PO id have. If there are two or more result values having the same PO id (that is, if there are two or more values having the same PO id in the PI reg), the pooling block 550 may perform a pooling operation , Which can reduce the latency of the pooling operation.

When performing the pooling operation, the pooling block 550 may increase the value of the PI data count 506 by 1 by checking the PO id value. The max value of the PI data count may be Px * Py as shown in Table 2 above. The pulling operation of the pulling block 550 may use an input having a maximum value among the two inputs or a method of obtaining an average value of the two inputs. In the case of a max pooling method, the pooling block 550 may select a larger value and store it again in the PI reg 555 and discard the smaller value if only two data are available. In the case of average pooling, the pooling block may average the two values and store them again in the PI reg (555).

The pulling block 550 can perform the quick pulling operation by utilizing the zero flag ZF in the pooling input data having the structure as shown in FIG. After performing the pooling operation, the pooling block 550 checks the value of the PI data count 506 of the corresponding PO id, and when the value of the PI data conut 506 reaches the max value (Px * Py) The output value of the output unit 560 can be output to the LO reg of the output unit 560. The pulling block 550 can output the pooling operation result value to the LO reg of the output unit 560 in accordance with the order of the POid. Since the PO id is no longer needed in the data structure to be transmitted, Can be removed. If the value of the PI data count 506 of POid has not reached the max value, the pooling block 550 stores the result of the pooling operation again in the PI reg 555, and until the data of the same PO id is received You can wait.

The data for which the pulling operation has been completed in the pulling block 550 may be stored in a layer output register (hereinafter referred to as LO reg) of the output unit 560.

The data of the LO reg can be stored by performing an activation function. The activation function may be calculated by referring to an activation function LUT 514 of the information processing unit 514 or by performing a direct activation function. The output unit 560 may check the Lid of the same data as 1111 of FIG. 11 stored in the LO reg and increase the corresponding LO data count 514 of the information processing unit 510 by 1 every time a result value is received . When the count value of the LO data count 504 reaches the maximum value, the values can be used as input data of the next layer. The CNN control unit 500 increments the value of the end layer count 502 by 1 and sets the value of the LO reg of the output unit 560 to the input value of the input unit 530 when the count value of the LO data count 504 reaches the maximum value It can be saved as input data in CI reg. The PE control unit 543 of the convolution block 540 may check the end layer count and prefetch the input data stored in the CI reg and may also prefetch the input data stored in the weight register 512 of the information processing unit 510 The weight data set can be pre-fetched. However, since all of the kernels (the kernel of the current layer output and the depth of the next layer input) are not calculated, the PE control unit 543 of the convolution block 540 sets the prefetch value of the weight data indicated by the start layer count It can not be removed. The kernel value can be confirmed by Kid and the kernel count 507 of the information processing unit 510. The maximum value of the LO data count 504 can be determined as shown in the layer configuration table in Table 2 above. If the LO data count is not the maximum value, the CNN device can wait until the result with the same L id that has not yet been calculated is output.

The channel set 520 may check the Kid of the data input to the LO reg and increase the Kernel count by one. If the Kernel count corresponding to K id is the maximum value, the channel set 520 can release the weight data pointed to by the start layer since it no longer needs to use the information of the layer indicated by the start layer. The MMU of the CNN control unit 500 can control the operation of the channel set 520 by referring to the configuration value of the next layer already set in the L cfg reg 513. Based on the result of analyzing the configuration of the next layer, if the layer to be calculated remains, the CNN control unit 500 inputs the data of the LO reg of the output unit 560 to the address map register 511, And stores the set input data in the CI reg of the input unit 520.

The fact that the count values of the LO data count 504 and the kernel count 507 are the maximum values means that no data corresponding to the current start layer and the end layer remains in the convolution block 540 and the pulling block 550 can do. Therefore, the CNN control unit 500 and the channel set 520 can increase the start layer, cancel the weight data, increase the end layer, and store the input data in the CI reg in each case.

However, if the count values of the kernel count 507 and the channel count 501 are the maximum values, the CNN control unit 500 can terminate the operation of the feature extraction layer and perform the operation of the classification . That is, the classification layer (FC layer) can be started when the calculation of all the input channels is completed by checking the values of the kernel count and the channel count. If the channel count value has not reached max, it can wait until all the input data of the other channel is learned. For example, if the CNN device has three channel sets and learns five channel data, the channel count may be five. In this case, the channel sets can simultaneously learn three channel data. At this time, when the learning of the three channel data ends, the channel count value may be 3 and the operation of the feature extraction layer may be performed until the maximum value is 5. That is, if the feature extraction operation of the channel set is terminated but the feature extraction operation of the next channel data is not terminated, until the feature extraction operation of the other channel data is terminated (for example, Until the next two channel data is learned, until the channel count becomes 5).

12 is a diagram illustrating a structure of input and output data of a convolution block in a classification layer in a CNN apparatus according to various embodiments of the present invention. 13 is a diagram showing a structure of layer output data in a CNN apparatus according to various embodiments of the present invention. The classification layer may include at least one fully connected layer (hereinafter referred to as FC layer) as shown in FIG.

In the classification layer operation, the arbiter of the CNN control unit 500 refers to the L-cfg 513 for the FC layer and inputs the input data having the structure shown in FIG. 12 to the FCI reg (Fully connected input register) of the input unit 530 Can be stored. For example, if Cx = Cy = 1 and Sx = Sy = 1, the same window as the FC can be obtained. The input data of the FC layer may be the output of all channel sets of the previous feature extraction layer. The FC layer can be input to a set of channels to calculate the output of all feature extraction layers. For example, when characteristics of three channel data are extracted from the feature extraction layer, the output values of the feature extraction layer can be input to three channel data in one channel set in the FC layer. The input data of the FC layer can be input to the input unit 530 sequentially. That is, the input data of the feature extraction layer is remapped by referring to the address map register 511, and then inputted to the input unit 530, while the input data of the FC layer can be sequentially input to the input unit 530 . The number of weight data can be input equally by the number of channel sets 520. The total number of weight elements may be (input data x output data). The K id of the FC layer can exist as many as the number of output data. The maximum value of the kernel count in each channel set 520 may be (number of output data / CH set).

In the FC layer, the convolution block 540 can perform the function of CO id using K id, unlike the feature extraction layer. The PE unit 545 multiplies the input data by the weight data, and the adding unit 547 adds the data having the same K id among the multiplied data. The adding unit 547 can identify the K id and increase the count value of the LI data count 503. [ The accumulator 549 may store the data in the LO reg of the output unit 550 when the value of the LI data count 503 corresponding to the K id of the input data is equal to the number of input data of the layer. In the FC layer, the pooling block 550 may not operate as described above. The output unit 500 can store the data in the LO reg in the order of K id, and the K id of the stored data can be removed.

Each PE of the PE unit 545 in the convolution block 540 may have a configuration as shown in FIG. 8B. The multiplier 810 may multiply the input data such as 1211 in FIG. 12 and the weight data such as 1213 in FIG. The OR gate 840 can perform a logical sum operation on the zero flag included in the input data having the structure as in 1211 of FIG. 12 and the zero flag included in the weight data having the structure as 1213 of FIG. The selector 830 may input a first input of the multiplier 810 and a second input of the ground (zero) signal, and may select and output the first input or the second input by the output of the gate 840. The selector 830 can selectively output the second input if any one of the zero flag signals included in the input data or the weight data is set to the zero flag ZF. That is, the selector 830 can select the output of the multiplier 810 if the zero flag ZF of the input data and weight data is not all set. The output data structure of the PE outputting the selector 830 may have a structure as shown in 1215 of FIG.

The CNN device can perform a drop out operation, which is one of the regularization methods for improving the performance of the deep learning algorithm in the classification layer. The drop-out function may mean dropout of the operation of a set number of PEs among a plurality of PEs. In various embodiments of the present invention, the convolution block 540 may apply a dropout signal to the PE performing the convolutional multiply operation, and the PE inputting the dropout signal may drop out the operation of the two input data have. That is, the PE receiving the dropout signal can calculate the input data (operand) as if it were zero (for example, ZF = 1) and transmit the result value. The dropout signal may be generated in the dropout generator 515 of the information processing unit 510. The probability that each PE of the PE unit 545 is selected may be determined through a parameter value of the drop-out generator 515. [ When the dropout signal is inputted, the gate 840 of the PE can apply the dropout signal to the selector 830 as a selection signal. Then, the selector 830 can select and output the ground data (zero data). Each PE of the PE unit 545 selects and outputs zero data when the ZF of the input data is set, the ZF of the weight data is set or the dropout signal is input, the ZF of the input data and the weight data is not set, When the dropout signal is not inputted, the output of the multiplier 810 can be selected.

When the data as shown in FIG. 12 outputted from the convolution block 540 is outputted, the activation function operation of the output unit 560 can be performed. The output unit 560 may increase the value of the LO data count 504 corresponding to L id by 1 each time the output data of the convolution block 540 is input. And the arbiter of the CNN control unit 500 can read the LO data count 504 when the value of the LO data count 504 reaches the maximum value layer output number). If the value of the LO data count 504 is not the maximum value, the output unit 560 can wait until the data having the L id is input. When output to the arbiter, L id is no longer needed and can be removed. In the case of the last FC layer, Z_F can also be removed. The output data structure of the LO reg of the output unit 560 may be data as shown in FIG. The arbiter of the CNN control unit 500 may collect the result values of the classification layer shown in FIG. 13 performed in each channel set in order and output it to the host through the interface wrapper 570.

14 is a diagram showing another configuration of a CNN system according to various embodiments of the present invention.

14, a CNN system includes a CNN control unit 500, an information processing unit 510, an input unit 530, a convolution block 540, a pulling block 550, an output unit 560, a memory 1400, A PCIe root point 1420, and an interface (PCIe) end poing 1430, as shown in FIG.

Memory 1400 may store data for performing a deep-running algorithm. The memory control unit 580 can control the read and write operations of the memory 590 and can provide input data to the CNN apparatus. The memory control unit 1410 can control the operation of the memory 1400. [ The memory control unit 1410 may be a control unit of the host device. The host device may be an electronic device. The PCIe root point 1420 and the interface 1430 may perform an interfacing operation satisfying the specification of peripheral component interconnect express (PCIe). PCIe is a serial interface for input and output and can be an interface with high system bus bandwidth, fewer I / O pins, less physical area, performance scalability of bus devices, detailed error detection and reporting structure. The PCIe root point 1420 and the interface 1430 may provide an interface function between the memory controller 1610 and the CNN device. For example, the data interface operation of the CNN device and the memory control unit 580 can be performed. For example, when the system is initialized, the memory controller 580 stores information (e.g., weight data, address map, layer configuration information, lookup table and / or dropout information May be provided to the CNN device via the PCIe root point 1420 and interface 1430. The CNN device may also provide the learning results of the channel data to the memory controller 1610 via the PCIe root point 1420 and interface 1430.

The CNN apparatus may include a CNN control unit 500, an information processing unit 510, an input unit 530, a convolution block 540, a pooling block 550, and an output unit 560. The configuration and operation of the CNN apparatus may be the same as the configuration and operation of FIG.

The CNN apparatus according to various embodiments of the present invention performs an initialization operation as shown in FIG. 3, performs a feature extraction layer operation on input data of all channels, When the layer operation is completed, the operation of the classification layer is performed. When the operation of the classification layer is completed, the learned result value can be transmitted to the host.

15A-15C are flow charts illustrating operation of a CNN device in accordance with various embodiments of the present invention. 15A to 15C can be operations of the CNN control unit 500 of the CNN apparatus.

Referring to FIGS. 15A to 15C, operation of the CNN apparatus can be started in step 1511. FIG. When the operation of the CNN apparatus is started, the CNN apparatus can perform the host initialization operation in step 1513. [ In the host initialization operation, the host may transmit information (initialization information) to be used in the feature extraction layer and configuration layer operation to the CNN device, and the CNN device may transmit the initialization information to the respective registers Lt; / RTI &gt; In the initialization operation, the information processing unit 510 stores the weight data sets in the weight register 512, stores the CI re-map address map information in the address map register 511, stores the layer configuration information in the L cfg And store the activation function look-up table in the activation function register 514.

After performing the initialization operation, the CNN apparatus starts the operation of the feature extraction layer in step 1515, and may start the operation of the arbiter of the CNN control unit 500 in step 1517. [ The CNN device may map the data channel to the available channel set 520 in step 1519. [ The CNN apparatus can set the channel count (501) in step 1521. The channel count can be set to (CH count + number of channel sets). After setting the channel count, the CNN apparatus can start the MMU operation of the CNN control unit 500 in step 1523 and start the feature extraction layer operation.

When the feature extraction layer is started, the CNN apparatus confirms the layer configuration by referring to the L cfg reg 513 in step 1531 and sets the count value of the start layer count / end layer count 502 in step 16533 . the count value of the start layer count / end layer count 502 is passed to the convolution block 540 to cause the convolution block to prefetch a set of weight data corresponding to the start and end layer count values stored in the weight register 512 have. The CNN apparatus can map (addr, input addr) map of start and end layer counts of the addresses of the LO reg and the CI reg by referring to the information of the address map 511 in step 1535.

The CNN device can then check the input data of the feature extraction layer in step 1537. The feature extraction layer may be a pair of a convolution layer and a pooling layer as shown in FIG. 1, and the input data may be learned by a plurality of feature extraction layer operations. At this time, if the input data is the input data of the first feature extraction layer (input data of the convolution input layer), the CNN device recognizes this in step 1537, reconstructs the data input from the external system in step 1539, Can be stored. At this time, the address of the CI reg can be determined as follows. If the input data is 0, zero-rate ZF can be set.

CIreg [addr] = {LO reg [Input addr], Z_F, L-cfg reg [L-cnt *]}

L-cfg reg [L-cnt *] = {CO_id, PO_id, L_id}

if (LO reg [Input addr] == 0) Z_F = 1

The CNN device can reconstruct the result value of the previous feature extraction layer and apply it as the input data of the next feature extraction layer. Accordingly, if the input data is input data after the second feature extraction layer (input data of the convolution internal layer), the CNN device recognizes this in step 1537, reconstructs the result value of the previous feature extraction layer in step 1541, CI reg. At this time, the address of the CI reg can be determined as follows. If the input data is 0, zero-rate ZF can be set.

CIreg [addr] = {LO reg [Input addr], Z_F, L-cfg reg [L-cnt *]}

L-cfg reg [L-cnt *] = {CO_id, PO_id, L_id}

if (LO reg [Input addr] == 0) Z_F = 1

The structure of the input data stored in the input unit 530 may have a structure as shown in FIG. In step 1543, the CNN apparatus performs a convolution operation on the input data stored in the input unit 530 and the weight data prefetched in the information processing unit 510. In step 1545, the CNN apparatus may perform a pulling operation on the convolution operation result value. The convolution operation according to various embodiments of the present invention can be performed in the same manner as in FIGS. 16A and 16B, and the pooling operation according to various embodiments of the present invention is performed by the method shown in FIGS. 17A and 17B .

After performing the pulling operation, the CNN apparatus can check in step 1547 whether the currently processed feature extraction layer is the last feature extraction layer. If the CNN apparatus is not the last feature extraction layer, the CNN apparatus determines that there is a next feature extraction layer, and returns to step 1531 to perform a calculation operation of the next feature extraction layer. If it is the last feature extraction layer, the CNN device recognizes this in step 1547 and checks in step 1551 whether the feature extraction layer operation of all channel data is completed. For example, a CNN device may have three channel sets and process the input data of five channels. The CNN apparatus performs the feature extraction layer operation of the input data of three channels and performs the feature extraction layer operation of the input data of the two channels successively when the feature extraction layer operation of the input data of the three channels is completed . Accordingly, if the feature extraction and excitation of the channel set is terminated but the feature extraction layer operation is not completed in the input data of all the channels, the CNN apparatus recognizes this in step 1551, and returns to step 1517 to input data It is possible to perform a feature extraction layer operation for the feature extraction function.

However, upon completion of the feature extraction operation on the input data of all the channels, the CNN device recognizes this in step 1551 and may start the operation of the classification layer in step 1553. [ The classification layer operation can be performed through a set of channels (All LO reg data from Feature Extraction layer SHOULD input to one HW CH set for classification). When the operation of the classification layer is started, the CNN device can read the layer configuration information in the L cfg reg 513 in step 1555 and set the strat layer count / end layer count 502.

The CNN device may then check the FC layer (fully connected layer) in step 1557. The input data for the first FC layer can be the final result of the feature extraction layer operation and the FC internal layer after the second can be the result of the previous FC layer operation. have. If it is the first FC layer, the CNN device recognizes this in step 1557, and in step 1559, reconstructs the input data as follows and stores it in the input part 530. At this time, if the input data is 0 data, the zero flag ZF can be set.

CIreg [addr] = {All LO reg [Input addr], Z_F, L-cfg reg [L-cnt *]}

L-cfg reg [L-cnt *] = {L_id}

if (LO reg [Input addr] == 0) Z_F = 1

If it is the second or later FC layer, the CNN device recognizes this in step 1557, and in step 1561, reconstructs the input data as follows and stores it in the input part 530. At this time, if the input data is 0 data, the zero flag ZF can be set.

CIreg [addr] = {LO reg [Input addr], Z_F, L-cfg reg [L-cnt *]}

L-cfg reg [L-cnt *] = {L_id}

if (LO reg [Input addr] == 0) Z_F = 1

The structure of the input data stored in the input unit 530 may have a structure as shown in 1211 of FIG. After the input is reconfigured, the CNN device may perform the convolution operation operation of the input data in step 1563. The convolution operation may be performed in the same manner as in Figs. 16A and 16B. Classification layer operations may not perform pooling operations. Accordingly, when the convolution operation operation is completed, the CNN apparatus can store the result of the convolution operation in the LO reg of the output unit 560 in step 1565.

After performing the FC layer operation, the CNN apparatus can check in step 1567 whether the currently performed FC layer is an internal layer or an output layer. The CNN device can check the FC output layer by referring to the start layer count, the end layer count, and the kernel count. If the current processed FC layer is the internal layer, the CNN apparatus returns to step 1555 to perform the operation of the next FC layer. If the current layer is an output layer, the CNN apparatus terminates the operation of the CNN apparatus in step 1569, Device.

16A and 16B are flow charts illustrating the operation of the convolution layer operation of a CNN device in accordance with various embodiments of the present invention.

16A and 16B, the convolution layer operation may be performed in a different manner in the feature extraction layer and the classification layer. The CNN device may start the convolution layer operation in step 1611. [ The CNN apparatus can prefetch the weight data to be convoluted with the input data in step 1613. [ The PE control unit 543 of the convolution block 540 may prefetch the weight data corresponding to the startlayer count / end layer count 5020 to the weight register 512. [ In step 1615, the CNN apparatus may configure an input data set including ZF, CO id, PO id, and L id as shown in FIG. 6 (911 in FIG. 9A). In step 1617, RTI ID = 0.0 &gt; ZF &lt; / RTI &gt; and Kid. In the case of the classification layer, the input data set may have the same structure as 1211 in FIG. 12, and the weight data set may have the same structure as 1213 in FIG. The input data may be the first input (PE input A = CIreg [addr]) of the convolution block 540 and the weight data set may be the second input of the convolution block 540 ], Z_F, K_id [L-cnt]}. The CNN apparatus checks whether there is a prepared PE (convolutional operation PE, empty PE) prepared in step 1619. If there is no prepared PE, the CNN apparatus can wait until the prepared PE is prepared in step 1621. If there is a prepared PE, the CNN device recognizes this in step 1619, and can input the input data and the weight data to the PE prepared in step 1623.

The convolution block 540 may be operated on both the feature extraction layer and the classification layer. If the currently executing operation is a feature extraction layer operation, the CNN device recognizes this in step 1631 and analyzes the zero flag ZF of the input data or weight data in step 1633. At this time, if the zero flag ZF is set, the CNN device recognizes this in step 1633, and can output zero data without performing the convolution multiply operation in step 1637. If the zero flag ZF is in the reset state, the CNN device recognizes the zero flag ZF in step 1633 and may perform the convolution multiply operation in step 1635. [ For example, the convolution multiplication operation can be performed through a PE having a structure as shown in FIG. 8A. In operation of the PE, the multiplier 810 multiplies the input data with the weight data, and the selector 830 can input the output of the combiner 810 and the zero data. At this time, the selector 830 selects and outputs the zero data if the ZF of the input data or the weight data is set, and can select the output of the multiplier 810 only when both the ZF of the input data and the weight data are in the reset state. In step 1641, the CNN apparatus can generate a result data set of the convolution multiplication operation of the feature extraction layer (Conv: {Result, Z_F, CO_id, PO_id, L_id, K_id}). The convolution multiplication result data set of the feature extraction layer may have a structure as shown in 915 of FIG. 9A.

If the operation currently being executed is a classification layer operation, the CNN device recognizes the operation in step 1631 and may check the dropout in step 1639. The dropout may be determined by the dropout generator 515. For example, the CNN device may drop out 50% of the PEs out of the PEs in the PE portion 545. If it is a drop-out, the CNN device recognizes this in step 1639, and can output zero data without performing a convolutional multiply operation in step 1637. If not, the CNN apparatus proceeds to step 1633 to analyze the zero flag ZF of the input data or weight data. At this time, if the zero flag ZF is set, the CNN device recognizes this in step 1633, and can output zero data without performing the convolution multiply operation in step 1637. If the zero flag ZF is in the reset state, the CNN device recognizes the zero flag ZF in step 1633 and may perform the convolution multiply operation in step 1635. [ For example, the convolution multiplication operation can be performed through a PE having a structure as shown in FIG. 8B. In operation of the PE, the multiplier 810 multiplies the input data with the weight data, and the selector 830 can input the output of the combiner 810 and the zero data. At this time, the selector 830 selects and outputs the zero data when the dropout signal is inputted, or when ZF of the input data or weight data is set, and when the ZF of the input data and the weight data is both reset The output of the multiplier 810 can be selected. Then, the CNN device can generate a result data set (FC :: {Result, Z_F, L_id, K_id}) of the convolution multiply operation of the classification layer in step 1641. The convolution multiplication result data set of the classification layer may have the same structure as 1215 of FIG.

If it is a feature extraction layer operation, the CNN apparatus recognizes this in step 1643 and analyzes the PO id in the convolution multiplication operation result data set 915 of FIG. 9A outputted from the PEs (PE A PE B) . If it is determined that the CO id of PE A is equal to the PO id of PE B in step 1645, the CNN apparatus adds the outputs of PE A and PE B in step 1649 and increases the CI data count by 1 in step 1641 . However, if the CO id of PE A differs from the PO id of PE B, the CNN device may recognize this in step 1645 and bypass the addition operation in step 1647. Thereafter, the CNN apparatus checks in step 1653 whether it is the final addition, and if not, it returns to step 1645 and continues the addition operation. If it is determined in step 1653 that the last addition is performed, the CNN apparatus can check whether the accumulated value in step 1655 is the maximum value (CI data cnt [CO id] == MAX) of the CI data count 503. If the maximum value is not the maximum value (for example, the data set as 963 in FIG. 9C), the process returns to step 1645 to continue the addition operation. If the maximum value is present, the CO ID is removed in step 1657, For example, data such as 961 in FIG. 9C) in the PI reg of the pulling block 550. After that, the CNN apparatus ends the calculation of the convolution layer in step 1659, and may start the pooling layer operation in step 1661. [

If it is a classification layer operation, the CNN device recognizes this in step 1643 and analyzes the K id in the convolution multiplication operation result data set 1215 of FIG. 12 outputted from the PEs PE A PE B to perform an addition operation can do. If it is determined that the CO id of PE A is equal to the K id of PE B in step 1671, the CNN apparatus adds the outputs of PE A and PE B in step 1675 and increases the LI data count by 1 in step 1677 . However, if the K id of the PE A is different from the K id of the PE B, the CNN device recognizes this in step 1671 and bypasses the addition operation in step 1643. Thereafter, the CNN apparatus checks whether the final addition is performed in step 1679. If not, the CNN apparatus returns to step 1671 and continues the addition operation. If it is determined in step 1679 that the last addition is performed, the CNN apparatus can check whether the accumulated value in step 1655 is the maximum value (LI data cnt [K id] == MAX) of the LI data count 503. If the maximum value is not the maximum value, the process returns to step 1671 and the addition operation is continued. If the maximum value is obtained, the result is stored in the LO reg of the output unit 560 in step 1683. After that, the CNN apparatus ends the operation of the convolution layer in step 1685 and starts the operation of the output unit 560 in step 1687 (FC LO reg block start). The FC LO reg block start can be performed in the same manner as in FIG.

17A and 17B are flow charts illustrating the operation of a pulling layer operation of a CNN device in accordance with various embodiments of the present invention.

Referring to FIGS. 17A and 17B, in step 1711, the CNN apparatus can start the pooling operation of the feature extraction layer. The data set stored in the PI reg can have the structure shown in FIG. When the pooling operation is started, the CNN device can retrieve data having the same PO id in the PI reg. At this time, if the PO id of the data is not the same, the process can proceed to step 1715 and wait. If the PO id is the same (for example, PO id (A) == PO id (B)), the CNN device recognizes this in step 1713 and can perform the pooling operation in step 1717. At this time, the pulling operation may be a method of selecting data having a large value among two data or obtaining an average value of two data. After performing the pulling operation, the CNN apparatus can check in step 1721 whether the PI data count 506 is the maximum value. If the PI data count 506 is not the maximum value, the CNN apparatus stores the pooling operation value in the PI reg in step 1723 and proceeds to step 1713. [ If the value of the PI data count 506 is the maximum value, the CNN apparatus removes the PO id from the pooled operation data set in step 1725, and refers to the activation function LUT 514 in step 1727 to set the activation function of the pooled operation data And may be stored in the LO reg of the output unit 560.

The CNN apparatus increases the value of the LO data count 504 corresponding to L id by 1 in step 1731 and checks whether the value of the LO data count 504 is the maximum value in step 1733. At this time, if the value of the LO data count 504 is not the maximum value, the CNN apparatus can wait until the pooling operation result value is received in step 1741 (Wait for another pooling result data set which including L id and K id ). If the value of the LO data count 504 is the maximum value, the CNN apparatus increases the value of the kernel count 507 in step 1735 and checks whether the value of the kernel count 507 is the maximum value in step 1737. If the value of the kernel count 507 is not the maximum value, the CNN apparatus increases the end layer count value in step 1739 and proceeds to step 1741.

If the value of the kernel count 507 is the maximum value, the CNN device releases the input data at step 1743 (MMU, MMU :: CI reg (start layer release)) and releases the weight data at step 1745 (Weight data release at the PE Ctrl, PE Ctrl :: Weight (start layer) release). In step 1747, the CNN device increments the start layer count 502 and stores the result data in the LO reg (result data store LO reg, store address is PO id). The CNN device may then terminate the pooling operation at step 1751.

18 is a flow chart illustrating the output operation of the classification layer of the CNN device according to various embodiments of the present invention.

Referring to FIG. 18, in step 1811, the CNN apparatus may start the output operation of the FC layer (LO reg block start). In step 1813, the CNN apparatus may wait for receiving the convolution operation result data. When data is received from the convolution block, the CNN increases the LO data count by 1 in step 1815 and checks in step 1817 whether the LO data count is the maximum value (LO data cnt == MAX (#of FC layer output)) have. If the LO data count value is not the maximum value, the CNN apparatus returns to step 1813 and waits for the next input. If the value of the LO data count is the maximum value, the CNN device recognizes this in step 1817 and terminates the output operation (FC LO reg block end) in step 1819 and enables the operation of the MMU of the CNN control part 500.

CNN devices according to various embodiments of the present invention may be suitable for parallel processing of neuromorphic algorithms (parallel processing of neuromorphic algorithms) and are suitable for performing other deep running algorithms can do. The CNN apparatus according to various embodiments of the present invention may also be used in a processor that performs image recognition, classification, and analysis.

A CNN device according to various embodiments of the present invention can operate a CNN algorithm (convolutional neural network algorithm) at high speed and low power in a hardware device and can also be used in a portable device have. CNN devices can operate CNN algorithms of various sizes through configuration. The CNN device can have a zero pass function and can operate at a high speed in a convolution computation block (PE, ADD computation block). The CNN device can prefetch the weight data in the convolution operation block, which can reduce the load time of the memory and improve the processing speed of the deep learning algorithm.

While the invention has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention. Therefore, the scope of the present invention should not be limited by the described embodiments, but should be determined by the scope of the appended claims, as well as the appended claims.

Claims (30)

An information processing unit including a register for storing weight data;
At least one set of channels for performing a feature extraction layer and a classification layer operation based on input data and weight data; And
And a CNN control unit operatively connected to the information processing unit and the channel set,
Wherein the CNN control unit performs operation of the feature extraction layer when the channel set is set, and controls operation of the classpacific layer when the operation of the feature extraction layer is completed.
The method according to claim 1,
The channel set
An input unit for storing input data;
A convolution block for convoluting the input data and the weight data;
A pooling block for pooling the convoluted data; And
And an output section for storing the convolution operation data or the pooling operation data.
3. The method of claim 2,
The CNN control unit
And controls the convolution block and the pulling block of the channel set to perform a convolution layer and a pulling layer operation in the feature extraction layer operation.
The method of claim 3,
The CNN control unit
Upon completion of the feature extraction layer operation, a classification layer operation is performed,
And performs a connection layer operation by driving a convolution block of the channel set in a classification operation.
5. The method of claim 4,
The convolution block
A plurality of process elements for receiving input data and weight data and performing a multiplication operation;
A plurality of adders for adding the result values of the same ID calculated in said process elements;
An accumulator for accumulating summed result values of the same IDs output from the adders, and outputting the summed result values to a pooling operation unit when a maximum value is reached; And
And a process element control section for inputting the input data of the input section and the weight data of the weight register to the process elements prepared in the processor elements, respectively.
6. The method of claim 5,
The process element control unit
Fetching the corresponding weight data in the weight register by checking the start and end layer count values,
And inputs the weight data of the layer prefetched with the input data to the process element ready for calculation.
The method of claim 3,
The pulling block
Wherein when there are two or more convolution output values during a convolution operation in the convolution block, the convolution output values are pooled and stored in the output unit.
5. The method of claim 4,
The information processing unit may further include an address map register for storing an address map provided for each layer,
Wherein the CNN control unit confirms a current layer configuration of data stored in the output register, and accesses data of the address map register and outputs the data to the input unit.
9. The method of claim 8,
Wherein the address map is received at an initialization time point to an external system.
5. The method of claim 4,
The CNN control unit
If the input data is 0, a zero flag is set and zero padded input data is stored in the input unit,
The convolution block
And the convolution multiply operation of the input data in which the zero flag is set is omitted.
11. The method of claim 10,
The processor elements of the convolution unit
A multiplier for multiplying the input data and the weight data;
A calculator for logically computing the input data and the weight data; And
And a selector for selectively outputting the output of the multiplier or the zero data by the output of the operator,
And the selector selects the zero data if the input data or weight data is a zero flag.
12. The method of claim 11,
The process element control unit
And if the weight data is 0, sets the zero flag and inputs the zero flag to the process element.
5. The method of claim 4,
The CNN control unit
A composite neural network computation device for performing a feature extraction operation by inserting input data of remaining channels into the channel set when the channel count value is not a maximum value after completion of feature extraction of channel data,
5. The method of claim 4,
The CNN control unit
A resultant data of all the channels calculated in the feature extraction layer offset is input to one channel set when the first link layer operation of the classification layer is performed.
15. The method of claim 14,
The information processing unit
And a dropout signal generator for generating a dropout signal,
The convolution block
And when the dropout signal is generated, processes the input data as zero data.
16. The method of claim 15,
The CNN control unit
If the input data is 0, a zero flag is set,
The convolution block
And a plurality of progress elements for performing a convolutional multiply operation on the input data and the weight data, wherein the process element performs a convolution multiply operation when the input data in which the zero flag is set, the weight data in which the zero flag is set, To the neural network.
An information processing unit including a register for storing weight data, and at least one channel set including a convolution block and a pooling block,
Performing operations of a feature extraction layer by activating a convolution block and a pulling block of the channel set;
And when the operation of the feature extraction layer is completed, activating a convolution block of the channel set to perform a class-occupation layer operation. 18. The method of claim 17,
The process of performing the feature extraction layer operation
Storing input data;
Performing a convolution operation on the input data and the weight data;
Performing a pooling operation on the convoluted data; And
And outputting the pulling operation data.
19. The method of claim 18,
The convolution operation process
A step of multiplying the input data and the weighted data;
Adding result values of the same ID among the multiplication result values;
And accumulating summed result values of the same IDs.
20. The method of claim 19,
The convolution operation process
Further comprising the step of prefetching the corresponding weight data in the weight register by checking the start and end layer count values.
19. The method of claim 18,
The pooling operation process
And if there are two or more convolution operation output values in the convolution process, performing the pulling operation on the convolution output values.
22. The method of claim 21,
The pooling operation process
And selecting a maximum value among the two convolution operation output values.
22. The method of claim 21,
The pooling operation process
And calculating an average value of the two convolution operation output values.
19. The method of claim 18,
The information processing unit may further include an address map register for storing an address map provided for each layer,
The process of storing the input data
An operation method of a composite neural network computation apparatus for checking the layer configuration of the polling operation output data and mapping the polling operation output data to an address of an address map register according to the confirmed layer configuration and storing the polling operation output data as input data .
25. The method of claim 24,
And receiving the address map at an initialization time point to an external system.
19. The method of claim 18,
The process of storing the input data
If the input data is 0, the zero flag is set,
The convolution operation process
And the multiplication operation of the input data in which the zero flag is set is omitted.
27. The method of claim 26,
The convolution operation process
And a multiplier for multiplying the input data and the weight data,
Analyzing a zero flag of the input data and the weight data;
Outputting zero data if the zero flag of the input data or weight data is set; And
And outputting the multiplier if the zero flag of the input data and the weight data is in a reset state.
The method of claim 22, wherein
The process of performing the feature extraction operation
A step of checking a channel count value for which feature extraction of channel data is completed; And
And performing the feature extraction operation by assigning input data of remaining channels to the channel set if the channel count value is not a maximum value.
19. The method of claim 18,
The process of performing the operation of the classification layer
Inputting result data of all channels extracted from the feature extraction layer to one channel set;
Performing a convolution operation on the input data and the weight data; And
And outputting the convolution operation data.
30. The method of claim 29,
The information processing unit
And a dropout signal generator for generating a dropout signal,
The convolution operation process
And a plurality of progressive elements for performing a convolutional multiply operation on the input data and the weight data,
Wherein the process element skips a convolutional multiply operation when input data in which the zero flag is set, weight data in which a zero flag is set, or a dropout signal is input.

Images