KR101950786B1 - Acceleration Method for Artificial Neural Network System - Google Patents

Abstract

The present invention relates to a method of accelerating the calculation of an artificial neural network for distributed processing. According to the present invention, an artificial neural network includes a single host accelerating device and a plurality of slave accelerating devices so input data about input neurons is able to be M in depth and have N hierarchies. A synapse weighted value and input data about neurons forming an artificial neural network are able to be calculated through each accelerating device since the input data is divided by depth, a hierarchical structure, an artificial neural network, or combinations thereof, and thus, memories and peripheral devices are able to be optimized when formed as hardware. As a result, a plurality of accelerating devices comprising low-cost/low-power integrated circuits are connected to conduct neural network calculation and thus, the present invention is capable of saving costs for manufacturing, as compared to conventional technology with high-priced hardware, and being applied to fields requiring low power.

Description

{Acceleration Method for Artificial Neural Network System}

The present invention relates to a method for accelerating an artificial neural network operation for distributed processing, and more particularly, to a method for accelerating an artificial neural network operation for distributed processing, in which data on input neurons necessary for operation of an artificial neural network and synapse weights are transmitted to a plurality of artificial neural networks It is possible to satisfy the performance required for the artificial neural network operation without using high-power and high-cost hardware such as GPGPU by dispersing and processing the artificial neural network in the artificial neural network, And an operation acceleration method.

Recently, artificial intelligence technology has been introduced into various industrial fields with the development of artificial intelligence technology.

Deep Learning (Deep Learning) technology, which has evolved into a neural network technology composed of neuron networks, plays an important role in this artificial intelligence technology, which is based on a perceptron algorithm that receives a large number of signals on the back and outputs one signal. .

In order to perform artificial neural network technology such as the deep learning technique, many weights and computations for input neurons are required. Therefore, in the case of artificial neural network-related acceleration techniques according to the related art, a graphics processing unit (GPU) It is difficult to satisfy the required computation performance without using expensive hardware such as general purpose graphics processing unit (GPGPU) capable of performing general-purpose operations other than graphic operations using resources There was a problem.

In addition, since the GPGPU requires high power to operate the GPGPU, it is difficult to apply it to fields requiring low power such as Internet Of Things (IOT). In order to process big data with deep learning technology In the case of a data center where a large-scale computer system is installed, since a large-scale power is required while constructing an acceleration device necessary for realizing the artificial neural network technology as a GPGPU, a serious problem in which the initial construction cost and the maintenance cost increase exponentially .

Therefore, in the case of implementing the artificial neural network related acceleration technique, there is a desperate need for realistic and applicable technology capable of satisfying computation performance without requiring expensive hardware such as GPGPU and further reducing power consumption. to be.

Patent Publication No. KR 10-2011-0027916 (published on March 17, 2011)

SUMMARY OF THE INVENTION The present invention has been conceived in order to solve the above problems, and it is an object of the present invention to provide a neural network, which comprises a plurality of neural network processing accelerators for distributed processing, , It is possible to satisfy the performance required for the artificial neural network operation without using the high power and high cost hardware such as GPGPU, and to provide an artificial neural network operation acceleration device, an acceleration system And an acceleration method thereof.

An apparatus for accelerating an artificial neural network for distributed processing according to an embodiment of the present invention is an apparatus for processing an artificial neural network operation in which input neurons are formed hierarchically, comprising: input data and a synapse weight for the input neurons; An external main memory for storing; An internal buffer memory for storing synapse weights and input data required for each cycle constituting the artificial neural network operation among the synapse weights stored in the external main memory and input data; A DMA module for directly transmitting and receiving data to and from the external main memory and the internal buffer memory; A series of sequential processes of reading the synapse weights and input data stored in the internal buffer memory and performing an artificial neural network operation and storing the result of the operation in the external main memory are repeatedly performed for each cycle constituting the artificial neural network operation Neural network computing; A CPU for storing input data and synapse weights for input neurons in the external main memory and the internal buffer memory, and for controlling operations of the neural network computation apparatus; And a universal communication mediating block capable of transmitting and receiving input data and synapse weight for the input neurons and an operation result performed by the neural network computation device to and from another acceleration device physically connected regardless of the type of the integrated circuit can do.

The artificial neural network operation accelerating apparatus for distributed processing according to an embodiment of the present invention may further comprise a data input apparatus composed of a sensor interface or a peripheral.

The apparatus for accelerating an artificial neural network for a distributed processing according to an embodiment of the present invention may further include an external flash memory for storing the total synapse weights necessary for performing the artificial neural network operation.

The general-purpose communication mediating block can be formed to be capable of communication mediation when the type of the integrated circuit constituting the physically connected accelerating device is SoC (System on Chip) or FPGA (Field Programmable Gate Array).

The general-purpose communication mediating block includes a remapping block for remapping an address specifying an element of the receiving integrated circuit among signals applied from the bus master interface connected to the transmitting integrated circuit and a width of the bus ID signal .

An artificial neural network acceleration system using an artificial neural network operation acceleration apparatus for distributed processing according to an embodiment of the present invention is an acceleration system for processing an artificial neural network operation in which input data for input neurons have a plurality of depths, A host accelerator connected to a flash memory for storing a total synapse weight necessary for a neural network operation and having a communication mediating block for a host at a plurality of depths or more; And a plurality of slave acceleration devices each having at least one slave communication intermediary block physically connected to a host communication intermediary block of the host accelerator and corresponding to the plurality of depth numbers one by one, The host acceleration device causes the slave acceleration device related to each of the plurality of depths to distribute the synapse weight and the input data in a parallel manner to process the neural network operation and collects the intermediate calculation results of the slave acceleration device Can be performed.

An artificial neural network acceleration system using an artificial neural network operation acceleration apparatus for distributed processing according to an embodiment of the present invention is an acceleration system for processing an artificial neural network operation in which input neurons are composed of a plurality of layers, A host accelerator connected to a flash memory storing a total synapse weight and having at least one pair for transmitting and receiving a host communication intermediary block; And a plurality of slave acceleration devices each having at least one pair for transmitting and receiving a communication mediating block for a slave to be sequentially connected in a pipeline form to the host accelerator device, The neural network operation may be performed by sequentially dispersing the synaptic weights and the input data for the input neurons constituting the artificial neural network operation.

Each of the host acceleration device and the slave acceleration device may be configured as an integrated circuit of any one of SoC scheme and FPGA scheme, and may be configured as a communication intermediation block for host and a slave communication intermediation It is possible to mutually transmit and receive each other through the blocks.

Wherein the host accelerator and the slave accelerator each comprise an external main memory for storing input data and synapse weights for input neurons; An internal buffer memory for storing synapse weights and input data required for each cycle constituting the artificial neural network operation among the synapse weights stored in the external main memory and the input data; A DMA module for directly transmitting and receiving data to and from the external main memory and the internal buffer memory; A series of sequential processes of reading the synapse weights and input data stored in the internal buffer memory and performing an artificial neural network operation and storing the result of the operation in the external main memory are repeatedly performed for each cycle constituting the artificial neural network operation Neural network computing; And a CPU for storing input data and synapse weights for the input neurons in the external main memory and the internal buffer memory and for controlling the operation of the neural network operation apparatus.

The communication mediating block for a host and the communication mediating block for a slave may operate as a master and a slave by simultaneously having a bus master interface and a bus slave interface, And a remapping block for remapping an address designating a component of the receiving integrated circuit and a width of the bus ID signal.

An artificial neural network acceleration system using an artificial neural network operation acceleration apparatus for distributed processing according to an embodiment of the present invention is characterized in that input data to input neurons includes an artificial neural network operation having a plurality of (M) depths and a plurality (N) A host acceleration device connected to a flash memory storing a total synaptic weight necessary for the artificial neural network operation and having a communication mediating block for a host at least equal to the number of the plurality of (N) hierarchical levels; And a communication mediating block for a slave connected to the host mediating block or connected to another accelerating device of the host accelerator device, And a plurality of slave acceleration devices formed in a one-to-one correspondence with the number of cycles (MxN).

Each of the host acceleration device and the slave acceleration device may be configured as an integrated circuit of any one of SoC scheme and FPGA scheme, and may be configured as a communication intermediation block for host and a slave communication intermediation It is possible to mutually transmit and receive each other through the blocks.

Wherein the host accelerator and the slave accelerator each comprise an external main memory for storing input data and synapse weights for input neurons; An internal buffer memory for storing synapse weights and input data required for each cycle constituting the artificial neural network operation among the synapse weights stored in the external main memory and the input data; A DMA module for directly transmitting and receiving data to and from the external main memory and the internal buffer memory; A series of sequential processes of reading the synapse weights and input data stored in the internal buffer memory and performing an artificial neural network operation and storing the result of the operation in the external main memory are repeatedly performed for each cycle constituting the artificial neural network operation Neural network computing; And a CPU for storing input data and synapse weights for the input neurons in the external main memory and the internal buffer memory and for controlling the operation of the neural network operation apparatus.

The communication intermediation block for a host and the communication intermediation block for a slave communicate with each other through an address specifying an element of the receiving integrated circuit among signals applied from the bus master interface connected to the transmitting integrated circuit and a width of a bus ID signal And a remapping block for remapping.

In an artificial neural network acceleration system using an artificial neural network operation acceleration apparatus for distributed processing according to an embodiment of the present invention, in an artificial neural network acceleration system composed of a plurality of artificial neural networks by at least one artificial neural network acceleration system, The input data and the synaptic weights for each of the plurality of artificial neural networks can be divided and processed for each of the artificial neural network units.

A method for accelerating an artificial neural network using an artificial neural network operation acceleration apparatus for distributed processing according to an embodiment of the present invention includes an input layer and N hidden layers using an artificial neural network operation acceleration apparatus for a distributed processing composed of a single integrated circuit A method for accelerating an artificial neural network processing having a hierarchical structure, the method comprising the steps of: (a1) storing the entire synaptic weights of input neurons for an artificial neural network operation stored in an external flash memory when power is applied to the acceleration apparatus; ); (A2) storing initial input data input through a data input device via an DMA module in an external main memory; (A3) storing input data stored in the external main memory and synaptic weights corresponding to the input data in an internal buffer memory as needed for each cycle constituting an input layer of an artificial neural network; The neural network computation unit reads the synapse weight and input data corresponding to each cycle constituting the artificial neural network and stores the synaptic weight and input data in the internal buffer memory and performs an artificial neural network operation until the operation for the entire layer is completed, Storing (a4) in the external main memory for use as input data of the next layer; A process for reading the synaptic weights and input data for input neurons for artificial neural network operations on hidden layers from the external main memory as needed for each cycle and storing them in an internal buffer memory and then performing the step a4 is performed on N hidden layers (A5) repeatedly performing the step (a5).

The method for accelerating an artificial neural network using an artificial neural network operation acceleration apparatus for distributed processing according to an embodiment of the present invention includes a single host acceleration apparatus and a plurality of slave acceleration apparatuses so that the input data for input neurons has M depths Wherein when a power is applied to the single host accelerator apparatus and the plurality of slave accelerator apparatuses, a synapse weight for input neurons for an artificial neural network operation stored in an external flash memory is allotted to the host accelerator Storing (b1) in an external main memory of the device; Wherein the host accelerator sends each synapse weight corresponding to the M depths among the synapse weights stored in the external main memory of the host accelerator apparatus to the external main memory of the plurality of slave accelerator apparatuses in a parallel manner (B2); Storing (b3) a synapse weight necessary for each cycle constituting the input layer of the artificial neural network among synapse weights stored in the respective external main memories of the plurality of slave acceleration devices, in the internal buffer memory of each slave acceleration device; (B4) storing the first input data input by the single host accelerator device through the data input device in the external main memory of the host accelerator device; The host accelerator transmits each input data corresponding to the M depths of the input data stored in the external main memory of the host accelerator device to the external main memory of the plurality of slave accelerator devices in a parallel manner (B5); (B6) storing input data stored in an external main memory of each of the slave acceleration devices in the slave acceleration device internal buffer memory as necessary for each cycle constituting the artificial neural network; The neural network computing device of each of the slave acceleration devices reads the synaptic weight and input data stored in the internal buffer memory of each of the slave acceleration devices corresponding to each cycle constituting the artificial neural network, (B7) performing an artificial neural network operation up to the slave acceleration device and storing the operation result in the external main memory of each of the slave acceleration devices; The host acceleration device receives the intermediate calculation result stored in the external main memory of each of the slave acceleration devices, sequentially stores the intermediate result in the external main memory of the host acceleration device, (B8) in an external main memory of the host accelerator apparatus; In order to use the final computation result stored in the external main memory of the host accelerator as input data for operation of the next layer, sequentially in parallel manner corresponding to M depths in each external main memory of the plurality of slave accelerators (B9); (B10) storing (b10) a synapse weight necessary for each cycle constituting a next layer among synapse weights stored in each external main memory of the plurality of slave acceleration devices, in an internal buffer memory of each slave acceleration device; And (b11) repeating the steps b6 to b10 until all layers of the input data are completely computed.

The method for accelerating an artificial neural network using an artificial neural network operation acceleration apparatus for distributed processing according to an embodiment of the present invention includes a single host acceleration apparatus and a plurality of slave acceleration apparatuses so that the input data for input neurons has M depths Wherein when a power is applied to the single host accelerator apparatus and the plurality of slave accelerator apparatuses, a synapse weight for input neurons for an artificial neural network operation stored in an external flash memory is allotted to the host accelerator Storing (c1) in the external main memory of the device; Wherein the host accelerator sends each synapse weight corresponding to the M depths among the synapse weights stored in the external main memory of the host accelerator apparatus to the external main memory of the plurality of slave accelerator apparatuses in a parallel manner And c2. (C3) storing a synaptic weight necessary for each cycle constituting the input layer of the artificial neural network, among the synaptic weights stored in the respective external main memories of the plurality of slave ac- celerators, in an internal buffer memory of each of the slave ac- celerators; (C4) storing the first input data input from the single host accelerator through the data input device in the external main memory of the host accelerator; The host accelerator stores the input data stored in the external main memory and the synaptic weights in the internal buffer memory as needed for a cycle constituting the input layer of the neural network, performs a neural network operation on the input layer using the neural network processor, (C5) in an external main memory for use as input data of a next layer; The host accelerator transmits each input data corresponding to the M depths of the input data stored in the external main memory of the host accelerator device to the external main memory of the plurality of slave accelerator devices in a parallel manner (C6) transmitting and storing the data sequentially; (C7) storing the input data stored in the external main memory of each of the slave acceleration devices and the synaptic weights corresponding thereto in the slave acceleration device internal buffer memory as necessary for each cycle constituting the artificial neural network; Wherein the neural network computing device of each of the slave acceleration devices repeatedly reads the synapse weight and input data stored in the internal buffer memory of each of the slave acceleration devices corresponding to each cycle constituting the artificial neural network, (C8) repeatedly performing an artificial neural network operation until the operation for the slave acceleration device is completed and storing the operation result in the external main memory of each of the slave acceleration devices; And the host accelerator receives the intermediate operation result stored in the external main memory of each of the slave accelerators, sequentially stores the intermediate result in the external main memory of the host accelerator, and then collects the final result, And storing the result in the external main memory of the host accelerator (c9).

A method of accelerating an artificial neural network using an artificial neural network operation accelerating apparatus for distributed processing according to an embodiment of the present invention is a method in which a connection structure of a single host accelerator apparatus and a plurality of slave accepter apparatuses are sequentially arranged in a pipeline form, A method for accelerating an artificial neural network processing having input data for neurons having a hierarchy including an input layer and N hidden layers, the method comprising: when power is applied to the single host acceleration device and the plurality of slave acceleration devices, (D1) storing the entire synaptic weights for input neurons for artificial neural network operations stored in an external main memory of the host acceleration device; Wherein the host accelerator sequentially transmits respective synapse weights corresponding to the N layers among the synapse weights stored in the external main memory of the host accelerator through the universal communication mediation block to each of the external main memories of the plurality of slave accelerators Transmitting and storing (d2); (D3) storing the input data of the input layer to which the single host accelerator device is input through the data input device in the external main memory of the host accelerator device; Storing input data and a synapse weight of an input layer stored in an external main memory of the host accelerator device in an internal buffer memory (d4); A step (d5) of reading a synapse weight value and input data stored in the internal buffer memory by the neural network computation device of the host acceleration device, performing a neural network operation on the input layer, and storing the computation result in the external main memory; (D6) storing the operation result in the external main memory of the slave acceleration device corresponding to the next layer as input data, and then storing the input data and the synapse weight corresponding to the layer in the internal buffer memory of the slave acceleration device; A step (d7) of reading the synaptic weights and input data stored in the internal buffer memory by the neural network calculator of the slave acceleration apparatus, performing a neural network operation on the layer, and storing the operation result in the external main memory; (D8) repeatedly performing the steps d6 to d7 for the N hidden layers and storing the final operation result in the external main memory of the host accelerator or transmitting the result to the peripheral device.

A method for accelerating an artificial neural network using an artificial neural network operation acceleration apparatus for distributed processing according to an embodiment of the present invention includes a single host acceleration apparatus and a plurality of slave acceleration apparatuses so that input data for input neurons has M depths A method for accelerating an artificial neural network process comprising N layers, the method comprising the steps of: when a power is applied to the single host accelerator and a plurality of slave accelerators, a synapse weight for input neurons for an artificial neural network operation stored in an external flash memory Storing (e1) the entirety in an external main memory of the host acceleration device; The host accelerator transmits the entire synapse weights corresponding to the N layers among the synapse weights stored in the external main memory of the host accelerator through the universal communication mediation block to the first of each of the M depths constituting each layer (E2) in which the respective synapse weights are stored in all the slave acceleration devices corresponding to different depths connected to the received N slave acceleration devices, respectively, when sequentially transmitted to the external main memory of each of the N slave acceleration devices corresponding to the depth, ; (E3) storing the input data of the input layer input from the single host accelerator through the data input device in the external main memory of the host accelerator; The input data of the input layer stored in the external main memory of the host accelerator is firstly stored in the external main memory of the slave acceleration device corresponding to the first depth of the input layer, (E4) in which input data corresponding to M slave accelerating apparatus external main memories corresponding to the plurality of slave accelerating apparatuses are dispersed and sequentially stored; (E5) storing input data and synapse weights of the layer in M slave accelerator internal buffer memories constituting the input layer; (E6) performing a neural network operation by the neural network computation apparatus of the M slave acceleration apparatuses constituting the input layer and storing the computation result in its own external main memory; The calculation result stored in the M slave acceleration devices is transmitted to the slave acceleration device corresponding to the first depth of the input layer to perform the final calculation of the corresponding layer and the final calculation result is transmitted to the slave acceleration device corresponding to the first depth of the next layer (E7) transmitting the input data to the layer; The entire input data of the corresponding layer is firstly stored in the external main memory of the slave acceleration device corresponding to the first depth of the next layer and then the input data corresponding to the M slave accelerator external main memories constituting the corresponding layer are dispersed sequentially (E8); And repeating steps e5 through e8 until the operation of the Nth layer is completed, and transmitting the final operation result to the host accelerator (e9).

A method for accelerating an artificial neural network using an artificial neural network operation accelerating apparatus for distributed processing according to an embodiment of the present invention is a method for accelerating an artificial neural network in which input data for input neurons have a plurality of depths, Input data for input neurons and synapse weights can be distributed and processed in each of the corresponding acceleration devices.

A method for accelerating an artificial neural network using an artificial neural network operation accelerator for distributed processing according to an embodiment of the present invention is a method for accelerating an artificial neural network in which input neurons are configured in a plurality of hierarchical structures, The input data for the input neurons and the synaptic weights can be distributed and processed.

A method of accelerating an artificial neural network using an artificial neural network operation accelerating apparatus for distributed processing according to an embodiment of the present invention is a method for accelerating an artificial neural network in which input data for input neurons are composed of a plurality of depths and a plurality of hierarchical structures , Input data for input neurons and synapse weights can be distributed and processed in each of the accelerators corresponding to a plurality of depths constituting a plurality of layers.

A method for accelerating an artificial neural network using an artificial neural network operation accelerating apparatus for distributed processing according to an embodiment of the present invention is a method for accelerating a compound artificial neural network composed of a plurality of artificial neural networks by at least one artificial neural network acceleration method, The input data and the synaptic weights of the plurality of artificial neural networks can be divided and processed into respective artificial neural network units constituting the plurality of artificial neural networks.

As described above, according to the present invention, data on input neurons necessary for computation of an artificial neural network and synapse weights are transmitted to a plurality of artificial dispersion processing artificial neural networks having universal communication mediating blocks capable of mediating communication irrespective of kinds of integrated circuits It is possible to satisfy the performance required for the artificial neural network operation without using the high power and high cost hardware such as GPGPU by dispersing and processing in the neural network operation accelerating device and to accelerate artificial neural network operation which can design the artificial neural network flexibly according to the target performance An apparatus, an acceleration system, and an acceleration method therefor.

In addition, the present invention reduces the manufacturing cost compared with the use of an expensive GPGPU implemented by a single hardware according to the prior art by connecting a plurality of accelerators composed of low-power / low-cost integrated circuits to perform neural network calculation And can be applied to fields requiring low power.

Further, the present invention can flexibly apply additional functions to an acceleration system implemented by the same type or a heterogeneous type of integrated circuit by implementing the universal communication mediating block capable of communication mediation regardless of the type of the integrated circuit in the acceleration device It is possible to actively cope with various demands of users.

Further, the present invention has the effect of expanding or reducing the acceleration device flexibly according to the target performance by performing the distributed parallel processing of the artificial neural network using the multiple acceleration device.

Further, according to the present invention, input data and synapse weights for neurons constituting an artificial neural network are divided into a depth, a hierarchical structure, a neural network, or a combination unit constituting input data, In the case of hardware implementation, the memory and the peripheral device can be optimized, and as a result, the product development cost can be lowered.

In addition, the present invention realizes an accelerating system using various kinds of integrated circuits, and thus can be actively used in various artificial neural network structures to be applied in the future.

FIG. 1 is a conceptual diagram schematically illustrating a neural network including an input layer and a hidden layer to which an embodiment of the present invention is applied.
FIG. 2 is a schematic diagram of a neural network that extends the artificial neural network shown in FIG.
3 is a block diagram schematically showing the configuration of an integrated circuit of an artificial neural network operation acceleration apparatus for distributed processing according to an embodiment of the present invention.
Fig. 4 is a diagram showing a detailed configuration of a general-purpose communication mediating block constituting the acceleration device shown in Fig. 3;
5 is a block diagram schematically illustrating a multi-integrated circuit configuration of an acceleration system for processing an artificial neural network operation in which input data has a plurality of depths according to an embodiment of the present invention.
6 is a diagram schematically illustrating an artificial neural network model according to an embodiment of the acceleration system shown in FIG.
FIG. 7 is a diagram schematically showing an artificial neural network model according to another embodiment of the acceleration system shown in FIG. 5; FIG.
8 is a block diagram schematically illustrating a multi-integrated circuit configuration of an acceleration system for processing an artificial neural network operation in which input neurons are formed in a plurality of layers according to an embodiment of the present invention.
9 is a diagram schematically illustrating an artificial neural network model according to an embodiment of the acceleration system shown in FIG.
10 is a block diagram schematically showing a multi-integrated circuit configuration of an acceleration system for processing an artificial neural network operation composed of a plurality (M) of depths and a plurality (N) of layers.
11 is a diagram schematically illustrating an artificial neural network model according to an embodiment of the acceleration system shown in FIG.
12 is a diagram schematically illustrating a combined artificial neural network model including a plurality of artificial neural networks according to an embodiment of the present invention.

The description of the present invention is merely an example for structural or functional explanation, and the scope of the present invention should not be construed as being limited by the embodiments described in the text. That is, the embodiments are to be construed as being variously embodied and having various forms, so that the scope of the present invention should be understood to include equivalents capable of realizing technical ideas.

Meanwhile, the meaning of the terms described in the present invention should be understood as follows.

The terms "first "," second ", and the like are intended to distinguish one element from another, and the scope of the right should not be limited by these terms. For example, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

It is to be understood that when an element is referred to as being "connected" to another element, it may be directly connected to the other element, but there may be other elements in between. On the other hand, when an element is referred to as being "directly connected" to another element, it should be understood that there are no other elements in between. On the other hand, other expressions that describe the relationship between components, such as "between" and "between" or "neighboring to" and "directly adjacent to" should be interpreted as well.

It is to be understood that the singular " include " or "have" are to be construed as including the stated feature, number, step, operation, It is to be understood that the combination is intended to specify that it does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

In each step, the identification code (e.g., a, b, c, etc.) is used for convenience of explanation, the identification code does not describe the order of each step, Unless otherwise stated, it may occur differently from the stated order. That is, each step may occur in the same order as described, may be performed substantially concurrently, or may be performed in reverse order.

All terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined. Commonly used predefined terms should be interpreted to be consistent with the meanings in the context of the related art and can not be interpreted as having ideal or overly formal meaning unless explicitly defined in the present invention.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

FIG. 1 is a conceptual diagram schematically showing a model of a neural network including an input layer, a hidden layer and an output layer to which an embodiment of the present invention is applied.

As shown in the figure, the artificial neural network model of FIG. 1 has a hierarchical structure including an input layer, a plurality of hidden layers, and an output layer.

Here, the circles in each layer are called neurons, and the arrows connected to the neurons in the next layer in each neuron refer to a neuron junction, called a synapse.

For example, x1 represents one of the neurons of the input layer, and a1 represents one of the neurons of the hidden layer-1.

In order to express the neural network model of FIG. 1 mathematically, when A is a calculation result, W is a weight, X is an input, and B is a bias, Respectively.

[Equation 1]

In addition, the expression (1) can be simplified by expressing it using an inner product of a matrix and expressed by the following equation (2).

&Quot; (2) "

FIG. 2 is a schematic diagram of a neural network that extends the artificial neural network shown in FIG.

As shown in FIG. 2, an artificial neural network having two or more hidden layers is referred to as a deep neural network.

The artificial neural network applied to the embodiment of the present invention is composed of an in-depth neural network that is mainly applied to an image field. Generally, data corresponding to input neurons correspond to a multi-dimensional (N-Tensor) And can be expressed in a simplified form as a feature map.

That is, as shown in the figure, in the embodiment of the present invention, the feature map constituting each layer can be represented by a dimension composed of Width, Height, and Depth, wherein Depth is a dimension of Width and Height In the present specification, the dimension extended by the term 'depth' can be expressed numerically as Depth.

More specifically, 'depth' is used when dealing with the expansion of a dimension in Convolutional Neural Network (Convolution Neural Network) to which an embodiment of the present invention is applied. In general neural network, for example, It is represented by 224 × 224 × 3 (horizontal, vertical and color channels) in the case of general RGB (Red, Green, Blue), which corresponds to the horizontal, vertical and depth of the convolutional neural network, have.

That is, in the convolution neural network according to the embodiment of the present invention, when the input is an image, each layer has three dimensions of horizontal, vertical, and depth, where the depth is not the depth of the entire neural network, We can talk about the third dimension in the activation volume corresponding to the activation volume.

In addition, the arrows shown between the respective layers shown in FIG. 2 are simplified representations of transferring the operation result of each layer to the input of the next adjacent layer.

Here, Layer N is an expression of Fully Connected Layer, which can be expressed as a dimension of [input, output] as an output of a reconstructed input of a dimension of a previous layer in one dimension, and output of an output layer Can be used as the final computation result.

3 is a block diagram schematically showing the configuration of an integrated circuit of an artificial neural network operation acceleration apparatus for distributed processing according to an embodiment of the present invention.

3 is an apparatus for implementing the distributed processing of an artificial neural network, which is a core idea of the present invention, in which input data of an artificial neural network formed by hierarchical construction of input neurons and operations on a synapse weight are distributed and processed Since the universal communication mediating block 100, which will be described later, can be connected regardless of the type of the integrated circuit, it is possible to connect various integrated circuits of the FPGA or SoC type to each other, It is possible.

The artificial neural network operation acceleration apparatus for distributed processing according to an embodiment of the present invention will now be described in detail with reference to the drawings.

The artificial neural network operation acceleration apparatus 10 for distributed processing according to the embodiment of the present invention includes a flash memory 1, an external main memory 11, an internal buffer memory 12, a DMA module 13, a neural network computing device 14, a CPU 15, a data input device 16, and a communication mediation block 100.

More specifically, the flash memory 1 is constituted by a nonvolatile memory such as a SD (Secure Digital) card. The entire synaptic weight required for the artificial neural network operation according to the embodiment of the present invention is stored, It can be transferred to the external main memory 11 via the DMA module 13 by an operation control signal of the CPU.

The external main memory 11 is a universal storage unit that can store input data and synapse weights for the input neurons under the management of an operating system in order to perform an artificial neural network operation according to an embodiment of the present invention. Or SDRAM (Dual Data Rate SDRAM), and can be used only when power is supplied to the acceleration device. In the case where the 'external memory' is described in this specification, the external main memory 11 may be referred to .

Meanwhile, as shown in the figure, the flash memory 1 and the external main memory 11 can be provided with a memory interface 2 having control logic for writing or reading data.

The internal buffer memory 12 temporarily stores the synaptic weights and input data required for each cycle constituting the artificial neural network operation among the synaptic weights stored in the external main memory 11 and the input data, And may store all or a part of the input data and synapse weights stored in the external main memory 11 according to the storage capacity.

Also, The DMA module 13 is configured to directly transmit and receive data to and from the external main memory 11 and the internal buffer memory 12. The DMA module 13 includes a DMA (Direct Memory Access) (Direct Memory Access) function.

Here, the DMA module 13 applied to the embodiment of the present invention will be described in more detail. For example, when a central processing unit such as a CPU 15, which will be described later, reads data with a peripheral device having a relatively low processing speed / It can be used to substitute data write / write for the purpose of improving the original usage of hardware operator by avoiding waiting for another operation until writing is completed.

Particularly, in the embodiment of the present invention, as described above,

'1) The weight stored in the external flash memory 1 is stored in the external main memory 11

2) storing the weight values and input data stored in the external main memory 11 in the internal buffer memory 12

3) transferring the calculation result stored in the external main memory 11 to another acceleration device through a peripheral device interface such as a general-purpose communication mediation block 100

4) The result of the operation of the neural network arithmetic unit 14 to be described later is stored in the external main memory 11

5) Input data of a peripheral device such as a data input device 16 to be described later is stored in the external main memory 11

6) The operation result stored in the external main memory 11 of the acceleration device based on the general-purpose communication mediating block 100 described later is transferred to the external main memory 11 of the other acceleration device as input data,

Can be used for the same purpose.

In addition, the neural network computing device 14 may include a series of arithmetic processing units for reading the synapse weights and input data stored in the internal buffer memory 12 to perform an artificial neural network operation and then storing the operation results in the external main memory 11 A sequential process can be repeatedly processed for each cycle constituting an artificial neural network operation.

The CPU 15 controls the operation of storing the input data and the synapse weight for the input neurons in the external main memory 11 and the internal buffer memory 12 and the operation of the neural network computation apparatus 14 And can operate as a central processing unit of the acceleration device,

The data input device 16 is a data input device configured by a sensor interface or a peripheral. When the input data is an image as in the embodiment of the present invention, And stores the received data in the external main memory 11.

The general-purpose communication mediating block 100 may further include input data and synapse weights for the input neurons, and calculation results obtained by the neural network computing device 14, It can transmit and receive with the acceleration device.

At this time, the general-purpose communication mediating block 100 can perform transmission / reception of addresses, data, and control signals with other accelerators, and the types of integrated circuits constituting the physically- System on Chip) or FPGA (Field Programmable Gate Array).

According to the embodiment of the present invention, it is preferable that the bus 3 protocol for communication between elements on the integrated circuit of the accelerator is an industry standard AMBA (Advanced Microcontroller Bus Architecture) The block 100 may receive an Advanced Extensible Interface (AXI) signal or an Advanced High Performance Bus (AHB) signal and may convert the signal into a communication input / output interface signal between the acceleration devices.

The general-purpose communication mediation block 100 included in the acceleration device 10 according to the embodiment of the present invention will be described in detail with reference to FIG.

Fig. 4 is a diagram showing a detailed configuration of a general-purpose communication mediating block constituting the acceleration device shown in Fig. 3;

As shown in the figure, the general purpose communication mediating block 100 constituting the acceleration apparatus according to the embodiment of the present invention includes a remapping block 110, a bus control signal matching block 120, a monitor block 130, And a transmission module 101 and a reception module 102 corresponding to the transmission / reception interface.

According to an embodiment of the present invention, a plurality of physically connected accelerators are connected to a bus control signal (Bus Control Signal: Bus ID, Burst, Size, R / W ...) An address signal (Bus Address Signal), and a bus data signal (Bus Data Signal) are transmitted and received, and a message generated by the processor and an interrupt signal can be transmitted.

Here, the remapping block 110 determines the address and the width of the bus ID signal designating the components of the receiving integrated circuit among the signals applied from the bus master interface connected to the transmitting integrated circuit of the accelerator, And re-mapping.

More specifically, the addresses of the internal components applied by the integrated circuit constituting each of the acceleration devices are different from each other. Without taking such a difference into account, an arbitrary accelerator (transmitting side) ), It is of course not possible to access the target area due to an address conflict. For example, the unique addresses assigned to DDR, Flash, CPU, and Video Codec are different for each acceleration device. Therefore, in order to perform normal communication, it is necessary to have a promise of remapping addresses in advance.

The remapping block 110 may perform bus ID remapping to match the bus ID signal applied from the bus master interface with the width of the bus ID signal of the neighboring apparatus (receiving side).

Here, since the address channel and the data channel are independently separated from each other in the AXI bus standard, the address-data pair can be matched using the bus ID because the address and data can not be connected by the numerical access method. Due to the use of such a bus ID, the bus ID width may be different for each configuration of the bus layer. The different chips are composed of different bus systems.

The difference results in a difference in bus ID width. Therefore, bus ID widths must be matched for communication between the buses of different chips, which means that bus ID widths are expanded or decreased as needed. As with address re-mapping, .

Therefore, it is preferable that the remapping block 310 includes a remapping appointment table to be referred to when performing remapping to match the address and the width of the bus ID signal.

Also, in an embodiment of the present invention, the bus control signal matching block 120 analyzes a pattern of a bus control signal among signals applied from the bus master interface, O pins of the communication interface between the accelerating apparatuses can be reused when the bus control signals applied in the following sequence are the same so that the previous bus control signals can be reused, The utilization rate can be maximized and consequently the communication speed can be maximized.

In addition, according to the embodiment of the present invention, the monitor block 130 can monitor a message applied from the processor interface and all the interrupt signals applied from the interrupt interface, When the address signal and the data are transmitted, the interrupt signal and the processor's message are transmitted with the highest priority by transmitting the message and the interrupt signal including the interrupt signal, thereby maximizing the communication efficiency.

2 to 4, an artificial neural network operation acceleration apparatus for distributed processing, which is composed of a single integrated circuit, is used to calculate a neural network model for the artificial neural network model shown in FIG. 2 having a hierarchical structure including an input layer and N hidden layers A method of accelerating the processing will be described as follows.

First, when power is applied to the acceleration device 10, the entire synaptic weights of the input neurons for the artificial neural network operation stored in the external flash memory 1 are stored in the external main memory by the control signal of the CPU 15 Step a1 is carried out.

Next, step (a2) of storing the initial input data input through the data input device 16 in the external main memory 11 is performed. Here, as in the embodiment of the present invention, when the input data is a video, the video data received from the camera may be stored via the DMA module 13. [

Next, a step (a3) of storing the input data stored in the external main memory 11 and the synaptic weight corresponding to the input data in the internal buffer memory as necessary for each cycle constituting the input layer of the artificial neural network is performed can do. At this time, the weight and the input data may be stored in whole or in part according to the size of the artificial neural network.

Next, the neural network computing unit 14 reads the synapse weight and input data stored in the internal buffer memory 12 corresponding to each cycle constituting the artificial neural network, (A4) of performing the neural network operation and storing the result of the operation in the external main memory 11 for use as input data of the next layer.

Here, when the weight and input data necessary for the calculation are stored in the internal memory, the CPU 15 transmits the calculation start instruction to the neural network calculation unit 14, and the neural network calculation unit 14 calculates the weight and input data without passing through the bus It is possible to directly read and operate the internal buffer memory 12. In this case, when the input data and the weight of the neural network can not be stored in the internal memory, a part of the input data and the weight may be repeatedly stored in the internal memory 12 until the entire layer is calculated.

Next, when the result of the operation on the input layer is stored in the external main memory 11, the synaptic weight for the input neurons for artificial neural network operations on the hidden layer and the input data are stored in the external main memory 11), stores it in the internal buffer memory, and then performs the process of step a4 repeatedly on the N hidden layers (a5).

5 is a block diagram schematically illustrating a multi-integrated circuit configuration of an acceleration system for processing an artificial neural network operation in which input data has a plurality of depths according to an embodiment of the present invention.

5, an acceleration system 200 for processing an artificial neural network operation in which input data for input neurons has a plurality of depths according to an embodiment of the present invention includes a single host acceleration device 210, , And a plurality of slave acceleration devices 220.

More specifically, the host accelerator 210 is connected to the flash memory 201, which stores the total synaptic weights necessary for the artificial neural network operation, and the host communication intermediary block 201 is connected to the plurality of depths .

The slave acceleration device 220 may include at least one slave communication intermediary block 202 physically connected to the host communication intermediary block 201 of the host accelerator 210, May be formed in plural numbers so as to correspond one-to-one to depth numbers.

At this time, the host accelerator 210 causes the slave acceleration apparatus 220 related to each of the plurality of depths to distribute the synapse weight and the input data in a parallel manner to process an artificial neural network operation, and the slave acceleration apparatus 220) to calculate the final operation.

In the embodiment of the present invention, the host communication intermediary block 201 and the slave communication intermediary block 202 preferably have the same configuration as the general-purpose communication intermediary block 100 described above with reference to FIG. 4, The detailed description will be omitted.

In addition, in the embodiment of the present invention, the host acceleration device 210 and the slave acceleration device 220 may be configured as an integrated circuit of any one of a SoC scheme and an FPGA scheme, The host communication intermediary block 201 and the slave communication intermediary block 2020 can communicate with each other.

The host accelerator 210 and the slave accelerator 220 are connected to the external main memories 211 and 221, the internal buffer memories 212 and 222, the DMA modules 213 and 223, Computing devices 214 and 224, and CPUs 215 and 225, and detailed description of the same components as those shown in FIG. 3 will be omitted.

Next, referring to Figs. 6 and 7, embodiments of the acceleration system shown in Fig. 5 will be described.

FIG. 6 is a block diagram schematically showing an artificial neural network according to an embodiment of the acceleration system shown in FIG. 5, in which an artificial neural network that distributes and processes dimensions of characteristic maps constituting input data into three depths A, It is a diagram of the model.

The process of implementing the artificial neural network model shown in FIG. 6 with the acceleration system shown in FIG. 5 will now be described.

First, when power is applied to a single host accelerator 210 and a plurality of slave accelerators 220 and all the accelerators are ready for an operation, the external flash memory (not shown) connected to the host accelerator 210 (B1) of storing the entire synaptic weights of the input neurons for the artificial neural network operation stored in the external main memory 211 of the host accelerator 210. [

Next, the host accelerator 210 transmits the synapse weights stored in the external main memory 211 of the host accelerator 210 through the universal communication mediation blocks 201 and 202 to the three depths A, B, And sequentially transmitting and storing the respective synapse weights to the external main memory 221 of the plurality of slave acceleration devices 220 in a parallel manner and storing the synapse weights in step b2.

Next, a synapse weight necessary for each cycle constituting the input layer of the artificial neural network among the synapse weights stored in the respective external main memories 221 of the plurality of slave acceleration apparatuses 220 is transmitted to each of the slave acceleration apparatuses 220 And storing it in the internal buffer memory 222 (step b3).

Next, the single host accelerator 210 performs step (b4) of storing the initial input data input through the data input device 16 in the external main memory 211 of the host accelerator 210 In the embodiment of the present invention, in the case of the image data, the input is received through the sensor in-phase, and the peripheral can receive input from the peripheral device.

Next, the host accelerator 210 corresponds to three depths A, B, and C of the input data stored in the external main memory 211 of the host accelerator 210 through the universal communication mediation blocks 201 and 202 (B5) sequentially transmitting and storing the input data to the external main memory 221 of the plurality of slave acceleration devices 220 in a parallel manner.

The input data stored in the external main memory 221 of each of the slave acceleration devices 220 is input to each slave acceleration device 220 internal buffer memory 222 as necessary for each cycle constituting the artificial neural network, (B6). ≪ / RTI >

The neural network computation unit 224 of each of the slave acceleration devices 220 generates a synapse weight value 222 corresponding to each cycle constituting the artificial neural network and stored in the internal buffer memory 222 of each of the slave acceleration devices 220, (B7) of reading the input data, performing an artificial neural network operation until the operation on the entire layer is completed, and storing the operation result in the external main memory 221 of each of the slave acceleration devices 220 can do.

Next, the host accelerator 210 receives the result of the intermediate operation stored in the external main memory 221 of each of the slave accelerators 220 and sequentially outputs the result of the intermediate operation to the external main memory 211 of the host accelerator 210 (B8) of performing final computation on the entire layer and storing the final computation result in the external main memory 211 of the host accelerator 210. In this case,

Next, in order to use the final computation result stored in the external main memory 211 of the host accelerator 210 as input data for computation of the next layer, each external main memory 221 of the three slave acceleration devices A step (b9) of successively transferring and storing the data in the parallel manner corresponding to the three depths may be performed.

Next, a step (b10) of storing the synaptic weights necessary for each cycle constituting the next layer among the synapse weights stored in the respective external main memories of the plurality of slave acceleration devices, in the internal buffer memory of each slave acceleration device can do.

Finally, the step (b11) of performing the steps b6 to b10 repeatedly may be performed until the operation of all layers for the input data is completed.

FIG. 7 is a diagram schematically showing an artificial neural network according to another embodiment of the acceleration system shown in FIG. 5. FIG.

FIG. 7 is a diagram schematically showing an artificial neural network according to another embodiment of the acceleration system shown in FIG. 5, in which the dimension of the characteristic map constituting the input data is logically excluded from the input layer and the output layer, Which is distributed in parallel to the depth of the network.

As described above, when the artificial neural network is independently separated, it is not necessary to transfer the intermediate calculation results for the remaining output layers excluding the input layers and the output layers to the host acceleration device for each operation of each layer, It is possible to avoid a data transfer time between the accelerating apparatuses, thereby providing an optimal distributed processing method.

The process of implementing the artificial neural network model shown in FIG. 7 with the acceleration system shown in FIG. 5 will now be described.

First, when power is applied to a single host accelerator 210 and two slave accelerators 220, the synapse weights for the input neurons for the artificial neural network operation stored in the external flash memory 1 are all (C1) in the external main memory 211 of the host acceleration apparatus.

Next, the host accelerator 210 transmits the respective synapse weights corresponding to the two depths of the synapse weights stored in the external main memory 211 of the host accelerator 210 through the universal communication mediation block 201, To the external main memory 221 of the plurality of slave acceleration devices 220 in parallel in a parallel manner and store the sequentially transferred data in the parallel main memory 221 in step c2.

Next, a synapse weight necessary for each cycle constituting the input layer of the artificial neural network among the synapse weights stored in the respective external main memories 221 of the plurality of slave acceleration apparatuses 220 is transmitted to each of the slave acceleration apparatuses 220 (C3) in the internal buffer memory 222. [0213] FIG.

Next, the single host accelerator 210 performs a step (c4) of storing the initial input data input through the data input device 16 in the external main memory 211 of the host accelerator 210 can do.

Next, the host accelerator 210 stores the input data stored in the external main memory 211 and the synaptic weights in the internal buffer memory 212 as needed for a cycle constituting the input layer of the neural network, 214 to perform a neural network operation on the input layer, and then store the operation result in the external main memory 211 for use as input data of the next layer (c5).

Next, the host accelerator 210 transmits the input data (operation result) stored in the external main memory 211 of the host accelerator 210 through the universal communication mediation block 201, (C6) sequentially transmitting and storing each input data (operation result) to each external main memory 221 of the plurality of slave acceleration devices 220 in a parallel manner.

Next, input data (calculation result of the input layer) stored in the external main memory 221 of each of the slave acceleration apparatuses 220 and corresponding synaptic weights are stored in the respective memories (C7) in the slave acceleration device 220 internal buffer memory 222 of the slave device.

The neural network computing device 224 of each of the slave acceleration devices 220 generates a synapse weight value 222 corresponding to each cycle constituting the artificial neural network and stored in the internal buffer memory 222 of each of the slave acceleration devices 220. [ And repeatedly reads the input data to repeatedly perform the artificial neural network operation until the operation of the entire layer constituting the artificial neural network is completed and outputs the operation result to the external main memory 221 of each of the slave acceleration devices 220. [ (C8). ≪ / RTI >

Lastly, the host accelerator 210 receives the result of the intermediate operation stored in the external main memory 221 of each of the slave accelerators 220 and sequentially outputs the result of the intermediate operation to the external main memory 211 of the host accelerator 210 (C9) of performing final computation on the entire neural network and storing the final computation result in the external main memory 211 of the host accelerator 210. In this case,

As described above, the present invention provides a method of accelerating an artificial neural network in which input data for input neurons have a plurality of depths, wherein each of the accelerators corresponding to the plurality of depths includes input data for input neurons and a synapse weight Can be dispersed and processed.

8 is a block diagram schematically illustrating a multi-integrated circuit configuration of an acceleration system for processing an artificial neural network operation in which input neurons are formed in a plurality of layers according to an embodiment of the present invention.

As shown in the figure, an acceleration system 300 for processing an artificial neural network operation in which input neurons according to an embodiment of the present invention are composed of a plurality of layers, includes a host acceleration device 310 and a plurality of slave acceleration devices (320).

More specifically, the host accelerator 310 is connected to the flash memory 1 storing the entire synaptic weights necessary for the artificial neural network operation, and includes at least one pair of host communication communication blocks 301 for transmission and reception .

In addition, the slave acceleration device 320 includes at least one pair of slave communication mediating blocks 302 for transmitting and receiving in order to be sequentially pipelined to the host accelerator 310, .

Accordingly, an acceleration system 300 for processing an artificial neural network operation in which input neurons according to an embodiment of the present invention are composed of a plurality of layers is provided to the host acceleration apparatus 310 and the slave acceleration apparatus 320, It is possible to sequentially process the neural network operation by sequentially dispersing the synaptic weights and the input data for the input neurons constituting the neural network operation.

Further, in the embodiment of the present invention, the communication mediating block 301 for the host and the communication mediating block 302 for the slave are preferably the same as the general-purpose communication mediating block 100 described in Fig. 4 , And a detailed explanation thereof will be omitted.

Also, in the embodiment of the present invention, the host accelerator 310 and the slave accelerator 320 may be configured as an integrated circuit of any one of a SoC system and an FPGA system, Can be mutually communicated via the host communication intermediary block 301 and the slave communication intermediary block 302. [

In addition, as shown in the figure, the host accelerator 310 and the slave accelerator 320 are respectively connected to external main memories 311 and 321, internal buffer memories 312 and 322, DMA modules 313 and 323, Computing devices 314 and 324, and CPUs 315 and 325, and detailed description of the same components as those shown in FIG. 3 will be omitted.

9 is a diagram schematically illustrating an artificial neural network model according to an embodiment of the acceleration system shown in FIG.

To illustrate the process of implementing the artificial neural network model shown in FIG. 9 with the acceleration system shown in FIG. 8, a connection structure of a single host acceleration device and a plurality of slave acceleration devices using the artificial neural network system shown in FIG. They can be placed sequentially in a pipeline form as a whole.

The method of accelerating the artificial neural network processing having the hierarchical structure according to the embodiment of the present invention is a method of accelerating the processing of the artificial neural network having the hierarchical structure according to the embodiment of the present invention, when the power is applied to a single host accelerator 310 and a plurality of slave accelerators 320, (D1) storing the entire synaptic weights of the input neurons for the artificial neural network operation stored in the external main memory 311 of the host accelerator 310.

Next, the host accelerator 310 transmits the synapse weights corresponding to the N layers among the synapse weights stored in the external main memory 311 of the host accelerator 310 through the universal communication mediation block 301 and 302, To the external main memory 321 of the plurality of slave acceleration devices 320, and storing the transferred data d2.

Next, the single host accelerator 310 performs step (d3) of storing the input data of the input layer input through the data input device 16 in the external main memory of the host accelerator 310 .

Next, the input data of the input layer stored in the external main memory 311 of the host accelerator 310 and the synaptic weight may be stored in the internal buffer memory 312 (step d4).

Next, the synapse weight and input data stored in the internal buffer memory 312 are read by the neural network calculator 314 of the host accelerator 310 to perform a neural network operation on the input layer, (D5) of storing the calculation result in the storage unit (311).

Next, the operation result is stored as input data in the external main memory 321 of the slave acceleration device 320 corresponding to the next layer, and then the result is stored in the internal buffer memory 322 of the slave acceleration device 320 (D6) of storing the corresponding input data and the synaptic weights.

Next, the neural network calculator 324 of the slave accelerator 320 reads the synapse weight and input data stored in the internal buffer memory 322, performs a neural network operation on the corresponding layer, And the step d7 of storing it in the memory 321 can be performed.

Next, the steps d6 to d7 are repeatedly performed for the N hidden layers, and the final operation result is stored in the external main memory 311 of the host accelerator 310 or transmitted to the peripheral device (step d8) can do.

Meanwhile, according to the embodiment of the present invention, when the neural network having N layers is distributed and processed, N layers can be divided by the number of slaves or independently allocated to the slaves in proportion to the amount of computation of each layer.

As described above, when the acceleration apparatuses are allocated to the respective layers distributed by hierarchies, pipelining can be performed, and the efficiency of computation for continuous stream input is increased.

Accordingly, the present invention provides a method for accelerating an artificial neural network in which input neurons are configured in a plurality of hierarchical structures, the method comprising: distributing input data and synaptic weights for input neurons to respective accelerators corresponding to the plurality of layers, Can be provided.

10 is a block diagram schematically showing a multi-integrated circuit configuration of an acceleration system for processing an artificial neural network operation composed of a plurality (M) of depths and a plurality (N) of layers.

As shown in the figure, an acceleration system for processing an artificial neural network operation in which input data for input neurons are composed of a plurality (M) of depths and a plurality (N) of layers according to an embodiment of the present invention, And may include an acceleration device 410 and a plurality of slave acceleration devices 420.

More specifically, the host accelerator 410 is connected to a flash memory (not shown) storing the total synaptic weights necessary for the artificial neural network operation, and a host communication intermediary block (not shown) Or more.

The slave acceleration device 420 may include one or more slave communication intermediary blocks 402 connected to the host communication intermediary block of the host accelerator 410 or connected to another acceleration device, (M × N) required for a plurality of (M) number of layers and a plurality (N) of layers.

In the embodiment of the present invention, the communication mediating block for host 402 and the communication mediating block 402 for slave are preferably the same as the general purpose communication mediating block 100 described in Fig. 4, The detailed description will be omitted.

In addition, in the embodiment of the present invention, the host accelerator 410 and the slave accelerator 420 may be configured as an integrated circuit of any one of a SoC system and an FPGA system, The communication intermediary block for host 402 and the communication intermediary block 402 for slave can be mutually communicable.

The host accelerator 410 and the slave accelerator 420 may be connected to each other through an external main memory 421, an internal buffer memory 422, a DMA module 423, A CPU 424, and a CPU 425. A detailed description of the same components as those shown in FIG. 3 will be omitted.

As shown in the figure, when the depth and the number of layers of data for the input neurons of the artificial neural network are large, it may be difficult to satisfy the target performance required by a single or a small number of acceleration apparatuses.

In such a case, the above-described depth dispersion method and the hierarchical dispersion method of FIGS. 5 to 9 may be simultaneously applied to the acceleration apparatus according to the present invention to maximally disperse and operate independently, thereby improving the computation performance.

At this time, as shown in the figure, the host accelerator 410 transmits weight and input data for an artificial neural network operation to an upper input terminal of a slave accelerator 420 constituting each layer, Upon completion, the final operation result is received from the last layer of the slave acceleration device.

Referring to the drawing, L of the slave acceleration device 420 denotes an assigned layer, and the number of the layers can be represented by 1 to N, D denotes an allocated depth (or Depth) M can be expressed.

11 is a diagram schematically illustrating an artificial neural network model according to an embodiment of the acceleration system shown in FIG.

10, the slave acceleration apparatus includes a slave (L1, D1) to a slave (L1, DM) direction for the calculation of A-1, A-2, A-3 of the input layer And other layers may be allocated in the same manner.

On the other hand, each of the slave acceleration devices can be variably allocated according to the computation ability and the number of the acceleration devices.

The process of implementing the artificial neural network model shown in FIG. 11 with the acceleration system shown in FIG. 10 will be described below.

First, when power is applied to a single host accelerator 410 and a plurality of slave accelerators 420, the entire synapse weight for input neurons for an artificial neural network operation stored in an external flash memory is transmitted to the host accelerator 410, (E1) in the external main memory.

Next, the host accelerator 410 transmits all the synapse weights corresponding to the N layers among the synapse weights stored in the external main memory of the host accelerator through the universal communication mediation block to M The slave acceleration devices 420L1D1, 420L2D1, and 420LND1 corresponding to the first depth of each layer in the depth are sequentially transmitted to the external main memory of each of the slave acceleration devices 420L1D1, 420L2D1, and 420LND1, (E2) in which the respective synaptic weights are stored in the entire region.

Next, the single host accelerator 410 may perform step (e3) of storing the input data of the input layer, which is input through the data input device, in the external main memory of the host accelerator 410.

The input data of the input layer stored in the external main memory of the host accelerator 410 is input to the external main memory 421 of the slave acceleration device 420L1D1 corresponding to the first depth of the input layer, The input data may be stored and the input data corresponding to the depths of the M slave acceleration devices corresponding to the depths constituting the input layer may be dispersed and sequentially stored (step e4).

Next, step (e5) of storing the input data and the synapse weight of the layer in the M slave accelerator internal buffer memory 422 constituting the input layer may be performed.

Next, a neural network operation may be performed by the neural network operation device of the M slave acceleration devices constituting the input layer, and the operation result may be stored in its external main memory (e6).

Next, the calculation result stored in the M slave acceleration devices is transmitted to the slave acceleration device 420L1D1 corresponding to the first depth of the input layer to perform the final calculation of the corresponding layer, and the final calculation result is transmitted to the first depth of the next layer (E7) to the slave acceleration device 420L2D1 as the input data of the corresponding layer.

The entire input data of the corresponding layer is firstly stored in the external main memory of the slave acceleration device corresponding to the first depth of the next layer and then the input data corresponding to the M slave accelerator external main memories constituting the corresponding layer are dispersed sequentially (Step e8) of storing the data in the memory.

Steps e5 to e8 may be repeated until the operation of the Nth layer is completed, and the final operation result may be transmitted to the host accelerator (e9).

As described above, the present invention provides a method for accelerating an artificial neural network in which input data for input neurons are composed of a plurality of depths and a plurality of hierarchical structures, A method of distributing input data and synapse weights to input neurons may be provided.

12 is a diagram schematically illustrating a combined artificial neural network model including a plurality of artificial neural networks according to an embodiment of the present invention.

As shown in the figure, in the embodiment of the present invention, three examples of a combined artificial neural network model are shown.

First, a first embodiment of an artificial neural network model according to an embodiment of the present invention is a first artificial neural network 510 for object detection.

Here, the first complex artificial neural network 510 may include a neural network 511 that conceptually extracts features of an object from an input image and a neural network 512 that detects the position of an object in the input image. have.

Next, a second embodiment of an artificial neural network model according to an embodiment of the present invention includes a second composite artificial neural network (hereinafter referred to as " artificial neural network ") for image caption in which a situation of a person or an object in a video is expressed as a word 520).

Here, the second compound artificial neural network 520 may include a neural network 521 for classifying an object in an image and a neural network 522 for generating a caption in an object classified to describe the image have.

At this time, since the second composite artificial neural network 520 for image caption processes image data, the size of the artificial neural network is considerably larger than that of a general neural network.

Therefore, in order to improve the computation performance, the second composite artificial neural network 520 is divided into two artificial neural networks 521 and 522, and then the depth, the layer, and the depth of each artificial neural network And can be assigned to the acceleration device.

Particularly, in the case of the neural network 522 for generating captions in the objects classified to describe the image, since the memory cells can be sequentially connected in the time series order, the artificial neural network acceleration system It is preferable to perform the dispersion processing.

Meanwhile, the compound artificial neural network 530 according to the third embodiment of the compound artificial neural network model of the present invention may be applied to a convolutional neural network for input data 531 that can be arranged in time series, A neural network 533 for extracting features through a 3-dimensional convolutional neural network 532 extended on a time axis and recognizing gestures and actions by determining correlation and temporal continuity between the features, . ≪ / RTI >

At this time, the neural network having a large amount of computation for processing continuous image data as in the case of the three-dimensional articulated neural network can be distributedly processed by dividing the internal neural network with respect to the depth, layer, and time series described above.

As described above, according to an exemplary embodiment of the present invention, a complex artificial neural network acceleration system configured by a plurality of artificial neural networks by one or more artificial neural network acceleration systems includes input data for input neurons and synaptic weights to the plurality of artificial neural networks It is possible to divide and process each artificial neural network unit.

As described above, according to the present invention, data on input neurons necessary for an operation of an artificial neural network and synapse weights are supplied to a plurality of artificial neural network operations for distributed processing having a universal communication mediating block capable of mediating communication irrespective of kinds of integrated circuits It is able to satisfy the performance required for artificial neural network computation without using high power and high cost hardware such as GPGPU by dispersing and processing in the acceleration apparatus, and also, it is possible to design artificial neural network flexibly according to target performance, An acceleration system, and an acceleration method thereof.

In addition, the present invention reduces the manufacturing cost compared with the use of an expensive GPGPU implemented by a single hardware according to the prior art by connecting a plurality of accelerators composed of low-power / low-cost integrated circuits to perform neural network calculation And can be applied to fields requiring low power.

Further, the present invention can flexibly apply additional functions to an acceleration system implemented by the same type or a heterogeneous type of integrated circuit by implementing the universal communication mediating block capable of communication mediation regardless of the type of the integrated circuit in the acceleration device It is possible to actively cope with various demands of users.

Further, the present invention has the effect of expanding or reducing the acceleration device flexibly according to the target performance by performing the distributed parallel processing of the artificial neural network using the multiple acceleration device.

Further, according to the present invention, input data and synaptic weights for neurons constituting an artificial neural network are divided into a depth, a hierarchical structure, a neural network, or a combination unit constituting input data, In the case of hardware implementation, the memory and the peripheral device can be optimized, and as a result, the product development cost can be lowered.

In addition, the present invention realizes an accelerating system using various kinds of integrated circuits, and thus can be actively used in various artificial neural network structures to be applied in the future.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is clearly understood that the same is by way of illustration and example only and is not to be taken by way of limitation, It is within the scope of the present invention that component changes to such an extent that they can be coped evenly within a range that does not deviate from the scope of the present invention.

1: Flash memory
10: Accelerator
11, 211, 221, 311, 321, 421: external main memory
12, 212, 222, 312, 322, 422: internal buffer memory
13, 213, 223, 313, 323, 423: DMA module
14, 214, 224, 314, 324, 424:
15, 215, 225, 315, 325, 425: CPU
16: Data input device
100: general purpose communication mediation block
101: Transmission module (interface)
102: receiving module (interface)
110: remapping block
120: Bus control signal matching block
130: Monitor block
200, 300, 400: Acceleration system
201, 301, 401: mediation block for host
202, 302, 402: a communication mediation block for a slave
210, 310, 410: Host Accelerator
220, 320, 420: Slave acceleration device
510, 520, 530: Composite artificial neural network
511,512,521,522,531,532: Artificial neural network

Claims (24)

delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete CLAIMS 1. A method for accelerating an artificial neural network processing wherein input data for input neurons, including a single host acceleration device and a plurality of slave acceleration devices, have M depths,
Storing the entire synaptic weights for input neurons for neural network operations stored in an external flash memory in an external main memory of the host accelerator when power is applied to the single host accelerator and the plurality of slave accelerators; );
Wherein the host accelerator sends each synapse weight corresponding to the M depths among the synapse weights stored in the external main memory of the host accelerator apparatus to the external main memory of the plurality of slave accelerator apparatuses in a parallel manner (B2);
(B3) storing a synaptic weight necessary for each cycle constituting the input layer of the artificial neural network, among the synaptic weights stored in the respective external main memories of the plurality of slave acceleration devices, in the internal buffer memory of each slave acceleration device;
(B4) storing the first input data input by the single host accelerator device through the data input device in the external main memory of the host accelerator device;
The host accelerator transmits each input data corresponding to the M depths of the input data stored in the external main memory of the host accelerator device to the external main memory of the plurality of slave accelerator devices in a parallel manner (B5);
(B6) storing input data stored in an external main memory of each of the slave acceleration devices in the slave acceleration device internal buffer memory as necessary for each cycle constituting the artificial neural network;
The neural network computing device of each of the slave acceleration devices reads the synaptic weight and input data stored in the internal buffer memory of each of the slave acceleration devices corresponding to each cycle constituting the artificial neural network, (B7) performing an artificial neural network operation up to the slave acceleration device and storing the operation result in the external main memory of each of the slave acceleration devices;
The host acceleration device receives the intermediate calculation result stored in the external main memory of each of the slave acceleration devices, sequentially stores the intermediate result in the external main memory of the host acceleration device, (B8) in an external main memory of the host accelerator apparatus;
In order to use the final computation result stored in the external main memory of the host accelerator as input data for operation of the next layer, sequentially in parallel manner corresponding to M depths in each external main memory of the plurality of slave accelerators (B9);
(B10) storing (b10) a synapse weight necessary for each cycle constituting a next layer among synapse weights stored in each external main memory of the plurality of slave acceleration devices, in an internal buffer memory of each slave acceleration device; And
(B11) repeating steps (b6) to (b10) until all layers of the input data are completely computed. The artificial neural network acceleration method
CLAIMS 1. A method for accelerating an artificial neural network processing wherein input data for input neurons, including a single host acceleration device and a plurality of slave acceleration devices, have M depths,
Storing the entire synaptic weights of input neurons for neural network operations stored in an external flash memory in an external main memory of the host accelerator when power is applied to the single host accelerator and the plurality of slave accelerator apparatuses, );
Wherein the host accelerator sends each synapse weight corresponding to the M depths among the synapse weights stored in the external main memory of the host accelerator apparatus to the external main memory of the plurality of slave accelerator apparatuses in a parallel manner And c2.
(C3) storing a synaptic weight necessary for each cycle constituting the input layer of the artificial neural network, among the synaptic weights stored in the respective external main memories of the plurality of slave ac- celerators, in an internal buffer memory of each of the slave ac- celerators;
(C4) storing the first input data input from the single host accelerator through the data input device in the external main memory of the host accelerator;
The host accelerator stores the input data stored in the external main memory and the synaptic weights in the internal buffer memory as needed for a cycle constituting the input layer of the neural network, performs a neural network operation on the input layer using the neural network processor, (C5) in an external main memory for use as input data of a next layer;
The host accelerator transmits each input data corresponding to the M depths of the input data stored in the external main memory of the host accelerator device to the external main memory of the plurality of slave accelerator devices in a parallel manner (C6) transmitting and storing the data sequentially;
(C7) storing the input data stored in the external main memory of each of the slave acceleration devices and the synaptic weights corresponding thereto in the slave acceleration device internal buffer memory as necessary for each cycle constituting the artificial neural network;
Wherein the neural network computing device of each of the slave acceleration devices repeatedly reads the synapse weight and input data stored in the internal buffer memory of each of the slave acceleration devices corresponding to each cycle constituting the artificial neural network, (C8) repeatedly performing an artificial neural network operation until the operation for the slave acceleration device is completed and storing the operation result in the external main memory of each of the slave acceleration devices; And
The host accelerator receives the intermediate operation result stored in the external main memory of each of the slave accelerators, sequentially stores the intermediate operation result in the external main memory of the host accelerator, and then performs final operation on the entire neural network, (C9) storing in the external main memory of the host accelerator apparatus
A connection structure of a single host acceleration device and a plurality of slave acceleration devices are sequentially arranged in a pipeline form so that input data for input neurons has a hierarchical structure including an input layer and N hidden layers, In a method of accelerating,
Storing the entire synaptic weights for input neurons for neural network operation stored in an external flash memory in an external main memory of the host accelerator when power is applied to the single host accelerator and the plurality of slave accelerators; );
Wherein the host accelerator sequentially transmits respective synapse weights corresponding to the N layers among the synapse weights stored in the external main memory of the host accelerator through the universal communication mediation block to each of the external main memories of the plurality of slave acceleration devices Transmitting and storing (d2);
(D3) storing the input data of the input layer to which the single host accelerator device is input through the data input device in the external main memory of the host accelerator device;
Storing input data and a synapse weight of an input layer stored in an external main memory of the host accelerator device in an internal buffer memory (d4);
A step (d5) of reading a synapse weight value and input data stored in the internal buffer memory by the neural network computation device of the host acceleration device, performing a neural network operation on the input layer, and storing the computation result in the external main memory;
(D6) storing the operation result in the external main memory of the slave acceleration device corresponding to the next layer as input data, and then storing the input data and the synapse weight corresponding to the layer in the internal buffer memory of the slave acceleration device;
A step (d7) of reading the synaptic weights and input data stored in the internal buffer memory by the neural network calculator of the slave acceleration apparatus, performing a neural network operation on the layer, and storing the operation result in the external main memory;
(D8) repeatedly performing the steps d6 to d7 for the N hidden layers and storing the final operation result in an external main memory of the host accelerator or transmitting the result to a peripheral device Way
A method for accelerating neural network processing wherein input data for input neurons, comprising a single host acceleration device and a plurality of slave acceleration devices, are comprised of M depths and N layers,
Storing the entire synaptic weights for input neurons for neural network operation stored in an external flash memory in an external main memory of the host accelerator when power is applied to the single host accelerator and the plurality of slave accelerators );
The host accelerator transmits the entire synapse weights corresponding to the N layers among the synapse weights stored in the external main memory of the host accelerator through the universal communication mediation block to the first of each of the M depths constituting each layer (E2) in which the respective synapse weights are stored in all the slave acceleration devices corresponding to different depths connected to the received N slave acceleration devices, respectively, when sequentially transmitted to the external main memory of each of the N slave acceleration devices corresponding to the depth, ;
(E3) storing the input data of the input layer input from the single host accelerator through the data input device in the external main memory of the host accelerator;
The input data of the input layer stored in the external main memory of the host accelerator is firstly stored in the external main memory of the slave acceleration device corresponding to the first depth of the input layer, (E4) in which input data corresponding to M slave accelerating apparatus external main memories corresponding to the plurality of slave accelerating apparatuses are dispersed and sequentially stored;
(E5) storing input data and synapse weights of the layer in M slave accelerator internal buffer memories constituting the input layer;
(E6) performing a neural network operation by the neural network computation apparatus of the M slave acceleration apparatuses constituting the input layer and storing the computation result in its own external main memory;
The calculation result stored in the M slave acceleration devices is transmitted to the slave acceleration device corresponding to the first depth of the input layer to perform the final calculation of the corresponding layer and the final calculation result is transmitted to the slave acceleration device corresponding to the first depth of the next layer (E7) transmitting the input data to the layer;
The entire input data of the corresponding layer is firstly stored in the external main memory of the slave acceleration device corresponding to the first depth of the next layer and then the input data corresponding to the M slave accelerator external main memories constituting the corresponding layer are dispersed sequentially (E8); And
(E9) repeating the same processes as in steps e5 to e8 until the operation of the Nth layer is completed, and transmitting the final computation result to the host accelerator apparatus
delete delete delete delete

Images