KR20210114912A - Neural Network Apparatus for Resource Efficient Inference - Google Patents

Abstract

The embodiment of the present invention relates to an artificial neural network device for resource efficient reasoning, wherein a network structure can be adaptively changed according to hardware performance such as memory performance or computational performance, and a separate additional learning procedure is not required. The artificial neural network device includes a plurality of nodes consisting of an NХM matrix of N (N is a natural number greater than or equal to 2) layer group rows and M (M is a natural number) channel group columns, and the plurality of nodes have a network structure, in which an output of the node corresponding to the same level or low level channel in each layer group is connected to a node corresponding to the high level channel in the next layer group, and an output of a node corresponding to a high level channel in each layer group is not connected to a node corresponding to a low-level channel in the next layer group.

Description

Artificial Neural Network Apparatus for Resource Efficient Inference

This embodiment relates to an artificial neural network device. More particularly, it relates to an artificial neural network apparatus for more efficiently slicing a network structure according to available resources.

The content described in this section merely provides background information for the present embodiment and does not constitute the prior art.

A neural network, a core computational model of deep learning technology, is a type of artificial neural network arranged in multiple layers so that each neuron has characteristics similar to the response characteristics of overlapping regions in the human visual nervous system.

Such a neural network has shown remarkable performance in various fields. In particular, some deep neural networks (DNNs) such as ResNet and DenseNet show very high learning performance. On the other hand, deep neural networks generally have sophisticated architectural designs that require too much capacity to be applied in real-world environments. That is, there is a problem in that time or memory budget constraints follow when a sophisticated but resource-intensive model is directly applied to a real environment, particularly, a mobile device or an embedded system.

To solve this problem, a technique that can improve memory or time efficiency by removing the redundancy of the existing network model has been proposed. However, these conventional techniques have a limitation in that the model needs to be re-educated whenever a specification or constraint is changed.

The present embodiment is to provide an artificial neural network device for resource-efficient inference that does not require a separate additional learning procedure while being able to adaptively change the network structure according to hardware performance such as memory performance or computational performance. There is a purpose.

The present embodiment includes a plurality of nodes composed of an NХM matrix of N (N is a natural number greater than or equal to 2) layer group rows and M (M is a natural number) channel group columns, and the plurality of nodes are in each layer group. The output of the node corresponding to the same level or low level channel is connected to the node corresponding to the high level channel in the next layer group, and the output of the node corresponding to the high level channel in each layer group Provided is an artificial neural network apparatus having a network structure that is not connected to a node corresponding to a low-level channel in the next layer group, and is implemented to be slicable for each channel group or for each layer group.

In addition, according to another aspect of the present embodiment, an operation method of an artificial neural network device including a plurality of nodes including N (N is a natural number greater than or equal to 2) layer group rows and an NХM matrix of M (M is a natural number) channel group columns , the output of a node corresponding to a low level channel in each layer group among the plurality of nodes transmits data to a node corresponding to the same level and high level channel in the next layer group, A node corresponding to a high-level channel in each layer group does not transmit data to a node corresponding to a low-level channel in the next layer group, but is implemented so that slicing is possible for each channel group or for each layer group. A method of operating a neural network device is provided.

In addition, according to another aspect of the present embodiment, there is provided a computer program stored in a computer-readable recording medium to execute each process included in the operating method of the artificial neural network device described above.

As described above, according to the present embodiment, an artificial neural network for resource-efficient inference that does not require a separate additional learning procedure while being able to adaptively change the network structure according to hardware performance such as memory performance or computational performance. There is an effect that can provide the device.

1 is an exemplary diagram for explaining the effect of the network slicing method according to the present embodiment.
2A to 2B are diagrams for explaining the network structure of the artificial neural network apparatus according to the present embodiment.
3A to 3C are exemplary diagrams illustrating a network slicing form of the artificial neural network apparatus according to the present embodiment.
4 is a conceptual diagram for explaining a ResNet device based on the network structure according to the present embodiment.
5 is an exemplary diagram for explaining classification accuracy according to the artificial neural network apparatus according to the present embodiment.
6 to 7 are exemplary views for explaining a method of applying a loss weight of the artificial neural network apparatus according to the present embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

A neural network, a core computational model of deep learning technology, is a type of artificial neural network arranged in multiple layers so that each neuron has characteristics similar to the response characteristics of overlapping regions in the human visual nervous system.

The present invention discloses a network structure of such a neural network-based device. In more detail, we propose a double-overlapping network device for resource-efficient inference that does not require a separate additional learning procedure while adaptively changing the network structure according to hardware performance such as memory performance or computational performance.

Meanwhile, in the following description, the term "neural network" may refer to software that is helpful in an adaptive environment and implements statistical learning algorithms useful for deep learning. A neural network may include a plurality of Artificial Nodes, known as “Neurons”, “processing elements”, “units”, or other similar terms. A plurality of artificial nodes may be connected together to form a network that mimics a biological neural network.

1 is an exemplary diagram for explaining the effect of the network slicing method according to the present embodiment. Meanwhile, FIG. 1 illustrates the difference between the effect of the conventional network slicing method and the network slicing method according to the present embodiment.

In general, deep neural networks (DNNs) have sophisticated architectural designs that require too much capacity to be applied in real-world environments. Due to this, a technique that can improve memory or time efficiency by removing the redundancy of the existing network model has been proposed. Among them, the network slicing method is a technique of building a small network from a large network through reconfiguration of the original network.

However, in the case of the existing network slicing method, there is a limit in satisfying a plurality of constraints according to the network slicing considering only a single control parameter (ex: width or depth). For example, referring to (a) of FIG. 1 , in the case of a network slicing model generated based on a single control parameter, sufficient resources are provided in various environments (ex: environment 1, environment 2) having different operation speeds and memory sizes. There is a limit to not being able to use it.

On the other hand, in the case of the network slicing method according to the present embodiment, as network slicing with a higher degree of freedom (ex: considering both width and depth) than the existing network slicing method is possible, a plurality of constraints are simultaneously satisfied. Various network slicing models can be created. For example, referring to FIG. 1 (b), in the case of the network slicing model according to the present embodiment, only available resources are utilized while maintaining original performance as high as possible even in various environments having different operation speeds and memory sizes. can have an effect.

On the other hand, in general, it is widely known that one of the main factors determining the operation speed is the number of layers, and the maximum required memory has a dominant influence on the number of channels. There may be several directions for dynamically reducing computational speed and memory usage, but the basic is to prepare multiple models with different resource requirements. However, this requires several training steps and consumes a lot of space.

Models with slight differences in the number of layers or channels can be expected to share mainly similarities. Due to this point, the present embodiment proposes a neural network structure called DNNet (Doubly Nested Network). DNNet has a structure in which parameters are shared as much as possible by overlapping all models into one, thereby allowing slices for each channel or layer while maintaining high original performance.

2A to 2B are diagrams for explaining the network structure of the artificial neural network apparatus according to the present embodiment.

In the case of a conventional convolutional neural network (CNN), a channel of each feature map has a network structure in which it is fully connected to a channel of a feature map of an adjacent layer. According to such a network structure, the output activation value of a specific layer may be calculated by aggregating all output activations of the previous layer. In other words, simply omitting some channels from the input function of a particular layer can lead to unexpected output activation, which in turn propagates to the next layer, causing serious performance degradation.

In order to solve this problem, in the present embodiment, the synaptic connection between two consecutive layers is restricted so that the slicing part model does not depend on the output of other parts not included in the slicing model.

2A is a diagram for explaining a basic structure of a network of an artificial neural network apparatus according to the present embodiment.

As shown in FIG. 2A , the artificial neural network apparatus 200 according to the present embodiment is an N×M matrix of N (N = natural number) layer group 202 rows and M (M = natural number) channel group 204 columns. It includes a plurality of nodes configured. Here, each node means at least one feature map in a layer group, and a channel means the number of these feature maps.

Meanwhile, in the following description, the artificial neural network apparatus 200 according to the present embodiment is exemplified as including a plurality of nodes composed of a 3×3 matrix of three layer group rows and three channel group columns, but is not necessarily limited thereto. no.

Referring to FIG. 2A , in the plurality of nodes in the artificial neural network apparatus 200 according to the present embodiment, the output of a node corresponding to a low level channel in each layer group is a high level in the next layer group. It is connected to a node corresponding to a channel, and has a network structure in which an output of a node corresponding to a high-level channel in each layer group is not connected to a node corresponding to a low-level channel in the next layer group.

In addition, a plurality of nodes in the artificial neural network apparatus 200 have a network structure in which outputs of nodes corresponding to the same channel group are connected to each other. On the other hand, here, as an example, the connection means that a value calculated by an activation function (ex: ReLu) in each node is reflected and transmitted to the next node by weight.

That is, in the case of the artificial neural network apparatus 200 according to the present embodiment, the channels are topologically aligned and slicable in the depth direction, which may be configured such that some channels are relatively important than other channels, There is an effect that the network can be newly configured except for the nodes of the level channel.

More specifically, the artificial neural network apparatus 200 according to the present embodiment divides the entire network into a predefined number of groups along a channel, designs a channel-causal convolution filter, and forms one layer. A network structure may be implemented so that the i-th channel group of n can be calculated only as activation values of the first to i-th channel groups in the previous layer. According to the network structure of the artificial neural network apparatus 200 according to this embodiment, the partial model having only the i-th or lower sub-channel group is (i+1)-th or higher channel group in both the learning and inference process and allow them to work independently.

Meanwhile, for this purpose, the artificial neural network apparatus 200 may be trained in advance so that nodes corresponding to the first channel group among the plurality of channel groups can each independently perform a learning operation.

The artificial neural network apparatus 200 according to the present embodiment may be implemented to enable slicing not only for channels but also for each layer. To this end, the artificial neural network apparatus 200 may be implemented so that the learned function map of the previous layer can be reused in a subsequent layer by adding the classification module 206 to a continuous layer of the network.

That is, referring to FIG. 2B , the artificial neural network apparatus 200 according to the present embodiment may include a classification module 206 corresponding to each node. In more detail, the classification module 206 is connected to a corresponding node corresponding to the corresponding classification module 206 and a node corresponding to a low-level channel in a layer group of the corresponding node to classify the result value received from each node. can be implemented as

The classification module 206 is adaptively activated or deactivated according to the structure of the network, so that the learned function map of the previous layer can be reused in a subsequent layer when necessary.

In another embodiment, the classification module 206 may be implemented such that a corresponding classification module is provided for at least the last layer group when the network structure is reconfigured.

Meanwhile, according to the network structure as described above, in the artificial neural network apparatus 200 according to the present embodiment, the network structure may be adaptively reconfigured according to at least one or more environmental factors to which the corresponding model is applied.

In the present embodiment, in the artificial neural network apparatus 200, the number of layer groups and channel groups constituting the network structure may be adaptively adjusted, respectively, according to the above environmental factors. Here, at least one or more environmental factors to which the artificial neural network apparatus 200 is applied may be hardware performance factors of at least one of a memory size and an operation speed required in a learning process.

For example, the artificial neural network apparatus 200 may be adjusted so that the number of channel groups decreases when the environmental factor is the size of the memory, and the number of layer groups decreases when the environmental factor is the operation speed. In this case, the adjustment values for the number of layer groups and channel groups according to the size of the memory and the operation speed may be complementary to each other.

More specifically, in the present embodiment, a slicing model for a network structure according to a combination of at least one or more environmental factors is predefined, and the artificial neural network apparatus 200 adapts the network structure based on the slicing model corresponding to the current environmental factor. can be reconstructed.

To this end, in the present embodiment, a control module (not shown) for reconfiguring the network structure by using the slicing model may be additionally provided inside or outside the artificial neural network apparatus 200 . That is, the control module collects information on environmental factors to which the artificial neural network apparatus 200 is currently applied, selects a combination (memory-operation) corresponding thereto, and based on the slicing model matched to the selected combination, the artificial neural network The network structure of the device 200 may be reconfigured.

3A to 3C are exemplary diagrams illustrating a network slicing form of the artificial neural network apparatus 200 according to the present embodiment. Meanwhile, FIGS. 3A to 3C illustrate a slicing model having a reconstructed network structure of (2,2), (3,2), and (2,3), respectively.

Referring to FIGS. 3A to 3C , it can be seen that the number of layer groups and channel groups constituting the network structure is adaptively adjusted according to the requirements of the size of the memory and the operation speed. In addition, it can be confirmed that the classification module 206 corresponding to each node is adaptively activated or deactivated according to the reconfigured network structure. Meanwhile, in the present embodiment, after reconfiguration of the network structure, at least one classification module among the classification modules corresponding to the last layer group is activated. In this case, it is preferable that the classification module corresponding to the node corresponding to the highest level channel in the last layer group is activated, but the present invention is not limited thereto.

4 is a conceptual diagram for explaining a ResNet device based on the network structure according to the present embodiment.

Referring to FIG. 4 , in the case of the ResNet device based on the network structure according to the present embodiment, it can be seen that the connection relationship between each feature map is arbitrarily adjusted according to environmental factors to which the corresponding model is applied.

Meanwhile, in the case of the network structure according to the present embodiment, a classification module 206 corresponding to each of a plurality of nodes, that is, each feature map is provided. Accordingly, in the present embodiment, there is a need to appropriately reduce the number of parameters or the amount of computation for the classification module 206 .

To this end, in the present embodiment, a preprocessing process called Global Average Pooling (GAP) may be performed on the feature map input to the classification module 206 . For example, in the case of GAP, a vector having as many values as the number of channels is generated by changing a value in a two-dimensional space (xy) to one value for each channel in a three-dimensional (xyz) feature map, Through this, the size of the feature map input to the classification module 206 may be reduced.

5 is an exemplary diagram for explaining classification accuracy according to the artificial neural network apparatus according to the present embodiment. Meanwhile, FIG. 5 is a comparison graph illustrating accuracy measurement results according to the number of operations between a slicing model generated based on the network structure according to the present embodiment and another conventional slicing model.

Referring to FIG. 5 , in the case of a slicing model generated based on the network structure according to the present embodiment, there is little difference from a fully trained artificial neural network model, but higher accuracy at the same number of operations compared to other conventional slicing models It can be confirmed that it has

6 to 7 are exemplary views for explaining a method of applying a loss weight of the artificial neural network apparatus according to the present embodiment. Meanwhile, FIG. 6 shows the performance of all possible neural network models when all loss weights are uniformly assigned. 7 shows the relative performance of all possible neural network models compared to the uniform loss weighting performance of FIG. 6 when the loss weights are adjusted for each model according to user requirements.

6 is a diagram illustrating classification accuracy for all partial models (= slicing models) corresponding to an artificial neural network device having a network structure including a maximum of 16 layer groups and 22 channel groups.

Referring to FIG. 6 , it can be seen that the slicing model generated based on the network structure according to the present embodiment is implemented to have a high level of accuracy without seriously degrading the performance of the largest model.

Meanwhile, in the case of the artificial neural network apparatus according to the present embodiment, a loss function related to accuracy is predefined for each classification module 206 during training. At this time, in the case of the artificial neural network apparatus according to the present embodiment, as the classification module 206 corresponding to each node is adaptively activated or deactivated according to the slicing type of the network structure, the accuracy of all classification modules is higher than the preset threshold. A loss function must be defined to satisfy it.

However, according to an embodiment, the artificial neural network apparatus according to the present embodiment may be implemented to have higher accuracy with respect to a specific classification module than other classification modules. That is, in the present embodiment, the corresponding classification module may be trained in advance to have a relatively high weight for the loss function compared to other classification modules in response to environmental factors to which the artificial neural network apparatus is applied.

For example, when a slicing model with a small network structure is preferred in relation to the artificial neural network apparatus according to the present embodiment (Low Cost Preferred), as shown in FIG. A weight for the loss function may be set to be high.

Conversely, when a slicing model with a large network structure is preferred (High Performance-Preferred), as shown in FIG. have.

In another embodiment, regardless of the size of the network structure, as shown in (c) of FIG. 7 , a weight for the loss function may be set high for a specific classification module existing at a specific location.

As described above, in the artificial neural network apparatus having the network structure according to the present embodiment, the output of a node corresponding to a low level channel in each layer group among a plurality of nodes is the same in the next layer group as a high level Data is transmitted to the node corresponding to the channel, but the node corresponding to the high-level channel in each layer group operates so that data is not transmitted to the node corresponding to the low-level channel in the next layer group.

The operation method of the artificial neural network apparatus according to the present embodiment is implemented as a program and is a readable recording medium (CD-ROM, RAM, ROM, memory card, hard disk, magneto-optical disk, storage device, etc.) can be recorded in

The above description is merely illustrative of the technical idea of this embodiment, and various modifications and variations will be possible by those skilled in the art to which this embodiment belongs without departing from the essential characteristics of the present embodiment. Accordingly, the present embodiments are intended to explain rather than limit the technical spirit of the present embodiment, and the scope of the technical spirit of the present embodiment is not limited by these embodiments. The protection scope of this embodiment should be interpreted by the following claims, and all technical ideas within the equivalent range should be interpreted as being included in the scope of the present embodiment.

200: artificial neural network device 202: layer group
204: channel group 206: classification module

Claims (10)

It includes a plurality of nodes composed of N (N is a natural number greater than or equal to 2) layer group rows and an NХM matrix of M (M is a natural number) channel group columns,
In the plurality of nodes, an output of a node corresponding to the same level or low level channel in each layer group is connected to a node corresponding to a high level channel in the next layer group,
has a network structure in which an output of a node corresponding to a high-level channel in each layer group is not connected to a node corresponding to a low-level channel in the next layer group;
The artificial neural network device, characterized in that it is implemented to be slicable for each channel group or for each layer group. The method of claim 1,
The plurality of nodes,
An artificial neural network device, characterized in that outputs of nodes corresponding to the same channel group are connected to each other. The method of claim 1,
The plurality of nodes,
The artificial neural network apparatus, characterized in that the network structure is adaptively reconfigured according to at least one or more environmental factors to which the artificial neural network apparatus is applied. 4. The method of claim 3,
The environmental factor is at least one of a hardware performance factor among the size and operation speed of memory required in the learning process,
In the plurality of nodes, the number of the layer groups and the channel groups constituting the network structure is adaptively adjusted according to the environmental factors. 5. The method of claim 4,
The plurality of nodes,
When the environmental factor is the size of the memory, the number of the channel groups is reduced,
The artificial neural network device, characterized in that when the environmental factor is the operation speed, the number of the layer groups is adjusted to decrease. 4. The method of claim 3,
A slicing model for the network structure according to the combination of the environmental factors is predefined,
In the plurality of nodes, the artificial neural network apparatus, characterized in that the network structure is reconfigured based on a slicing model corresponding to the environmental factor. The method of claim 1,
The plurality of nodes,
The artificial neural network apparatus, characterized in that the nodes corresponding to the first channel group among the channel groups are trained in advance to enable independent learning operations. The method of claim 1,
The network structure is
An artificial neural network device, characterized in that the i-th channel group of one layer (i is a natural number less than or equal to M) is calculated only from the activation values of the first to i-th channel groups in the previous layer. An operating method of an artificial neural network device including a plurality of nodes consisting of N (N is a natural number greater than or equal to 2) layer group rows and M (M is a natural number) NХM matrix of channel group columns,
An output of a node corresponding to a low level channel in each layer group among the plurality of nodes transmits data to a node corresponding to the same level and high level channel in the next layer group, but each layer The node corresponding to the high-level channel in the group does not transmit data to the node corresponding to the low-level channel in the next layer group,
The method of operating an artificial neural network device, characterized in that it is implemented to enable slicing for each channel group or for each layer group. A computer program stored in a computer-readable recording medium to execute each process included in the method of operating an artificial neural network device according to claim 9 .

Images