KR102180051B1 - Apparatus and method for executing deep learning based on early inference - Google Patents

Abstract

The deep learning device based on early inference according to the present invention includes a multiple output collection unit that collects intermediate output values from all layers of a previously learned deep learning network, and an additional output layer by clustering the collected intermediate output values for each layer. An output layer location inferring unit for inferring a location, and a network learning unit configured to configure a new deep learning network by connecting an output layer to the inferred additional output layer location, and learn the configured new deep learning network.

Description

Deep learning device based on early reasoning and its method {APPARATUS AND METHOD FOR EXECUTING DEEP LEARNING BASED ON EARLY INFERENCE}

The present invention relates to a deep learning technique, and in more detail, based on early inference that can connect the output layers only between layers with relatively high accuracy by grasping the effective location and number of additional output layers with low complexity. It relates to a deep learning device and method thereof.

As is known, the existing deep learning technique is composed of one input layer, several hidden layers, and one output layer, and such deep learning is widely used in various fields due to the high accuracy of deep learning.

However, the existing deep learning technique has a problem that it is difficult to use in a mobile environment due to relatively high complexity.

As one method to solve this problem, deep learning capable of early reasoning has been proposed, and deep learning capable of early reasoning is a method of connecting an additional output layer to the existing deep learning network.

At this time, the number and location of the output layer can be any number or anywhere, but the accuracy and complexity in the learning and inference stages vary depending on the location and number. For this purpose, it is necessary to have high accuracy and low complexity in the learning and inference stages. A new technique is needed to determine the number and location of additional output layers.

There are two basic methods for determining the location of an additional output layer according to the existing method.

1 is a conceptual diagram illustrating a first method of determining a location of an additional output layer according to an existing method.

Referring to FIG. 1, the first conventional method is a method of connecting an output layer to all possible positions of each layer, and as the number of output layers increases, it has high complexity in both the learning and inference stages. there is a problem.

In addition, the first conventional method has a problem in that the overall accuracy is lowered because an output layer is generated even in a portion with low accuracy.

FIG. 2 is a conceptual diagram illustrating an example of a second method of determining a position of an additional output layer according to an existing method.

Referring to FIG. 2, an example of the second conventional method is a method of determining accuracy by connecting an output layer corresponding to one input layer one by one to a place where an output layer can be connected, learning and testing. If the accuracy is high, the location can be selected as a location to connect the output layers, and this method can be carried out for places where all output layers can be connected.

As another example, there is a method of connecting an output layer to a place selected in the previous step (eg, after layer 1 and next to layer 3 selected in FIG. 2) in the existing deep learning network, and then proceeds with the final learning again. .

The second method as described above is a method of finding out and connecting between which layers the output layer should be installed for high accuracy, and the decrease in accuracy is negligible.

On the other hand, there is a problem of having a higher complexity in the learning stage than the previous method by learning by connecting the output layers one by one to all places where the output layers can connect, finding an effective output layer by testing, and then learning once more at the end.

In particular, since there are many places where additional output layers can be connected due to the nature of deep learning with a large number of hidden layers, there is a problem that it has a relatively large complexity compared to deep learning without an additional output layer in the learning stage.

Korean Patent Registration No. 10-1897962 (Announcement date: October 31, 2018)

An object of the present invention is to provide an early inference-based deep learning apparatus and method capable of connecting the output layers only between layers with relatively high accuracy by grasping the effective location and number of additional output layers with low complexity.

An object of the present invention is to provide a computer-readable recording medium in which a computer program is stored that enables a processor to perform a deep learning method based on early inference capable of realizing low complexity learning.

An object of the present invention is to provide a computer program stored in a computer-readable recording medium so that a processor can perform a deep learning method based on early reasoning that can realize low-complexity learning.

The problem to be solved by the present invention is not limited to the ones mentioned above, and another problem to be solved that is not mentioned can be clearly understood by those of ordinary skill in the art from the following descriptions. will be.

According to one aspect, the present invention provides a multiple output collection unit that collects intermediate output values from all layers of a previously learned deep learning network, and infers the location of an additional output layer by clustering the collected intermediate output values for each layer. It is possible to provide an early inference-based deep learning apparatus including an output layer location inference unit and a network learning unit configured to configure a new deep learning network by connecting an output layer to the inferred additional output layer location, and learn the configured new deep learning network. have.

The multi-output collection unit of the present invention may learn a deep learning network without an additional output layer, and multiply output the learned output values for each layer of the deep learning network in the form of classification, layers, and output values for all layers and classifications. have.

The network learning unit of the present invention may configure the new deep learning network by connecting an output layer to a location of an additional output layer inferred by the output layer location inference unit to a deep learning network without an additional output layer.

The output layer position inference unit of the present invention comprises a partitioning method, a hierarchical method, a density-based method, a grid-based method, and the collected intermediate output values. Clustering can be done through any of the model-based methods.

The partitioning method of the present invention may be a k-means algorithm or a Nearest nearest neighbor algorithm.

The layered method of the present invention may be a BIRCH algorithm or a Chameleon algorithm.

The density-based method of the present invention may be a DBSCAN algorithm or a DENCLUE algorithm.

The grid-based method of the present invention may be a STING algorithm.

The model-based method of the present invention may be an EM algorithm (Expectation-Maximization algorithm) or a Neural Network algorithm.

The output layer location inference unit of the present invention determines whether clusters of all classifications exist-the classification of the cluster is a classification that occupies the largest proportion among the classifications of the input values of elements included in the cluster-and all the clusters are each The position of the additional output layer may be inferred according to a condition that satisfies whether the classification of the input value in the cluster of is satisfied whether or not an element that is the classification of the cluster exists in a certain ratio or more.

According to another aspect, the present invention includes the steps of collecting intermediate output values from all layers of a previously learned deep learning network, clustering the collected intermediate output values for each layer to infer the position of an additional output layer, It is possible to provide a deep learning method based on early inference including configuring a new deep learning network by connecting an output layer to the inferred additional output layer location, and learning the configured new deep learning network.

In the collecting step of the present invention, a deep learning network without an additional output layer is learned, and output values for each class of each layer of the learned deep learning network can be multiplexed out in the form of classification, layer, and output values for all layers and classifications. have.

In the configuring of the present invention, the new deep learning network may be configured by connecting an output layer to a location of an additional output layer inferred by the output layer location inference unit to a deep learning network without an additional output layer.

In the inferring step of the present invention, whether there are clusters of all classifications-the classification of the clusters is a classification that occupies the largest percentage among the classifications of input values of elements included in the clusters-and all the clusters are The position of the additional output layer may be inferred according to a condition that satisfies whether the classification of the input value in the cluster of is satisfied whether or not an element that is the classification of the cluster exists in a certain ratio or more.

According to another aspect, the present invention includes the steps of collecting intermediate output values from all layers of a previously learned deep learning network, clustering the collected intermediate output values for each layer to infer the position of an additional output layer, and , A computer program for causing a processor to perform an early inference-based deep learning method comprising the steps of configuring a new deep learning network by connecting an output layer to the inferred additional output layer location, and learning the configured new deep learning network The stored computer-readable recording medium can be provided.

According to another aspect, the present invention includes the steps of collecting intermediate output values from all layers of a previously learned deep learning network, clustering the collected intermediate output values for each layer to infer the position of an additional output layer, and , Computing a new deep learning network by connecting an output layer to the inferred additional output layer location, and learning the configured new deep learning network, so that the processor can perform a deep learning method based on early inference. A computer program stored in a readable recording medium can be provided.

According to an embodiment of the present invention, by identifying the effective location and number of additional output layers with low complexity and using them to connect the output layers only between layers with relatively high accuracy, early inference is possible while maintaining the accuracy of the existing deep learning as much as possible. It can realize low complexity learning.

1 is a conceptual diagram illustrating a first method of determining a location of an additional output layer according to an existing method.
FIG. 2 is a conceptual diagram illustrating an example of a second method of determining a position of an additional output layer according to an existing method.
3 is a conceptual diagram for explaining a technology concept for performing deep learning based on early inference according to an embodiment of the present invention.
4 is a block diagram of an apparatus for deep learning based on early inference according to an embodiment of the present invention.
5 is a conceptual diagram illustrating a principle of collecting intermediate output values of all layers through a multiple output collection unit according to an embodiment of the present invention.
6 is an exemplary view showing a result of clustering intermediate output values collected through FIG. 5 for each layer.
7 is a conceptual diagram illustrating a concept of finding a location of an additional output layer through clustering according to an embodiment of the present invention.
8 is an exemplary diagram for explaining a concept of configuring a new deep learning network by connecting an output layer to a location of an additional output layer according to an embodiment of the present invention.
9 is a flow chart showing a main process of performing deep learning based on early inference according to an embodiment of the present invention.

Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various forms, and only these embodiments make the disclosure of the present invention complete, and those skilled in the art to which the present invention pertains. It is provided to fully inform the person of the scope of the invention, and the scope of the invention is only defined by the claims.

In describing the embodiments of the present invention, detailed descriptions of known functions or configurations will be omitted except when actually necessary in describing the embodiments of the present invention. In addition, terms to be described later are terms defined in consideration of functions in an embodiment of the present invention, which may vary according to the intention or custom of users or operators. Therefore, the definition should be made based on the contents throughout this specification.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

3 is a conceptual diagram for explaining a technology concept for performing deep learning based on early inference according to an embodiment of the present invention.

Referring to FIG. 3, in step 1, input is input from an existing deep learning network without additional learned layers (that is, a deep learning network without early output) and intermediate output values from all layers other than the output layer are collected. Together, collecting the median outputs from all layers can be defined as collecting multiple outputs.

Next, in step 2, the intermediate output values collected through the multiple outputs in the above-described step 1 are clustered for each layer, and after such clustering, they are added to each layer by referring to the classification of the input values of the elements (output values) of the clustered clusters. It can be determined whether or not it is effective to connect the output layers (additional output layers).

And, in step 3, a new deep learning network in which the output layer is connected to the effective additional output layer position determined in step 2 can be constructed and learned.

In the deep learning technique of the present invention as described above, the location and number of additional output layers are not limited, and if an effective output layer is not found, the additional output layers may not be connected. Also, if all layers are additional output layers effective, additional output layers can be connected to all layers.

In the deep learning technique of the present invention, complexity can be effectively reduced by collecting and clustering intermediate output values of all layers through multiple outputs in the step of learning one by one while connecting the basic output layers one by one.

Here, the deep learning model based on early inference according to the present invention may be used in various fields, such as a behavior recognition field, a vision recognition field, a face detection field, an expression detection field, an iris recognition field, and the like.

4 is a block diagram of a deep learning apparatus based on early inference according to an embodiment of the present invention, which includes a multiple output collection unit 402, an output layer location inference unit 404, and a network learning unit 406. I can.

Referring to FIG. 4, the multiple output collection unit 402 collects intermediate output values from all layers of the deep learning network that have been previously learned, and transfers intermediate output values of all the collected layers to the output layer location inference unit 404, etc. Can provide the function of.

That is, the multiple output collection unit 402 may collect intermediate output values of all layers by putting a sufficient number of inputs including all classifications into the learned deep learning network through multiple outputs.

In addition, the multiple output collection unit 402 learns a deep learning network without an additional output layer, and multiplex outputs output values for each layer of the learned deep learning network in the form of classification, layers, and output values for all layers and classifications. have.

Next, the output layer position inference unit 404 clusters the intermediate output values collected through the multiple output collection unit 402 for each layer (clustering step) to infer the position of the additional output layer (position inference step), and this A function such as transmitting the inferred additional output layer position to the network learning unit 406 may be provided.

Here, the output layer location inference unit 404 determines whether clusters of all classifications exist-the classification of the cluster is a classification that occupies the largest proportion of the classification of the input values of the elements included in the cluster-and all clusters are each The position of the additional output layer can be inferred according to the condition that the classification of the input value in the cluster of is satisfied whether or not an element that is the classification of the cluster exists at a certain ratio.

And, the output layer position inference unit 404, for example, a partition method (Partitional method), a hierarchical method (Hierarchical method), a density-based method (Density-based method), a grid-based method (Grid-based method), a model-based method You can cluster the collected intermediate output values by using any one of the (Model-based method).

Here, the segmentation method may be, for example, a k-means algorithm or a Nearest Neighbor algorithm, and the hierarchical method may be, for example, a BIRCH algorithm or a Chameleon algorithm. The density-based method may be, for example, a DBSCAN algorithm or a DENCLUE algorithm, and the grid-based method may be, for example, a STING algorithm, and the model-based method is, for example, an EM algorithm (Expectation-Maximization algorithm) or It may be a neural network algorithm.

5 is a conceptual diagram illustrating a principle of collecting intermediate output values of all layers through the multiple output collection unit 402 according to an embodiment of the present invention.

Referring to FIG. 5, for example, in order to classify Red, Green, Blue, (r ₀ , Red), (g ₀ , Green), (b ₀ , Blue), etc. are input and trained deep neural networks (DN, N _{When 0} ) is present, if the intermediate output from the n-th layer is expressed as (Red, n, r _n ) (assuming that the intermediate output in the form of r _n is generated from the n layer for the Red output), it is collected and It is possible to determine whether to connect an additional (additional) output layer by classifying by layer.

For example, wherever intermediate output is possible, (Red, 1, r ₁ ), (Red, 2, r ₂ ), (Red, 3, r ₃ ), (Green, 1, g ₁ ), (Green, 2 , g ₂ ), (Green, 3, g ₃ ), (Blue, 1, b ₁ ), (Blue, 2, b ₂ ), (Blue, 3, b ₃ ) can be collected, such a collection A sufficient number can be performed for all classifications.

6 is an exemplary view showing a result of clustering intermediate output values collected through FIG. 5 for each layer.

That is, the step of inferring the position of the additional output layer through clustering by the output layer location inferring unit 404 is a step to determine which layer is effective to connect the additional output layer to which layer by using the clusters classified through the clustering step.

At this time, an effective layer for connecting the additional output layer must satisfy the following conditions.

First, there must be clusters of all classifications in each layer, and the classification of the input value occupying the most for each cluster is called the classification of the cluster.

Second, for each cluster of each layer, the factor of which the classification of the input value is the classification of the cluster must occupy a ratio of more than a certain ( α ).

FIG. 7 shows a method of calculating a condition for finding an effective layer above using the cluster of FIG. 6 as an example when α=80% , and obtaining an effective position of an additional output layer through this. For example, only the first layer is an effective location by satisfying both conditions.

Referring to FIG. 4 again, the network learning unit 406 configures a new deep learning network by connecting the output layer to the position of the effective additional output layer inferred through the output layer location inference unit 404, and learns the configured new deep learning network. It can provide functions such as.

In addition, the network learning unit 406 may configure a new deep learning network by connecting the output layer to the location of the additional output layer inferred by the output layer location inference unit 404 to the deep learning network without the additional output layer.

That is, the network learning unit 406 may construct and learn a new deep learning network by using the effective position of the additional output layer inferred through clustering by the output layer location inference unit 404.

8 is an exemplary view illustrating a concept of configuring a new deep learning network by connecting an output layer to a location of an additional output layer according to an embodiment of the present invention.

Referring to FIG. 8, if it is inferred that the output layers connected to Layer 1 and Layer 3 are effective through clustering in the output layer location inferring unit 404, a deep learning network in which Layer 1 and Layer 3 output layers are additionally connected may be configured.

Here, by scaling down the step of connecting and learning one by one for all connectable locations of the additional output layer to the clustering step for each layer, the complexity can be effectively reduced.

Next, a series of processes for performing deep learning based on early inference using the deep learning apparatus according to the present embodiment having the configuration as described above will be described in detail.

9 is a flow chart showing a main process of performing deep learning based on early inference according to an embodiment of the present invention.

Referring to FIG. 9, the multi-output collection unit 402 collects intermediate output values from all layers of the previously learned deep learning network, that is, inputs a sufficient number including all classifications through multiple outputs to the learned deep learning network. To collect the median output values of all layers (step 902).

At this time, the multi-output collection unit 402 learns the deep learning network without an additional output layer, and multiplex outputs the output values for each layer of the learned deep learning network in the form of classification, layer, and output values for all layers and classifications. have.

Next, the output layer location inferring unit 404 infers the additional output layer location by clustering the intermediate output values collected through the multiple output collection unit 402 for each layer (step 904).

Here, the output layer location inferring unit 404 may infer the location of the additional output layer according to whether conditions (1) and (2) below are satisfied.

(1) Whether clusters of all classifications exist-Classification of clusters is the classification that occupies the largest proportion among the classifications of the input values of elements included in the cluster.

(2) For all clusters, whether the classification of the input value in each cluster exists in a certain percentage or more

At this time, the output layer location inference unit 404 includes, for example, a partitional method, a hierarchical method, a density-based method, a grid-based method, and a model-based method. You can cluster the collected intermediate output values by using any one of the (Model-based method).

Then, the network learning unit 406 configures a new deep learning network by connecting the output layer to the position of the effective additional output layer inferred through the output layer location inference unit 404 (step 906). That is, a new deep learning network is constructed using the effective position of the additional output layer inferred through clustering.

In this case, the network learning unit 406 may configure a new deep learning network by connecting the output layer to the location of the additional output layer inferred by the output layer location inference unit 404 to the deep learning network without an additional output layer.

That is, the network learning unit 406 may configure a new deep learning network by using the effective position of the additional output layer inferred through clustering by the output layer location inference unit 404. Thereafter, the network learning unit 406 learns the configured new deep learning network (step 908).

delete

Meanwhile, combinations of each block of the attached block diagram and each step of the flowchart may be performed by computer program instructions. Since these computer program instructions can be mounted on a processor of a general purpose computer, special purpose computer, or other programmable data processing equipment, the instructions executed by the processor of the computer or other programmable data processing equipment are shown in each block or flow chart of the block diagram. Each step creates a means to perform the functions described.

These computer program instructions can also be stored in a computer-usable or computer-readable memory, etc., which can be directed to a computer or other programmable data processing equipment to implement a function in a specific manner, so that the computer-usable or computer-readable memory It is also possible to produce an article of manufacture in which the instructions stored in the block diagram contain instruction means for performing the functions described in each block or flow chart.

In addition, since computer program instructions can be mounted on a computer or other programmable data processing equipment, a series of operation steps are performed on a computer or other programmable data processing equipment to create a process that is executed by a computer, It is also possible for the instructions to perform possible data processing equipment to provide steps for executing the functions described in each block of the block diagram and each step of the flowchart.

In addition, each block or each step may represent a module, segment, or part of code including at least one or more executable instructions for executing the specified logical function(s). It should also be noted that in some alternative embodiments, functions mentioned in blocks or steps may occur out of order. For example, two blocks or steps shown in succession may in fact be performed substantially simultaneously, or the blocks or steps may sometimes be performed in the reverse order depending on the corresponding function.

The above description is merely illustrative of the technical idea of the present invention, and those of ordinary skill in the art to which the present invention pertains, various substitutions, modifications, and changes, etc., within the scope not departing from the essential characteristics of the present invention. It will be easy to see that this is possible. That is, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain the technical idea, and the scope of the technical idea of the present invention is not limited by these embodiments.

Accordingly, the scope of protection of the present invention should be interpreted by the claims to be described later, and all technical thoughts within the scope equivalent thereto should be interpreted as being included in the scope of the present invention.

402: multiple output collection unit
404: output layer location inference unit
406: Network Learning Department

Claims (16)

A multiple output collection unit that collects intermediate output values from all layers of the previously learned deep learning network,
An output layer location inference unit for inferring an additional output layer location by clustering the collected intermediate output values for each layer,
A network learning unit for configuring a new deep learning network by connecting an output layer to the inferred additional output layer location, and learning the configured new deep learning network
Deep learning device based on early reasoning comprising a. The method of claim 1,
The multiple output collection unit,
Learning a deep learning network without an additional output layer, and multiplying the learned output values for each layer of the deep learning network in the form of classification, layer, and output values for all layers and classifications.
Deep learning device based on early reasoning. The method of claim 1,
The network learning unit,
Configuring the new deep learning network by connecting the output layer to the location of the additional output layer inferred by the output layer location inference unit to the deep learning network without an additional output layer.
Deep learning device based on early reasoning. The method of claim 1,
The output layer location inference unit,
The collected intermediate output values are divided into a partitional method, a hierarchical method, a density-based method, a grid-based method, and a model-based method. Clustering through either method
Deep learning device based on early reasoning. The method of claim 4,
The division method,
k-means algorithm or Nearest Nearest Neighbor algorithm
Deep learning device based on early reasoning. The method of claim 4,
The layered method,
BIRCH algorithm or Chameleon algorithm
Deep learning device based on early reasoning. The method of claim 4,
The density-based method,
DBSCAN algorithm or DENCLUE algorithm
Deep learning device based on early reasoning. The method of claim 4,
The grid-based method,
STING algorithm
Deep learning device based on early reasoning. The method of claim 4,
The model-based method,
EM algorithm (Expectation-Maximization algorithm) or Neural Network algorithm
Deep learning device based on early reasoning. The method of claim 1,
The output layer location inference unit,
Whether there are clusters of all classifications-the classification of the cluster is a classification that occupies the largest proportion among the classifications of the input values of elements included in the cluster; and,
For all the clusters, whether or not there are more than a certain percentage of the element whose classification of the input value is the classification of the cluster in each cluster.
To infer the position of the additional output layer according to the condition that satisfies
Deep learning device based on early reasoning. As a deep learning method based on early reasoning performed by the deep learning device based on early reasoning,
Collecting intermediate output values from all layers of the previously learned deep learning network,
Inferring an additional output layer location by clustering the collected intermediate output values for each layer; and
Configuring a new deep learning network by connecting an output layer to the inferred additional output layer location;
Learning the constructed new deep learning network
Early inference-based deep learning method including a. The method of claim 11,
The collecting step,
Learning a deep learning network without an additional output layer, and multiplying the learned output values for each layer of the deep learning network in the form of classification, layer, and output values for all layers and classifications.
Deep learning method based on early reasoning. The method of claim 11,
The configuring step,
Configuring the new deep learning network by connecting the output layer to the additional output layer location inferred in the output layer location inference step to a deep learning network without an additional output layer
Deep learning method based on early reasoning. The method of claim 11,
The inferring step,
Whether there are clusters of all classifications-the classification of the cluster is a classification that occupies the largest proportion among the classifications of the input values of elements included in the cluster; and,
In the clusters of all the classifications, whether or not there is a certain percentage or more of an element that is the classification of the input value in each cluster.
To infer the position of the additional output layer according to the condition that satisfies
Deep learning method based on early reasoning. Collecting intermediate output values from all layers of the previously learned deep learning network,
Inferring an additional output layer location by clustering the collected intermediate output values for each layer; and
Configuring a new deep learning network by connecting an output layer to the inferred additional output layer location;
Learning the constructed new deep learning network
A computer-readable recording medium storing a computer program that causes the processor to perform the deep learning method based on early inference, including. Collecting intermediate output values from all layers of the previously learned deep learning network,
Inferring an additional output layer location by clustering the collected intermediate output values for each layer; and
Configuring a new deep learning network by connecting an output layer to the inferred additional output layer location;
Learning the constructed new deep learning network
A computer program stored in a computer-readable recording medium so that the processor can perform the deep learning method based on early inference including a.

Images