KR20180052990A - Apparatus and method for learning deep neural network - Google Patents

Abstract

The present invention relates to a learning apparatus of a deep neural network (DNN) and a method thereof. The learning apparatus of a DNN includes a data set including at least one data category, a category-specific learning module for generating at least one DNN learned based on the at least one data category, and a selective DNN learning module for generating one integrated DNN by integrating the at least one DNN selected in association with an inputted learning target category for supplying a target function. Accordingly, the present invention can reduce the learning time of the DNN.

Description

[0001] APPARATUS AND METHOD FOR LEARNING DEEP NEURAL NETWORK [0002]

The present invention relates to a learning apparatus and method of a deep neural network (DNN), and more particularly, to a learning apparatus and method of a DNN capable of shortening a learning time of a DNN supporting one or more data categories will be.

Deep neural network (DNN) is an artificial neural network composed of multiple hidden layers between the input and output layers and is applied to various fields such as object recognition, speech recognition, and parsing.

DNN can proceed with learning based on the data set associated with the target function to support the target function. That is, a DNN that wishes to support different functions can proceed to learn based on a different data set.

In the learning method of this DNN, in order to learn the DNN supporting the target specific function in an environment in which the entire available data set can be classified into one or more categories, The DNN is learned from the beginning based on the data belonging to the data category related to the data category.

However, since each of the conventional techniques learns DNN from the beginning based on data belonging to a data category related to a target specific function every time the DNN is learned, it takes a lot of time to learn as the number of DNN layers increases .

SUMMARY OF THE INVENTION It is an object of the present invention to provide a DNN learning apparatus and method for shortening a DNN learning time for constructing a DNN supporting a specific function.

In order to achieve the above object, a DNN learning apparatus of the present invention includes a data set including at least one data category, at least one deep artificial neural network (DNN) learned based on the at least one data category, deep neural network), and an optional DNN learning module that combines the at least one DNN selected in association with the input learning category to support the target function to generate a single integrated DNN do.

In addition, the learning module for each category can learn each DNN for each individual data category of the data set, and generate n learned DNNs for n data categories.

In addition, the category-specific learning module may learn one DNN based on a plurality of data categories of the data set to generate m learned DNNs less than n for n data categories.

In addition, the optional DNN learning module may include an integrator that integrates at least one learned DNN of the learning module selected in association with the learning target category to generate an integrated DNN, And a controller that further learns the integrated DNN based on at least one data category of the set.

The DNN learning method of the present invention generates at least one learned deep neural network (DNN) learned based on at least one data category included in a data set in a category learning module Integrating at least one learned DNN of the selected learning module in association with an input learning category to support a target function in the optional DNN learning module to generate an integrated DNN; Determining in the optional DNN learning module whether the integrated DNN is to be further learned based on at least one data category of the data set associated with the learning target category; And generating a DNN that further learns the integrated DNN.

The DNN learning apparatus and method according to the present invention generate DNNs learned based on a specific data category of a data set so as to construct a DNN supporting a target function, and utilize data categories and learned DNNs By learning and integrating DNN, learning time can be shortened by learning DNN from the beginning based on data belonging to related data categories.

1 is a block diagram showing a DNN learning apparatus according to an embodiment.
2 is a flowchart illustrating a learning method of a DNN according to an embodiment.
FIG. 3 is a schematic diagram illustrating selective learning of a DNN for network traffic-based application classification according to a learning method of a DNN according to an embodiment.
FIG. 4 is a schematic diagram illustrating selective learning of DNN supporting natural language processing of various languages by a learning method of DNN according to an embodiment.
FIG. 5 is a schematic diagram illustrating selective learning of a hierarchical DNN supporting network traffic-based application classification according to an embodiment of a learning method of DNN.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It is to be understood, however, that the invention is not to be limited to the specific embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. Like reference numerals are used for like elements in describing each drawing.

The terms first, second, A, B, etc. may be used to describe various elements, but the elements should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In the present application, the terms "comprises" or "having" and the like are used to specify that there is a feature, a number, a step, an operation, an element, a component or a combination thereof described in the specification, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the contextual meaning of the related art and are to be interpreted as either ideal or overly formal in the sense of the present application Do not.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing a learning apparatus of a deep neural network (DNN) according to an embodiment. 1, the DNN learning apparatus according to the embodiment includes an input module 10, a data set 30, a learning module 50 for each category, an optional DNN learning module 70, And an output module 90.

When the learning target category is determined to support the target function, the input module 10 inputs the learning target category to the optional DNN learning module 70. Here, the learning target category is at least one data category to be used for learning the target DNN.

The data set 30 includes at least one data category 31. Here, the data included in the data set 30 can be classified into the data category 31 according to various criteria. For example, data belonging to a particular category (eg computer, person, etc.) can be categorized into the same data category. In addition, data showing similar results in the preprocessing process using DNN and the like can be classified into the same data category.

Accordingly, the data category 31 may include various types of data that can be used to train the DNN to support the target function. For example, the data category 31 may include image, image, voice, text, etc. or special types of data processed and processed therefrom.

That is, as described above, the data set 30 can generate one or more data categories 31 with data satisfying the same criterion, so that the data category 31 including the data satisfying the specific classification criterion is one Or more.

The category learning module 50 learns individually based on the data category 31 to generate at least one DNN. Here, the category-specific learning module 50 generates one integrated DNN generated from the optional DNN learning module 70 or a learned DNN 51 based on the integrated and adjusted DNN.

For example, the per-category learning module 50 may learn one DNN for each data category 31 of the data set 30 to generate one learned DNN 51. That is, one DNN is learned only based on a specific one data category 31. Thus, each DNN is learned for each individual data category 31 of the data set 30, and the category-specific learning module 50 generates n learned DNNs 51 for n data categories 31 can do.

In addition, the category-specific learning module 50 can learn one DNN on the basis of a plurality of data categories 31. Thus, the per-category learning module 50 can generate m learned DNNs 51 less than n for n data categories 31. [

The optional DNN learning module 70 includes a merger 71 and a fine-tuner 73.

Here, the optional DNN learning module 70 may include at least one of the selected learning modules 50 associated with the learning target category when the learning target category is input from the input module 10 to support the target function And then integrates the learned DNN 51 to generate an integrated DNN supporting the target function.

Alternatively, the optional DNN learning module 70 may be adapted to perform additional learning of the integrated DNN based on at least one data category 31 of the data set 30 associated with the learning target category, To generate an integrated and controlled DNN that supports the target function.

The integrator 71 may combine the learned DNNs 51 for each data category 31 to generate one integrated DNN. That is, the integrator 71 integrates at least one learned DNN 51 selected in association with the learning target category to generate one integrated DNN.

In integrating the DNNs 51 learned for each data category 31 into one DNN, the integrator 71 integrates the variable values (weight, bias, etc.) of the middle layers excluding the final output layer . It is also possible to integrate the variable values of all layers including the final output layer. In addition, the number of classes classified by the final output layer of the newly generated DNN can be adjusted differently from the number of classes classified by the final output layer of the integrated DNNs.

In addition, the integrator 71 may apply various methods to generate one integrated DNN by integrating the variable values of the learned DNNs 51. [

Here, the integrator 71 may integrate the variable values of the learned DNNs 51, reflecting the relative importance of the data categories 31 to be integrated. For example, based on a data category with a higher relative importance, the value of a variable in the learned DNN can be incorporated into a higher weight relative to the value of a previously learned DNN based on a data category that is not.

In addition, integrator 71 may determine the value of the variable DNNs 51 of the learned DNNs 51, reflecting the relative differences in the values of the variables within the DNN to be merged or merged, within a particular layer, Lt; / RTI > For example, a variable value that shows a larger value in a certain area can be integrated with a higher weight value than the other value.

In addition, the integrator 71 can integrate the variable values of the learned DNNs 51 based on the original value without converting the variable values of the learned DNNs 51. [

In addition, the integrator 71 may integrate the variable values of the learned DNNs 51 after converting the variable values of the learned DNNs 51 into a specific form. For example, the variable values of the learned DNNs 51 may be converted into the binary variable values, and then the variable values of the learned DNNs 51 may be integrated. Further, it is possible to integrate the variable values of the learned DNNs 51 after quantizing the variable values of the learned DNNs.

Also, the integrator 71 may integrate the variable values of the learned DNNs 51 after converting one or more variable values of the learned DNNs 51 into one variable value.

In addition, the integrator 71 may generate an integrated DNN based on the output values of the respective layers of the learned DNNs 51. [

The regulator 73 utilizes the data in the data category 31 of the data set 30 to proceed with further learning to the integrated DNN of the integrator 71. Here, further learning of the regulator 73 can be applied as needed. That is, when additional learning is required in the DNN integrated by the integrator 71, the adjuster 73 may proceed with further learning, i.e., fine-tuning.

That is, the adjuster 73 further learns the integrated DNN based on at least one data category 31 of the data set 30 associated with the learning target category.

At this time, the regulator 73 may use the entire data of the associated data category 31 in additionally learning the integrated DNN.

In addition, the regulator 73 may use some data in the associated data category 31 in further learning the integrated DNN.

In addition, the regulator 73 may use the data of the associated data category 31 until it achieves the desired specific accuracy in further learning the integrated DNN.

In addition, the adjuster 73 may learn the entire DNN in addition learning the integrated DNN.

In addition, the adjuster 73 may learn a portion of the integrated DNN in addition learning the integrated DNN.

In addition, the regulator 73 may layer-wise train the integrated DNN in the learning of the integrated DNN.

In addition, the regulator 73 may learn the integrated DNN based on the final output value in additionally learning the integrated DNN.

The output module 90 outputs the DNN generated by the optional DNN learning module 70.

The operation of the DNN learning apparatus according to the embodiment having such a configuration will be described as follows.

2 is a flowchart illustrating a learning method of a DNN according to an embodiment. 2, the DNN learning method according to the embodiment is characterized in that the data set 30 includes at least one data category 31 and the learning module 50 for each category has the data category 31, And generates at least one learned DNN 51 (S101).

Thereafter, the optional DNN learning module 70 receives the learning target category via the input module 10. Then, the integrator 71 integrates the learned DNNs 51 into one DNN based on the data category 31 related to the inputted learning target category to generate an integrated DNN (S103).

Then, the optional DNN learning module 70 determines whether the integrated DNN is additionally learned (S105).

Subsequently, when further learning of the integrated DNN is required, the regulator 73 further learns the integrated DNN based on the data category 31 related to the input learning target category to generate the adjusted DNN ( S107). The optional DNN learning module 70 outputs the integrated and adjusted DNN generated through the integrator 71 and the regulator 73 through the output module 90 (S109).

On the other hand, if additional learning of the integrated DNN is not required, the optional DNN learning module 70 outputs the integrated DNN generated through the integrator 71 (S109a).

Hereinafter, as a first application example of the learning method of DNN according to the embodiment, selective learning of DNN for network traffic based application classification will be described as follows.

FIG. 3 is a schematic diagram illustrating selective learning of a DNN for network traffic-based application classification according to a learning method of a DNN according to an embodiment. As shown in FIG. 3, the data set 30 first includes at least one data category 31, namely, BitTorrent, Apple, Skype, and Google. Here, the data set 30 includes characteristics of network traffic associated with a particular application. And the data category 31 is formed on the basis of the related application.

Then, the category learning module 50 individually learns based on the data category 31 and obtains at least one learned DNN 51, that is, the learned bit torrent, the learned apple, the learned skype, And the learned Google. Here, each learned DNN 51 can classify a specific application based on network traffic.

Thereafter, the optional DNN learning module 70 receives the learning target category via the input module 10. Here, the optional DNN learning module 70 can receive a list of applications to be classified and to be classified based on the network traffic based on the integrated and controlled DNN, which will be described later.

Then, the integrator 71 integrates the learned DNNs 51, that is, the learned bit torrent and the learned-learned apple into one DNN based on the data category 31 related to the inputted learning target category Generate an integrated DNN.

Then, the regulator 73 further learns the integrated DNN based on the data category 31 related to the input learning target category, that is, bit torrent and Apple, and generates the adjusted DNN.

And the optional DNN learning module 70 outputs the integrated and adjusted DNN generated through the integrator 71 and the regulator 73. [

Hereinafter, as a second application example of the learning method of DNN according to the embodiment, selective learning of DNN supporting natural language processing of various languages will be described as follows.

FIG. 4 is a schematic diagram illustrating selective learning of DNN supporting natural language processing of various languages by a learning method of DNN according to an embodiment. As shown in Figure 4, the data set 30 first comprises at least one data category 31, i.e., Korean, English, Chinese and German. Here, the data set 30 includes features of the language. And the data category 31 is formed based on the language.

Then, the learning module 50 for each category individually learns based on the data category 31 and generates at least one learned DNN 51, that is, the learned Korean, the learned English, Generates a learned German language. Here, each learned DNN 51 can understand and process a specific language based on the data category 31.

Thereafter, the optional DNN learning module 70 receives the learning target category via the input module 10. Here, the optional DNN learning module 70 may receive a list of languages to be processed by the integrated and adjusted DNN, which will be described later, as input.

Then, the integrator 71 integrates the learned DNNs 51, that is, the learned Korean and the learned English into one DNN based on the data category 31 related to the inputted learning target category, DNN < / RTI >

Then, the regulator 73 further learns the integrated DNN based on the data category 31 related to the inputted learning target category, i.e., Korean and English, and generates the adjusted DNN.

And the optional DNN learning module 70 outputs the integrated and adjusted DNN generated through the integrator 71 and the regulator 73. [

Hereinafter, as a third application example of the DNN learning method according to the embodiment, selective learning of hierarchical DNN supporting network traffic based application classification will be described as follows.

FIG. 5 is a schematic diagram illustrating selective learning of a hierarchical DNN supporting network traffic-based application classification according to an embodiment of a learning method of DNN. As shown in FIG. 5, the data set 30 first includes at least one data category 31, namely, BitTorrent, Apple, Skype, and Google. Here, the data set 30 includes characteristics of network traffic associated with a particular application. And the data category 31 is formed on the basis of the related application.

Then, the category learning module 50 individually learns based on the data category 31 and obtains at least one learned DNN 51, that is, the learned bit torrent, the learned apple, the learned skype, And the learned Google. Here, each learned DNN 51 can classify a specific application based on network traffic.

Thereafter, the optional DNN learning module 70 receives the learning target category via the input module 10. Here, the optional DNN learning module 70 can receive a list of applications to be classified and to be classified based on the network traffic based on the integrated and controlled DNN, which will be described later.

Then, the integrator 71 integrates the learned DNNs 51, that is, the learned bit torrent and the learned-learned apple into one DNN based on the data category 31 related to the inputted learning target category And generates a DNN that can infer the upper class, that is, the data category 31.

Then, the regulator 73 further learns the integrated DNN (the superclass) based on the data category 31 related to the inputted learning target category, that is, the bittorrent and Apple, and generates the adjusted DNN.

And the optional DNN learning module 70 outputs an integrated and adjusted DNN, i.e., a hierarchical DNN (learned bittorrent and learned apple) generated through integrator 71 and regulator 73. [ Here, the hierarchical DNN may include a DNN that deduces an integrated superclass based on the learned DNNs 51 and a learned DNN that deduces a subclass within the superclass.

Here, as in the third application example of the learning method of the DNN according to the embodiment, the learning apparatus of the DNN according to the embodiment can be applied to the hierarchical DNN.

In other words, hierarchical DNN proceeds step by step. For example, in the first step, we deduce an ancestor class (eg, dog) to which the given data belongs, and in the second step we can deduce a subclass in the inferred superclass (eg.

The DNN learning apparatus according to the embodiment classifies data categories related to a target function into upper classes based on a given data set 30 and DNNs 51 learned based on each data category 31 The enemy DNN can be constructed.

In addition, the DNN learning apparatus according to the embodiment constructs one DNN that deduces the upper class by integrating the learned DNNs inferring the lower class according to the operations of the integrator 71 and the regulator 73 can do. That is, the newly generated DNN can deduce the data category to which the given data belongs.

In addition, the learning apparatus of the DNN according to the embodiment may use the learned DNNs to directly infer the subclass in generating the hierarchical DNN, or may use the structure and variables of the learned DNNs You can change the value.

As described above, the DNN learning apparatus and method according to the embodiment generate the learned DNNs based on the specific data category of the data set so as to construct the DNN supporting the target function, It is possible to shorten the learning time because the learning speed is fast as compared with the conventional technique in which the DNN is learned from the beginning based on the data belonging to the related data category.

In addition, the learning apparatus and method of DNN according to the embodiment learn DNN based on the entire data set by integrating and adjusting DNN by using data category and pre-learned DNNs, and then learning DNN based on the entire data set, The accuracy of the DNN learning can be improved compared to the method of learning DNN by fine-tuning the learned DNN only based on the data belonging to it.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the present invention as defined by the following claims It can be understood that

10: input module 30: data set
31: Data category 50: Learning module by category
51: learned DNN 70: optional DNN learning module
71: integrator 73: regulator
90: Output module

Claims (1)

A data set comprising at least one data category;
A category learning module for generating at least one deep neural network (DNN) learned based on the at least one data category; And
And an optional DNN learning module for combining the at least one DNN selected in association with the input learning category to support a target function to generate one integrated DNN.

Images