KR102139729B1 - Electronic apparatus and method for re-learning of trained model thereof - Google Patents

Abstract

A method of re-learning a learning model is disclosed. The re-learning method includes receiving a learning model composed of a plurality of neurons, and a data set including a new task, identifying neurons associated with new tasks among the plurality of neurons, and parameters related to new tasks for the identified neurons And selectively re-learning, and reconstructing the input learning model by dynamically expanding the size of the learning model in which the selective re-learning is performed when the learning model in which the selective re-learning has a predetermined loss value. .

Description

ELECTRONIC APPARATUS AND METHOD FOR RE-LEARNING OF TRAINED MODEL THEREOF}

The present disclosure relates to a re-learning method of an electronic device and a learning model, and more specifically, to an electronic device and a learning model generating method capable of partially learning only tasks corresponding to a new concept when learning a new concept. It is about.

Lifelong learning is a field of continuous learning and real-time transfer learning. When new concepts are learned, it improves the performance of previously learned concepts and utilizes the existing knowledge to help learn new concepts. It has an ideal goal to give.

In the case of gradual learning according to the existing time series, when learning a new concept, the problem of degrading the overall performance by forgetting the previously learned concept is common, and this is called semantic drift.

As a conventional solution for solving this problem, there exist methods such as learning a new concept through network expansion while maintaining the learned existing concepts. However, at this time, the network expands by a fixed size, and the calculation cost increases rapidly, or there is a limitation that it cannot actively cope with the situation of the network model.

Accordingly, an object of the present disclosure is to provide an electronic device and a learning model generation method capable of performing partial learning only on a task corresponding to a new concept when learning a new concept.

A method of re-learning a learning model of the present disclosure for achieving the above-described object includes receiving a data set including a learning model composed of a plurality of neurons and a new task, and the new task among the plurality of neurons. Identifying related neurons, selectively re-learning parameters related to the new task for the identified neurons, and learning, in which the selective re-learning is performed when the learning model in which the selective re-learning is performed has a predetermined loss value. And reconstructing the input learning model by dynamically expanding the size of the model.

In this case, the selective re-learning step calculates a new parameter matrix such that the objective function having a loss function for the input learning model and a normalization term for scarcity is minimized, and uses the calculated new parameter matrix to calculate the new task. It can identify neurons associated with.

On the other hand, in the selective re-learning step, a new parameter matrix is calculated using the data set for network parameters composed of only the identified neurons, and the calculated new parameter matrix is reflected in the identified neurons of the learning model to be selective. Re-learning can be performed.

Meanwhile, in the reconstructing the input learning model, if the learning model in which the selective re-learning is performed has a predetermined loss value, a fixed number of neurons is added to each layer in the learning model in which the selective re-learning is performed, and the group is added. The input learning model may be reconstructed by removing unnecessary neurons from the added neurons using scarcity.

In this case, reconstructing the input learning model may include unnecessary neurons among the added neurons using an objective function having a loss function for the input learning model, a normalization term for sparseness, and a group normalization term for group scarcity. can confirm.

On the other hand, in the step of reconstructing the input learning model, if the change of the identified neuron has a predetermined value, the identified neuron is duplicated to expand the input learning model, and the identified neuron is used to replace the existing value. It is possible to reconstruct the input learning model.

Meanwhile, the electronic device according to an embodiment of the present disclosure identifies the memory by storing a learning model composed of a plurality of neurons and a data set including a new task, and a neuron associated with the new task among the plurality of neurons Selective re-learning of parameters related to the new task for the neurons, and if the learning model in which the selective re-learning is performed has a predetermined loss value, the input of the learning model in which the selective re-learning is performed is dynamically expanded. And a processor to reconstruct the learning model.

In this case, the processor calculates a new parameter matrix such that the objective function having a loss function for the input learning model and a normalization term for scarcity is minimized, and uses the calculated new parameter matrix to neuron related to the new task. Can be identified.

On the other hand, the processor calculates a new parameter matrix using the data set for network parameters composed only of the identified neurons, and reflects the calculated new parameter matrix on the identified neurons of the learning model for selective re-learning. It can be done.

Meanwhile, when the learning model in which the selective re-learning is performed has a predetermined loss value, the processor adds a fixed number of neurons for each layer to the learning model in which the selective re-learning is performed, and adds the group using group scarcity. The input learning model may be reconstructed by removing unnecessary neurons from among the neurons.

In this case, the processor may identify unnecessary neurons among the added neurons by using an objective function having a loss function for the input learning model, a normalization term for sparsity, and a group normalization term for group sparsity.

On the other hand, in the step of reconstructing the input learning model, if the change of the identified neuron has a predetermined value, the identified neuron is duplicated to expand the input learning model, and the identified neuron is used to replace the existing value. It is possible to reconstruct the input learning model.

On the other hand, in a computer-readable recording medium including a program for executing a re-learning method of a learning model in the electronic device of the present disclosure, the re-learning method of the learning model comprises a learning model composed of a plurality of neurons, and a new Receiving a data set including a task, identifying a neuron associated with the new task among the plurality of neurons, selectively re-learning parameters related to the new task for the identified neuron, and selective re-learning And if the performed learning model has a predetermined loss value, reconstructing the input learning model by dynamically expanding the size of the learning model in which the selective re-learning is performed.

As described above, according to various embodiments of the present disclosure, that is, when additional learning is performed based on an existing pre-trained network model, the entire task unit network is not duplicated, but the corresponding task unit network is duplicated and divided. By learning only the additional part, it is possible to save the learning time and computation amount.

1 is a block diagram showing a simple configuration of an electronic device according to an embodiment of the present disclosure;
2 is a block diagram showing a specific configuration of an electronic device according to an embodiment of the present disclosure;
3 is a view for explaining a dynamic expansion method,
4 is a view for explaining the contents of the incremental learning in the dynamic extension network,
5 is a diagram showing an incremental learning algorithm in a dynamic extension network,
6 is a diagram showing an algorithm for selective re-learning,
7 is a diagram showing an algorithm for a dynamic extension method,
8 is a diagram showing an algorithm for network segmentation and replication,
9 is a diagram showing average performance per average task for each training model and each data set;
10 is a diagram showing the accuracy of the network size for each training model and each data set,
11 is a view showing the effect of selective retraining,
12 is a view showing a semantic drift experiment for the MNIST-Variation data set,
13 is a flowchart illustrating a learning model planetary method according to an embodiment of the present disclosure;
14 is a flowchart illustrating an incremental learning of a dynamic extension network according to an embodiment of the present disclosure,
15 is a flow chart for explaining the selective re-learning step of FIG. 14.

Terms used in this specification will be briefly described, and the present disclosure will be described in detail.

Terms used in the embodiments of the present disclosure, while considering the functions in the present disclosure, general terms that are currently widely used are selected, but this may vary according to the intention or precedent of a person skilled in the art or the appearance of new technologies. . Also, in certain cases, some terms are arbitrarily selected by the applicant, and in this case, their meanings will be described in detail in the description of the corresponding disclosure. Therefore, the terms used in the present disclosure should be defined based on the meaning of the terms and the contents of the present disclosure, not simply the names of the terms.

The embodiments of the present disclosure may apply various transformations and have various embodiments, and thus, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope of the specific embodiments, it should be understood to include all conversions, equivalents, or substitutes included in the scope of the disclosed ideas and techniques. In the description of the embodiments, when it is determined that the detailed description of the related known technology may obscure the subject matter, the detailed description is omitted.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms are only used to distinguish one component from other components.

Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, "includes." Or "composed." Terms such as intended to designate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, one or more other features or numbers, steps, operation, component, part, or It should be understood that the possibility of the presence or addition of these combinations is not excluded in advance.

In the exemplary embodiment of the present disclosure, the'module' or the'unit' performs at least one function or operation, and may be implemented in hardware or software, or a combination of hardware and software. In addition, a plurality of'modules' or a plurality of'units' may be integrated with at least one module, except for a'module' or'unit', which needs to be implemented with specific hardware, and may be implemented with at least one processor.

Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present disclosure pertains can easily carry out the embodiments. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in order to clearly describe the present disclosure in the drawings, parts irrelevant to the description are omitted, and like reference numerals are assigned to similar parts throughout the specification.

Hereinafter, the present disclosure will be described in more detail with reference to the drawings.

1 is a block diagram illustrating a simple configuration of an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 1, the electronic device 100 may include a memory 110 and a processor 120. Here, the electronic device 100 may be a PC, a notebook PC, or a server capable of data calculation.

The memory 110 stores a learning model composed of a plurality of neurons. Here, the learning model is a model trained using an artificial intelligence algorithm. The artificial intelligence algorithm may be a deep neural network (DNN), a deep convolution neural network, a residual network, or the like. The learning model may be composed of a plurality of layers, that is, hierarchically.

The memory 110 may store a set of learning data for re-learning the learning model, and may store data for classification or recognition using the learning model.

Also, the memory 110 may store a program necessary for re-learning a learning model or a learning model re-learned by the corresponding program.

The memory 110 is a storage medium and an external storage medium in the electronic device 100, for example, a removable disk including a USB memory, a storage medium connected to a host, a web server through a network, or the like. Can be implemented.

The processor 120 performs control for each component in the electronic device 100. Specifically, when a boot command is input from the user, the processor 120 may boot using an operating system stored in the memory 110.

The processor 120 may select a learning model to be re-learned through the manipulation input unit 140 to be described later, and receive various parameters for re-learning the selected learning model through the manipulation input unit 140. Here, various parameters received may be hyperparameters or the like.

When various information is input, the processor 120 identifies neurons related to a new task among a plurality of neurons. Specifically, the processor 120 calculates a new parameter matrix such that the objective function having a loss function for the input learning model and a normalization term for scarcity is minimized, and uses the calculated new parameter matrix to generate neurons related to the new task. Can be identified. Here, the objective function may be expressed as Equation 2 as a function having a loss function and a normalization term for scarcity. Details of the objective function will be described later with reference to FIG. 3.

In addition, the processor 120 selectively re-learns parameters related to the new task for the identified neuron. Specifically, the processor 120 calculates a new parameter matrix using a data set for a network parameter composed of only neurons identified to minimize the objective function, such as Equation 3, which will be described later, and trains the calculated new parameter matrix. Selective re-learning can be performed by reflecting the identified neurons.

In addition, the processor 120 may selectively reconstruct retrained learning data. Specifically, the processor 120 may reconstruct the input learning model by dynamically expanding the size of the learning model in which the selective re-learning is performed when the learning model in which the selective re-learning has a predetermined loss value.

More specifically, the processor 120 adds a fixed number of neurons per layer to the learning model in which the selective re-learning is performed, and uses group scarcity when the learning model in which the selective re-learning has a predetermined loss value. The input learning model can be reconstructed by removing unnecessary neurons from the added neurons. In this case, the processor 120 may identify unnecessary neurons among the added neurons by using an objective function such as Equation 4 having a loss function for the input learning model, a normalization term for sparseness, and a group normalization term for group scarcity. have.

Alternatively, the processor 120 calculates a change in the previously identified neuron by calculating Equation 5, which will be described later, and when the change in the identified neuron has a predetermined value, duplicates the identified neuron to expand the input learning model and , The identified neurons can be reconstructed by having the existing values.

The processor 120 may perform various processes such as vision recognition, speech recognition, and natural language processing using the re-learned learning model. Specifically, if the learning model is related to image classification, the processor 120 may classify the input image using the retrained learning model and the input image.

As described above, the electronic device 100 according to the present embodiment re-learns only neurons that are meaningful to the task, not the entire weight, when extending the task, that is, when performing re-learning, so that efficient re-learning is possible. . In addition, re-learning only meaningful neurons can prevent semantic metastasis.

On the other hand, in the above, only a simple configuration constituting the electronic device is illustrated and described, but in the implementation, various configurations may be additionally provided. This will be described below with reference to FIG. 2.

2 is a block diagram illustrating a specific configuration of an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 2, the electronic device 100 may include a memory 110, a processor 120, a communication unit 130, a display 140, and an operation input unit 150.

The operation of the memory 110 and the processor 120 has been described with reference to FIG. 1, and duplicate description is omitted.

The communication unit 130 may be connected to another electronic device and receive a learning model and/or learning data from another electronic device. Also, the communication unit 130 may receive information requiring classification or evaluation, and provide classification and evaluation results to other electronic devices.

The communication unit 130 is formed to connect the electronic device 100 with an external device, and is connected to a terminal device through a local area network (LAN) and an internet network, as well as a universal serial bus (USB). It is also possible to connect via a port or a wireless communication (eg, WiFi 802.11a/b/g/n, NFC, Bluetooth) port.

The display 140 displays various information provided by the electronic device 100. Specifically, the display 140 may display a user interface window for selecting various functions provided by the electronic device 100. Specifically, the corresponding user interface window may include an item for selecting a learning model to perform re-learning or inputting parameters to be used in the re-learning process.

The display 140 may be a monitor such as an LCD, a CRT, or an OLED, or may be implemented as a touch screen capable of simultaneously performing the functions of the manipulation input unit 150, which will be described later.

In addition, the display 140 may display information about test results using a learning model. For example, if the corresponding learning model is a model for classifying images, the display 140 may display a classification result for the input image.

The manipulation input unit 150 may receive, from a user, learning data to perform re-learning and various parameters to be performed in the re-learning process.

The manipulation input unit 150 may be implemented with a plurality of buttons, keyboard, mouse, or the like, and may also be implemented with a touch screen capable of simultaneously performing the functions of the display 140 described above.

On the other hand, in the illustration and description of FIGS. 1 and 2, it has been described that the electronic device 100 includes only one processor, but the electronic device may include a plurality of processors, and a general CPU as well as a GPU may be utilized. have. Specifically, the above-described optimization operation may be performed using a plurality of GPUs.

Hereinafter, a method of changing the learning model will be described in detail.

Lifelong learning is a field of prior learning, and the main purpose of lifelong learning is to leverage the knowledge of previous tasks to improve performance or to achieve faster convergence/learning speeds than in models for later tasks.

There are various approaches to solving this problem. However, in this disclosure, in order to utilize the capabilities of the deep neural network, lifelong learning is considered on deep learning. Fortunately, in deep learning, storing or transferring knowledge can be done in a simple way through the learned network weights. The learned weights can be provided as knowledge of existing tasks, and new tasks can be influential simply by sharing their weights.

Therefore, lifelong learning can be regarded as a special case of online learning or incremental learning in a deep neural network.

There are various ways to do incremental learning, the simplest is to gradually refine the network to new tasks by continuing to train the network with new learning data.

However, simply retraining the network can degrade performance in both new and old tasks. If the new task is significantly different from the previous task, for example, if the previous task is to classify the animal image and the new task is to separate the car image, learning the features of the previous task is not useful for learning the new task. not. At the same time, re-learning a new task deviates from the original task and is no longer an optimal task, negatively affecting the existing task.

For example, features that describe a zebra's stripe pattern can change features, and significantly change its meaning, such as changing it to a meaning for an adaptive classification task that classifies a striped t-shirt or fence.

Accordingly, we considered how to ensure that the sharing of knowledge through the network is beneficial for all tasks in online/incremental learning of deep neural networks. Recently, work suggests the use of normalization to prevent large parameter changes. However, these proposals need to find a good solution for new tasks, and there is a problem to prevent changes in parameters of old tasks.

Therefore, in the present disclosure, if necessary, the network is expanded in each task t in order to utilize the new task while changing the network size, and only the corresponding part of the previous learning network is changed. In this way, each task t can be used as a subnetwork different from the previous task, and can share a significant portion of the subnetwork associated with it.

Meanwhile, the following points should be considered in order to gradually set up deep learning through selective parameter sharing and dynamic layer expansion.

1) Achieve scalability and efficiency of learning: As the size of the network increases, the connection to a much larger network is established through subsequent tasks, so the learning cost per task increases gradually. Therefore, there is a need for a method to keep the computational overhead of retraining low.

2) Determining when to expand the network and how many neurons to add: If the existing network fully describes the new task, the network does not need to expand in size. Conversely, if the task is very different from the existing one, it is necessary to add many neurons. Therefore, the learning model needs to dynamically add only the necessary number of neurons.

3) There is a need to prevent semantic drift or catastrophic forgetting, where network performance degrades for early examples/tasks that deviate from the initial configuration. In this regard, it has a potential semantic transfer meaning, in part because it re-learns the network, makes it suitable for tasks learned later, and establishes connections to previous sub-networks to add new neurons that can negatively affect the previous task. A mechanism for preventing movement is required.

In order to solve this problem, the present disclosure proposes a new dimension of a network model along with an efficient and effective incremental learning algorithm. These algorithms are called Dynamically Expandable Networks (DEN).

In a lifelong learning scenario, DEN can efficiently learn how to predict new tasks by making the most of the network learned from all previous tasks, while dynamically increasing network size by adding or splitting neurons when needed. This algorithm is applicable to general networks including convolutional networks.

Hereinafter, a conventional lifelong learning method and a learning method according to the present embodiment will be described with reference to FIG. 3.

3 is a diagram for explaining various re-learning models.

Figure 3a shows a re-learning model, such as Elastic Weight Consolidation. This model re-learns the entire network learned for the previous task while performing normalization to prevent a large difference from the original model. The retrained units and weights are indicated by dotted lines, and the solid lines indicate fixed units and weights.

3B shows a non-training model such as a Progressive Network. This model extends the network for the new task (t) while maintaining the network weight for the existing task.

3C shows a learning model according to the present disclosure. The learning model according to the present disclosure selectively re-learns an existing network to expand the size if necessary, thereby dynamically determining an optimal size during learning.

Hereinafter, a dynamically scalable network incremental learning according to the present disclosure will be described.

)에 달려있다. Consider the incremental training problem of deep neural networks in a lifelong learning scenario where the number of unknown tasks with unknown distribution of learning data arrives sequentially in the model. Specifically, training a model for successive T tasks (t = 1, ..., t, ..., T, T is infinite). Here, at a specific point in time (t), the task is learning data (

).

)에 대한

를 갖는 바이너리 분류 태스크를 고려할 수 있다. 평생 학습 환경에서 t-1까지의 이전 훈련 데이터 세트가 현재 시간 t에서 사용 가능하지 않다는 것이 주요 태스크이다. 이전 작업에 대한 모델 파라미터만 액세스 가능하다. The method for generalization of a specific task is, for simplicity, input features (

)for

Consider a binary classification task with. The main task is that the previous training dataset from t-1 in the lifelong learning environment is not available at the current time t. Only model parameters for previous work are accessible.

를 학습하는 것을 목표로 한다. At time t, the lifelong learning agent solves the following equation to model parameters

Aim to learn.

은 태스크 특정 손실 함수이고,

는 태스크 t에 대한 파라미터이고,

는 모델

를 적절하게 시행하기 위한 정규화(예를 들어, 엘리먼트-와이즈(element-wise)

놈(norm) )이다. 그리고 L(l=1,2,…,L)은 레이어의 개수, D는 데이터,

:

레이어에서의 t에 대한 가중치 텐서(tensor)이다. here

Is the task specific loss function,

Is a parameter for task t,

The model

Normalization (e.g., element-wise) to properly enforce

It is a norm. And L(l=1,2,…,L) is the number of layers, D is the data,

:

The weight tensor for t in the layer.

To address these tasks in lifelong learning, the network makes the most of the knowledge gained from previous tasks and dynamically scales in size when the cumulative knowledge alone is not enough to describe the new task. Specifically, the network can be dynamically adjusted to find the optimal solution. In other words, when additional learning is performed based on the network model previously learned, instead of learning again, the network of each task unit is duplicated and divided to learn only the additional part, thereby learning and computing time. You can save.

Hereinafter, the gradual learning process will be described with reference to FIGS. 4 and 5.

FIG. 4 is a diagram for explaining contents of incremental learning in a dynamic extension network, and FIG. 5 is a diagram showing an incremental learning algorithm in the dynamic extension network.

4 and 5, first, selective re-learning is performed. Specifically, neurons associated with new tasks are identified and network parameters associated with the related tasks are selectively retrained. The specific operation of the selective re-learning will be described later with reference to FIG. 6.

And it dynamically expands the network. Specifically, if selective retraining does not achieve the desired loss below a set threshold, group-sparse normalization is used to remove all unnecessary nerves and expand the network size downward. The dynamic expansion operation will be described later with reference to FIG. 7.

를 계산하여 훈련 도중 원래 값에서 너무 많이 벗어난 유닛을 식별하고 복제한다. 네트워크의 분할 및 복제에 대한 구체적인 동작은 도 8을 참조하여 후술한다. Then, divide or duplicate the network. Specifically, DEN is a drift for each unit

Calculate and identify units that deviate too much from the original value during training. The detailed operation for the division and replication of the network will be described later with reference to FIG. 8.

Hereinafter, the selective re-learning operation will be described with reference to FIG. 6.

6 is a diagram showing an algorithm for selective re-learning.

정규화로 네트워크를 학습한다. 이에 따라, 각 뉴런은 아래의 레이어에서 단지 몇 개의 뉴런에만 연결된다. The simplest way to train a model for a series of tasks is to retrain the entire model each time a new task arrives. However, such retraining is expensive for deep neural networks. Therefore, it is desirable to perform selective re-learning of the model by re-learning only the weights affected by the new task. Accordingly, in the present disclosure, in order to accelerate the scarcity of the weight

Train the network with normalization. Accordingly, each neuron is connected to only a few neurons in the underlying layer.

은 네트워크의

번째 레이어를 나타내며,

은 레이어

에서의 네트워크 파라미터이고,

은

의 희소성을 위한 엘리먼트-와이즈(element-wise)

놈의 정규화 파라미터로, regularize를 더할 때 그 크기를 정하는데 이용된다. 합성곱 레이어에서, 필터 상에 (2,1)-놈을 적용하여 이전 레이어의 필터만 선택하였다. here,

Of the network

The second layer,

Silver layer

Is a network parameter in

silver

-Wise for element scarcity

This is the normalization parameter of the norm, which is used to determine its size when adding regularize. In the convolutional layer, only the filter of the previous layer was selected by applying (2,1)-nome on the filter.

-정규화는 뉴런 간의 연결성이 희박하므로 서브 네트워크 연결 신규 태스크에 집중할 수 있다면 계산 오버 헤드를 크게 줄일 수 있다. 이를 위해 새로운 태스크가 모델에 도착하면 다음과 같은 수학식 2를 통해 신경망의 최상위 숨겨진 단위를 사용하여 태스크 t를 예측하기 위한 스파스(sparse) 선형 모델에 적합하게 만든다.

-Normalization lacks connectivity between neurons, so if you can focus on new tasks connecting to a subnetwork, you can greatly reduce the computational overhead. To this end, when a new task arrives at the model, the following equation (2) is used to fit the sparse linear model to predict the task t using the highest hidden unit of the neural network.

은

을 제외한 다른 모든 파라미터 집합을 나타낸다. 즉, 이 최적화를 해결하여 출력부(

)와 레이어 L-1에서 히든 레이어 간의 연결을 얻는다(레이어 L-1까지 다른 모든 파라미터를

으로 고정). 이러한 레이어에서의 스파스 연결을 구축하여, 학습 영향을 받는 네트워크 내의 단위 및 가중치를 식별할 수 있게 된다. 특히, 선택한 노드에서 시작하여 네트워크에서 너비 우선 탐색을 수행하여 경로가 있는 모든 유닛 (및 입력 특징)을 식별할 수 있다. 그 다음, 선택된 서브 네트워크(S)(

)의 가중치들만을 학습할 수 있다. here

silver

Except for all other parameter sets. In other words, by solving this optimization,

) And get the connection between layer L-1 and hidden layer (all other parameters up to layer L-1

Fixed with). By establishing sparse connections at these layers, it is possible to identify units and weights within the network that are affected by learning. In particular, it is possible to identify all units (and input features) that have a path by performing a width-first search on the network starting at the selected node. Then, the selected sub-network (S) (

) Can learn only the weights.

정규화를 사용한다. 이러한 선택적 재학습은 선택되지 않은 뉴런은 재학습의 영향을 받지 않기 때문에, 계산 오버헤드를 낮추고 부정적인 전이를 회피하는데 도움이된다. Element-wise because the sparse connection has already been established

Use normalization. This selective relearning helps reduce computational overhead and avoid negative metastasis, as unselected neurons are not affected by relearning.

를 얻는다. Referring to FIG. 6, first, l and S are initialized, and Equation 2 is used.

Get

의 i와 O_t 사이의 가중치가 0이 아니면, S에 뉴런 i를 추가한다. And the weight for task t in layer L

If the weight between i and O _t is not 0, neuron i is added to S.

인 뉴런 S가 존재할 때 S에 뉴런 i를 추가한다. Also,

Neuron i is added to S when phosphorus neuron S is present.

를 얻는다. And using Equation 3

Get

7 is a diagram showing an algorithm for a dynamic extension method.

Selective retraining is sufficient to perform a new task if the new task is highly relevant to the previous task, or if the partially gained knowledge gained from each task is sufficient to describe the new task.

However, if the learned features are not sufficient to represent a new task, additional neurons must be introduced into the network to describe the functionality required for the new task. On the other hand, it is not desirable in terms of performance and network size to add a certain number of units without requiring task difficulty or to request an iterative forward pass.

와 같이 확장할 수 있다. 그리고 새로운 태스크와 이전 태스크 간의 관련성에 따라 항상

단위를 추가하기를 원하지 않기 때문에 수학식 4와 같이 추가된 파라미터에 그룹 희소 정규화를 수행한다. Therefore, in this disclosure, group sparse normalization is used to dynamically determine how many neurons to add to each layer. Assuming that the layer of the network is expanded to a certain number (k), the two parameter matrices

Can be expanded with And depending on the relationship between the new task and the old task,

Since we do not want to add units, we perform group sparse normalization on the added parameters as in Equation 4.

는 각 뉴런에 대한 유입 가중치로 정의 된 그룹이다. 합성곱 레이어는, 각 그룹을 각 합성곱 필터의 활성화 맵으로 정의하였다. 이러한 그룹 희소 정규화는 전체 네트워크의 적정한 뉴런 수를 찾으며, 본 개시에서는 일부 네트워크에서만 이를 적용한다. 도 7은 이러한 알고리즘을 나타낸다. here

Is a group defined by the influx weight for each neuron. The convolution layer defined each group as an activation map of each convolution filter. This group sparse normalization finds the proper number of neurons in the entire network, and this disclosure applies only to some networks. 7 shows this algorithm.

)이 기설정된 손실 값(

)보다 크면, 네트워크 크기를 고정적으로 확장하고, 그룹 희소성을 통해 확장된 뉴런 중 불필요한 뉴런을 제거하는 동작을 반복적으로 수행할 수 있다. Referring to FIG. 7, the target loss value in selective re-learning (

) Is a preset loss value (

If it is greater than ), the network size can be fixedly expanded and an operation of removing unnecessary neurons among the expanded neurons through group scarcity can be repeatedly performed.

8 is a diagram showing an algorithm for network segmentation and replication.

정규화를 사용하여 원래 값으로부터 너무 많이 벗어나지 않도록 파라미터를 정규화하는 것이다. An important task in lifelong learning is semantic transition, fatal forgetting, in which the initial task is forgotten by the later learned task and the performance is reduced as a result. An easy way to prevent semantic transfer

The normalization is to normalize the parameters so that they do not deviate too much from the original value.

는 현재 태스크이고,

은 l 태스크

에 대해 훈련된 네트워크의 가중치 텐서(tensor)이다. 그리고,

는 정규화 파라미터이다. here

Is the current task,

Silver l task

Is the weighted tensor of the network trained for. And,

Is the normalization parameter.

정규화는 주어진

정도로 솔루션

가

에 가깝도록 강제하는 것이다.

가 작다면, 네트워크는 오래된 태스크를 잊어 버리는 동안 새로운 태스크를 더 반영하도록 배울 것이고,

가 크다면,

는 이전 태스크에서 배운 지식을 가능한 한 많이 보존하려고 할 것이다.

Normalization given

Degree solution

end

Is to force it closer to.

If is small, the network will learn to reflect more new tasks while forgetting old tasks,

If is large,

Will try to preserve as much of the knowledge learned in the previous task as possible.

정규화를 대신하여, 피셔 정보(Fisher information)로 각 요소를 가중하는 방법도 가능하다. 그럼에도, 태스크의 수가 매우 크거나, 나중의 태스크가 의미적으로 이전 태스크와 차이가 있다면, 이전 및 새로운 태스크 각각에 대한 좋은 솔루션을 찾기에 어렵게 된다.

Instead of normalization, it is also possible to weight each element with Fisher information. Nevertheless, if the number of tasks is very large, or if later tasks are semantically different from previous tasks, it will be difficult to find a good solution for each of the old and new tasks.

거리로 각 숨겨진 레이어에서의 의미적 전이의 양(

)을 측정한다. In this case, it is a good idea to separate the two different optimized tasks. By performing Equation 5, between t-1 and the weight coming from t

The amount of semantic transition in each hidden layer by distance (

).

라면, 우리는 훈련 중에 특징의 의미가 크게 변했다고 생각할 수 있는바, 이 뉴런을 두 개의 복사본으로 나눈다. 이러한 방식은 모든 히든 뉴런에 대해서 병렬적으로 수행될 수 있다. 뉴런을 복제한 이후에, 뉴런은 수학식 5를 이용하여 가중치가 다시 학습될 수 있다. 그러나 초기화 학습으로부터 합리적인 파라미터 초기화에 기초하여 두 번째 학습은 빠르게 수행될 수 있다. 도 8은 이러한 알고리즘을 나타낸다.

Ramen, we can think of a significant change in the meaning of the feature during training, so we divide this neuron into two copies. This method can be performed in parallel for all hidden neurons. After duplicating the neuron, the neuron can be re-trained using Equation (5). However, the second learning can be quickly performed based on reasonable parameter initialization from the initialization learning. 8 shows this algorithm.

Referring to FIG. 8, after the previous dynamic network expansion operation, if the change in the updated neuron is higher than the reference point, the corresponding neurons (B) are replicated to expand the network, and the updated neuron (B) is moved to the previous step (A). You can return.

를 설정하여 새로 추가된 유닛 j를 타임스탬프 처리하여 새로운 개념의 도입으로 인한 의미적 편차를 효과적으로 방지할 수 있다.On the other hand, in both the network expansion and network segmentation procedures, the learning step t is stored when added to the network.

By setting, it is possible to effectively prevent the semantic deviation caused by the introduction of a new concept by timestamping the newly added unit j.

Specifically, in inference, each task may use only the parameters introduced up to step t so as not to use the new hidden unit added to the learning process in the previous step. This is more flexible than modifying the weights learned until each stage of learning. Early learning tasks can benefit from later learning tasks, but they are advantageous because learning is better, but not separated.

Hereinafter, the effect of the selective re-learning method according to the present disclosure will be described with reference to FIGS. 9 to 12.

Hereinafter, a comparison target of an experimental condition applied to the selective re-learning method according to the present disclosure and a setting state therefor will be described first.

1) DNN-STL (Singl-Task Learning): Basic deep neural network trained individually for each task.

2) DNN-MTL (Multi-Task Learning): Basic deep neural network learned for all tasks at once.

로 초기화되고 SGD로 계속 훈련되며,

에 대해서`

정규화된 SGD로 훈련된다. 3) DNN-L2: Basic deep neural network, where Wt in each task t

Initialized to and continue training with SGD,

About `

Trained with normalized SGD.

4) DNN-EWC: It is a deep network trained with elastic weight reinforcement for elaboration.

5) DNN- Progressive: The network weight for each task is a fixed progressive network for tasks arriving later.

6) DEN. It is a re-learning method of the present disclosure.

Basic network settings.

의 정규화를 위한

로 0.1을 사용하였다. 그리고 수학식 2의 희소성

는

로 설정하였다. 그리고 각 태스크에 추가되는 유닛의 수로 k=10을 사용하였으며, 수학식 4에서의 그룹 희소성 정규화 항의

은 0.01을 설정하였으며, 네트워크 분할 및 복제에서의 l2 거리 임계치로

을 설정하였다. 1) Feed forward network: 312-128 two-layer networks with ReLU activation were used,

For normalization of

0.1 was used as the furnace. And the scarcity of equation (2)

The

Was set to. Also, k=10 was used as the number of units added to each task, and the group sparse normalization protest in Equation 4 was used.

Is set to 0.01, and as the l2 distance threshold in network segmentation and replication

Was set.

로 0.01을 사용하였으며, 희소성을 위하여

를 사용하였으며, 그룹 희소성을 위하여

을 사용하였다. 그리고 네트워크 분할 및 복제를 위한

에 대해서

을 각각 합성곱 레이어 및 완전 연결 레이어에 설정하였다. 2) Convolutional network: LeNet with 2 convolutional layers and 3 fully connected layers was used. Where l2 is normalized

0.01 was used, and for scarcity

And for group scarcity

Was used. And for network segmentation and replication

about

Was set to the convolutional layer and the fully connected layer, respectively.

All models and algorithms were implemented using the Tensorflow library. Hereinafter, the used data sets will be described.

1) MNIST-Variation. This data set consists of 62000 images of 0 to 9 handwritten digits.

2) CIFAR-10. This data set consists of 60000 images of common objects, including vehicles and animals. Each class has 6000 32x32 images, with 5000 images used for training and the rest for testing.

3) AWE. This data set consists of 30475 images of 50 animals.

Hereinafter, quantitative evaluation using the above-described model and data set will be described. Specifically, the model was verified for both prediction accuracy and efficiency.

9 is a diagram showing average performance per average task for each training model and each data set. Specifically, FIG. 9A is a diagram showing the number of tasks by model for the MNIST-variation data set, FIG. 9B is a diagram showing the number of tasks by model for the CIFAR-10 data set, and FIG. 9C is by model for the AWA data set It is a diagram showing the number of tasks.

9, it can be seen that CNN-MTL and DNN-STL have the best performance in CIFAR-10 and AWA, respectively. It is expected to perform well because it has been trained to perform all tasks at once or is best suited for each task.

On the other hand, other models can be trained online to trigger semantic drift. If the number of tasks is small, MTL works best in knowledge sharing through multi-task learning, but if the number of tasks is large, STL works better because of the larger learning size than MTL.

It can be seen that DEN has almost the same performance as other models, and surpasses the performance of others in data sets such as MNIST-deformation.

And it can be seen that retraining models combined with normalizations such as L2 and EWC perform poorly in all data sets.

Progressive networks work better than both, but performance is significantly worse in AWA. This is because a large number of tasks can make it difficult to find an appropriate network size. If the network is too small, it does not have enough learning ability to represent new tasks, and if it is too large, it tends to become excessive.

Referring to FIG. 9, when DEN according to the present disclosure is a lifelong learning method, when learning a new concept, it can be confirmed that the performance close to independent learning (STL) or simultaneous learning (MTL) is gathered or shows a result that is inferior. have.

10 is a diagram showing a performance difference according to a network size. Specifically, FIG. 10A is a diagram showing the network size per training model in the MNIST-Variation data set, FIG. 10B is a diagram showing the network size per training model in the CIFAR-10 data set, and FIG. 10C is in the AWA data set This is a diagram showing the network size of each learning model.

Referring to FIG. 10, it can be seen that the DEN according to the present disclosure has much better performance with a much smaller number of parameters than a progressive network, or has a much better performance using a similar number of parameters. DEN can also achieve the same level of performance as STL using sizes of 18.0%, 19.1% and 11:9% respectively in MNIST-Variation, Cigar-10 and AWA.

This is the main advantage of DEN being able to dynamically find the optimal size, as you learn a very compact model in MNIST-Variation while learning a substantially large network in AWA. DEN fine-tuning for all tasks (DEN-Finetune) yields the best performance model for all datasets. DEN is not only useful for lifelong learning, but can also be used to estimate network size when all tasks are present. Can be used in the first place.

In addition, it can be seen that when re-learning using the size of the learned network, it shows the highest performance in all data sets. Through this, it can be confirmed that the learning model according to the present disclosure can be utilized not only for lifelong learning scenarios, but also for estimating the size of a learning network when learning a plurality of tasks at once.

11 is a view showing the effect of selective retraining. Specifically, FIG. 11A is a view showing processing time by size, FIG. 11B is a view showing AUC, and FIG. 11C is a view showing expansion performance.

We examined how efficient and effective selective learning is by measuring the learning speed and accuracy for each task in the MNIST-Variation data set. To this end, we compared a model without network extension called DNN-Selective to DNN-L2.

11A and 11B, DNN-Selective achieves a slight decrease in accuracy compared to the entire network, while achieving a speed improvement of 6:51 times, confirming that selective re-learning is much more efficient than the entire re-learning of the network. have.

Referring to Figure 11c, it can be seen that DNN-L1 also performs better than the entire network trained without sparse normalization. This is because sparse networks eliminate much of the computation for forward and reverse passes. However, it is not efficient with DNN selection.

Hereinafter, the network expansion effect will be described with reference to FIG. 12.

12 is a view showing a semantic drift experiment for the MNIST-Variation data set. Specifically, FIG. 12A is a learning time for each task measured in CPU time, FIG. 12B is an average performance per task of the compared model, and FIG. 12C is an extended performance.

Selective retraining and layer expansion are performed, but the model's transformation and network expansion efficiency are compared with the model in which the network is not divided.

DNN-Dynamic achieves significantly better average AUCs than all models, including DNN-Constant, while significantly increasing the size of the network than DNN-Constant. This may be because the number of parameters is not only beneficial in terms of learning efficiency, but also prevents the model from being over-applied.

Referring to FIG. 12, it shows how the performance of the model changes at each training step t for tasks t = 1, t = 4 and t = 7.

It can be seen that DNN-L2 prevents semantic drift of the model trained in the early stages, but gradually deteriorates in subsequent tasks (t = 4, 7).

This is a phenomenon common to DNN-EWC. Based on these results, each task may require significantly different functionality from the previous task. MNIST-Variation data set. DNN-Progressive shows that there is no semantic drift for the previous task, and is expected because the previous task is not re-entered with parameters.

On the other hand, Timestamping is more effective than DNN-Progressive in later tasks with a slight performance degradation over time. Lastly, the time-stamped DEN's entire model shows no signs of deterioration in the learning stage, but it can be confirmed that it is superior to DNN-progressive.

These results indicate that DEN is very effective in preventing semantic drift.

When learning in the lifelong learning scenario progresses, the performance change for each task shows the tendency of performance decrease due to semantic drift as the number of objects to learn increases in the case of the existing technology (EWC). In the case of progressive, the existing technology, since the existing learned network is fixed and the network is linearly expanded whenever the number of objects to be learned increases, the performance is maintained, but the problem that the calculation time is exponentially increased due to the expansion of the network cannot be solved. . Since this technology is designed to maintain constant performance through a series of processes such as selective expansion, separation, and re-learning of the network, it is possible to efficiently expand the network, thereby minimizing the increase in computation time.

13 is a flowchart illustrating a re-learning method of a learning model according to an embodiment of the present disclosure.

A data set including a learning model composed of a plurality of neurons and a new task is received (S1310).

Among the plurality of neurons, neurons associated with a new task are identified, and parameters associated with the new task are selectively re-learned with respect to the identified neurons (S1320). Specifically, a new parameter matrix is calculated such that an objective function having a loss function for the input learning model and a normalization term for scarcity is minimized, and the new parameter matrix is used to identify neurons associated with the new task. In addition, a new parameter matrix may be calculated using a data set for network parameters composed of only the identified neurons, and selective re-learning may be performed by reflecting the calculated new parameter matrix in the identified neurons of the learning model.

Then, when the learning model in which the selective re-learning is performed has a predetermined loss value, the input learning model is reconstructed by dynamically expanding the size of the learning model in which the selective re-learning is performed (S1330). Specifically, when a learning model in which selective re-learning is performed has a predetermined loss value, a fixed number of neurons are added to each learning model in which selective re-learning is performed, and unnecessary neurons among neurons added using group sparsity By removing, you can reconstruct the input learning model.

Alternatively, if the change of the identified neuron has a predetermined value, the inputted learning model may be expanded by replicating the identified neuron, and the inputted learning model may be reconstructed by allowing the identified neuron to have an existing value.

Therefore, in the re-learning method of the learning model according to the present embodiment, when the task is extended, that is, when re-learning, only the neurons meaningful to the task are re-learned, not the overall weight, so efficient re-learning is possible . In addition, re-learning only meaningful neurons can prevent semantic metastasis. The re-learning method of the learning model shown in FIG. 13 may be executed on the electronic device having the configuration of FIG. 1 or 2, and may also be performed on the electronic device having other configurations.

In addition, the re-learning method of the learning model as described above may be implemented as a program including an executable algorithm that can be executed on a computer, and the above-described program is stored in a non-transitory computer readable medium. Can be provided.

The non-transitory readable medium means a medium that stores data semi-permanently and that can be read by a device, rather than a medium that stores data for a short time, such as registers, caches, and memory. Specifically, programs for performing the various methods described above may be stored and provided on a non-transitory readable medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, and ROM.

14 is a flowchart illustrating incremental learning of a dynamic extension network according to an embodiment of the present disclosure.

Referring to FIG. 14, first, t is initialized to 1.

Then, if t is less than T (S1410-Y) and 1 (S1420-Y), W ¹ is learned using Equation 1 (S1425).

Then, selective re-learning is performed (S1430). Specifically, neurons associated with a new task may be identified, and network parameters related to the related tasks may be selectively retrained. The specific operation of the selective re-learning will be described later with reference to FIG. 15.

Then, it is checked whether the loss value of the learning model in which the selective re-learning is performed has a predetermined loss value (S1435).

And if the loss value of the training model has a predetermined loss value (S1435-Y), if selective retraining does not achieve the desired loss below the set threshold, the network size is top-down while removing all unnecessary nerves using group-sparse normalization. Expand to (S1440). The dynamic expansion operation will be described later with reference to FIG. 16.

를 계산하여 훈련 도중 원래 값에서 너무 많이 벗어난 유닛을 식별하고 복제할 수 있다. 네트워크의 분할 및 복제에 대한 구체적인 동작은 도 17을 참조하여 후술한다. Then, if the change of the identified neuron has a predetermined value, the network is divided or duplicated (S1445). Specifically, DEN is a drift for each unit

You can calculate and identify units that deviate too much from the original value during training. The detailed operation for the division and replication of the network will be described later with reference to FIG. 17.

Thereafter, the t value is increased (S1450), and the above-described steps are repeated until T is reached.

Therefore, in the incremental learning according to the present embodiment, when extending a task, that is, when performing re-learning, only the neurons meaningful to the task are re-learned rather than the overall weight, and thus efficient re-learning is possible. In addition, re-learning only meaningful neurons can prevent semantic metastasis. The incremental learning as shown in FIG. 14 may be executed on the electronic device having the configuration of FIG. 1 or 2, or may be performed on the electronic device having other configurations.

In addition, the incremental learning as described above may be implemented as a program including an executable algorithm that can be executed on a computer, and the above-described program can be stored and provided in a non-transitory computer readable medium. have.

15 is a flowchart illustrating the selective re-learning step of FIG. 14.

를 계산한다(S1510). Referring to FIG. 15, l and S are initialized, and Equation 2 described above is used.

Calculate (S1510).

의 i와 O_t 사이의 가중치가 0이 아니면(S1520), S(서브네트워크)에 해당 뉴런 i를 추가한다(S1530). And the weight for task t in layer L

If the weight between i and O _t is not 0 (S1520), the corresponding neuron i is added to S (subnetwork) (S1530).

인 뉴런 S가 존재할 때(S1550), S에 뉴런 i를 추가한다(S1560). After that, the layer is larger than 0 (S1540),

When phosphorous neuron S is present (S1550), neuron i is added to S (S1560).

Thereafter, the layer is moved to the upper layer (S1570), and the operation of searching for neurons satisfying the above-described conditions 1 and 2 is repeatedly performed.

를 얻는다. By performing this process on a layer-by-layer basis, when identification of neurons requiring re-learning for all layers is completed (S1550-N), using Equation 3

Get

Accordingly, the selective re-learning according to the present exemplary embodiment re-learns only neurons that are meaningful to the task, not the overall weight when extending the task, that is, when performing re-learning, so efficient re-learning is possible. In addition, re-learning only meaningful neurons can prevent semantic metastasis. The incremental learning as shown in FIG. 15 may be executed on the electronic device having the configuration of FIG. 1 or 2, or may be performed on the electronic device having other configurations.

In addition, the selective re-learning as described above may be implemented as a program including an executable algorithm that can be executed on a computer, and the above-described program is stored and provided in a non-transitory computer readable medium. Can.

Hereinafter, the dynamic network expansion step of FIG. 14 will be described.

Calculate the target loss value in the previous selective relearning.

If the calculated loss value has a loss value greater than the reference point, the network size can be fixedly extended. Specifically, h ^N can be added to all layers.

Subsequently, Equation 4 is calculated, and if the current layer is not 0, an operation of removing unnecessary neurons from the extended neurons is repeatedly performed.

Then, the layer may be moved upward, and the above-described calculation operation may be repeated to optimize the neuron.

Therefore, the dynamic network expansion according to the present embodiment may dynamically expand the network if necessary in order to reflect more accurate new tasks. In addition, it is possible to optimize the network expansion by performing an operation of deleting unnecessary neurons rather than adding neurons to all layers in a batch. The dynamic network extension may be performed on the electronic device having the configuration of FIG. 1 or 2, or may be performed on the electronic device having other configurations.

In addition, the dynamic network extension as described above may be implemented as a program including executable algorithms executable on a computer, and the above-described program may be stored and provided in a non-transitory computer readable medium. Can.

Below is a flow chart for explaining the network separation and replication steps of FIG. 14.

After the previous dynamic network expansion operation, W ^t _L,K is calculated.

Then, the amount of change (cement draft) is calculated for all hidden neurons, and it is determined whether the calculated change amount of the neuron is higher than the reference point.

)보다 높으면, 해당 뉴런들(B)을 복제하여 네트워크를 확장하고, 업데이트된 뉴런(B)을 이전 단계(A)로 복귀할 수 있다. As a result of the judgment, the amount of change in the calculated neuron is the reference point (

), the network can be expanded by replicating the neurons (B), and the updated neurons (B) can be returned to the previous step (A).

And the above-described verification operation is repeatedly performed for all hidden neurons.

Therefore, in the network separation and replication method according to the present embodiment, when a change in neurons is greater than a reference value, separating and replicating a network can prevent semantic transfer. The network separation and duplication method may be executed on the electronic device having the configuration of FIG. 1 or 2, and may also be performed on the electronic device having other configurations.

In addition, the network separation and replication method as described above may be implemented as a program including an executable algorithm that can be executed on a computer, and the above-described program is stored in a non-transitory computer readable medium. Can be provided.

In addition, although the preferred embodiments of the present disclosure have been described and described above, the present disclosure is not limited to the specific embodiments described above, and the technical field to which the present disclosure belongs without departing from the gist of the present disclosure claimed in the claims. In addition, various modifications can be implemented by those having ordinary knowledge in the art, and these modifications should not be individually understood from the technical idea or prospect of the present disclosure.

100: electronic device 110: memory
120: processor 130: communication interface
140: display 150: operation input

Claims (13)

In the re-learning method of the learning model in the electronic device,
Receiving, from the electronic device, a data set including a learning model composed of a plurality of neurons and a new task;
Identifying, in the electronic device, a neuron associated with the new task among the plurality of neurons, and selectively re-learning parameters related to the new task with respect to the identified neuron; And
In the electronic device, if the learning model in which the selective re-learning is performed has a predetermined loss value, reconstructing the input learning model by dynamically expanding the size of the learning model in which the selective re-learning is performed. How to retrain the learning model. According to claim 1,
The selective re-learning step,
A new learning model matrix is calculated such that the objective function having a loss function for the input learning model and a normalization term for scarcity is minimized, and the learning model re-identifying a neuron related to the new task is calculated using the calculated new parameter matrix. Learning method. According to claim 1,
The selective re-learning step,
A learning model that performs a selective re-learning by performing a new re-learning by calculating a new parameter matrix using the data set for the network parameters composed of only the identified neurons and reflecting the calculated new parameter matrix to the identified neurons of the learning model. Learning method. According to claim 1,
Reconstructing the input learning model,
If the learning model in which the selective re-learning is performed has a predetermined loss value, a fixed number of neurons is added to each layer in the learning model in which the selective re-learning is performed, and unnecessary neurons among the added neurons are used by using group sparsity. A learning model re-learning method to remove and reconstruct the input learning model. According to claim 4,
Reconstructing the input learning model,
A learning model re-learning method for identifying unnecessary neurons among the added neurons by using an objective function having a loss function for the input learning model, a normalization term for scarcity, and a group normalization term for group scarcity. According to claim 1,
Reconstructing the input learning model,
If the change of the identified neuron has a predetermined value, the learning model is reconstructed by replicating the identified neuron to expand the input learning model, and allowing the identified neuron to have an existing value to reconstruct the input learning model. How to relearn. In the electronic device,
A memory in which a data set including a learning model composed of a plurality of neurons and a new task is stored; And
Among the plurality of neurons, a neuron associated with the new task is identified to selectively relearn parameters related to the new task with respect to the identified neuron, and when the learning model in which selective relearning is performed has a predetermined loss value, the selective regeneration is performed. And a processor that dynamically expands a size of a learning model in which learning is performed to reconstruct a learning model stored in the memory. The method of claim 7,
The processor,
An electronic device that calculates a new parameter matrix so that an objective function having a loss function for the learning model stored in the memory and a normalization term for scarcity is minimized, and uses the calculated new parameter matrix to identify neurons related to the new task . The method of claim 7,
The processor,
An electronic device that performs a selective re-learning by calculating a new parameter matrix using the data set for a network parameter composed only of the identified neurons, and reflecting the calculated new parameter matrix to the identified neurons of the learning model. The method of claim 7,
The processor,
If the learning model in which the selective re-learning is performed has a predetermined loss value, a fixed number of neurons is added to each layer in the learning model in which the selective re-learning is performed, and unnecessary neurons among the added neurons are used by using group sparsity. An electronic device that removes and reconstructs the input learning model. The method of claim 10,
The processor,
An electronic device that identifies unnecessary neurons among the added neurons by using an objective function having a loss function for a learning model stored in the memory, a normalization term for sparseness, and a group normalization term for group scarcity. The method of claim 7,
The processor,
When the change of the identified neuron has a predetermined value, the former duplicates the identified neuron to expand the learning model stored in the memory, and the identified neuron has an existing value to reconstruct the input learning model. Device. A computer-readable recording medium comprising a program for executing a method of re-learning a learning model in an electronic device, comprising:
The re-learning method of the learning model,
Receiving a data set including a learning model composed of a plurality of neurons and a new task;
Identifying a neuron associated with the new task among the plurality of neurons, and selectively re-learning parameters associated with the new task for the identified neuron; And
And reconstructing the input learning model by dynamically expanding the size of the learning model in which the selective re-learning is performed, when the learning model in which the selective re-learning is performed has a predetermined loss value. .

Images