KR102097724B1 - Method and apparatus for mitigating the catastrophic forgetting problem in cnn for image recognition - Google Patents

Abstract

Disclosed are a method for mitigating a worst oblivion phenomenon in a CNN for image recognition and a device thereof. According to an embodiment of the present invention, the method for mitigating a worst oblivion phenomenon in a CNN for image recognition comprises the steps of: sampling data to be learned for a task to be learned through a prediction process; and performing learning by using a new sampled task. In the step of performing learning by using the new sampled task, as the new task is delivered, a weight set and a fisher information set can be updated by calculating gradients and fisher information.

Description

METHOD AND APPARATUS FOR MITIGATING THE CATASTROPHIC FORGETTING PROBLEM IN CNN FOR IMAGE RECOGNITION}

The following embodiments relate to a method and an apparatus for alleviating the worst forgetting phenomenon in a CNN for image recognition, and more specifically, in order to alleviate the worst forgetting problem (CFP), the characteristics of the new image are significantly different from those previously learned. I am concerned with a method and apparatus for alleviating the worst forgetting phenomenon in a CNN for image recognition in which learning is performed using only images.

As recent deep learning-related research has become active, image classification, speech recognition, face identification, semantic segmentation, visual question answering (VQA), CNN (Convolutional Neural Networks) is used in various fields such as Atari games. As a typical case in the medical imaging field, fine-tuning can detect 14 pathologies including pneumonia.

However, it has been reported that neural networks, which are mainly used for deep learning, show a characteristic of losing information on existing data when learning a new data set, which has not been resolved to date. This is called the Catastrophic Forgetting Problem (CFP), and it occurs the same in CNN. Abraham claimed that the cause of the CFP was the stability-plasticity dilemma. Stability refers to the stability of the weight, that is, the degree to which the weight value does not change significantly and converges slowly according to the incoming input value. Plasticity is variability, which means how fast the weight fits according to the input value. The higher the stability, the better the ability to predict the already learned data, and the higher the plasticity, the greater the learning ability for new materials to learn in the future. Because the two characteristics are in trade-off, the worst forgetting problem (CFP) has occurred and research has been conducted to solve it.

1 is a view for explaining the occurrence of the worst forgetting problem according to an embodiment.

Referring to (a) of FIG. 1, it indicates the occurrence of the worst forgetting problem, and since the weight is adjusted for each data set, the classification ability for the previously learned data set is lost. At this time, variability and stability are determined according to the speed of updating the weight, and the higher the variability, the greater the CFP.

Referring to (b) of FIG. 1, it shows a solution of a random brute force attack, and in order to avoid the worst forgetting problem (CFP), a new data set and all image sets already learned are set as one set. Together, you have to go on learning again.

In this way, the simplest way to avoid the worst forgetting problem (CFP) is to discard the existing weight and generate a new weight when a new image set is input. Retrain all images, including new ones, without using the old weights the network learned.

However, in this method, as the number of tasks is increased as the number of images is added, the learning time is doubled, and the multiple is gradually increased. In addition, it is inefficient in terms of time and memory because new learning is conducted independently each time.

F(2017) , "On quadratic penalties in elastic weight consolidation", arXiv:1712.03847.

F (2017), "On quadratic penalties in elastic weight consolidation", arXiv: 1712.03847.

Embodiments describe a method and apparatus for alleviating the worst forgetting phenomenon in a CNN for image recognition, and more specifically, an image in which characteristics are significantly different from those previously learned among new images to alleviate the worst forgetting problem (CFP). Provides a technique to progress learning using only.

The embodiments further propose a predictable Elastic Weight Consolidation (EWC) that samples and trains only images that are difficult to classify among the images to be trained, thereby reducing the size of the task to be trained, and repeating this process for all tasks. An object of the present invention is to provide a method and apparatus for alleviating the worst oblivion phenomenon in a CNN for image recognition that can perform efficient learning.

A method for alleviating the worst forgetting phenomenon in a CNN for image recognition according to an embodiment may include sampling data to be learned about a task to be learned through a prediction process; And progressing learning using the sampled new task, and progressing learning using the sampled new task, as the new task is transmitted, gradient and fisher information. Calculate to update the weight set and the Fisher information set.

Sampling data to be learned about the task to be learned through a prediction process includes: aligning images in a task through prediction results for each image of the task to be learned and annotations of the image; And extracting a specific image among the aligned images to generate a new task.

The step of arranging the images in the task through the prediction result for each image of the task to be learned and the annotation of the image may include a difference between a prediction value, which is a prediction result for each image in the current network, and an actual annotation value of the image. Determining one real value per image through; And arranging the images in descending order according to the real value.

The step of generating a new task by extracting a specific image from the aligned images includes sampling the image having a large difference between a prediction value that is a prediction result for each image in the current network and an actual annotation value of the image, and then generating the new task. Can generate

An apparatus for alleviating the worst forgetting phenomenon in a CNN for image recognition according to another embodiment includes: a prediction unit sampling data to be learned about a task to be learned through a prediction process; And a learning unit that progresses learning using the sampled new task, and the learning unit updates the weight set and the fisher information set by calculating gradient and fisher information as the new task is delivered. can do.

The prediction unit may generate a new task by extracting a specific image among the aligned images after sorting the images in the task through the prediction result for each image of the task to be learned and the annotation of the image.

The predicting unit includes: a real-value calculation unit for determining one real value per image through a difference between a prediction value that is a prediction result for each image in the current network and an actual annotation value of the image; An alignment unit to sort the images in descending order according to the real value; And a sampling unit that generates the new task by sampling an image having a large difference between a prediction value that is a prediction result for each image of the current network and an actual annotation value of the image.

According to embodiments, a predictable elastic weight consolidation (EWC) that trains only images that are difficult to classify among images to be trained is proposed, thereby reducing the size of the task to be trained, and this process for all tasks. It is possible to provide a method and apparatus for alleviating the worst forgetting phenomenon in a CNN for image recognition that can perform efficient learning by repetition.

According to embodiments, in order to alleviate the worst forgetting problem (CFP), learning is performed using only images that have significantly different characteristics from those previously learned among the new images. By increasing variability, it is possible to improve the network's ability to classify images.

1 is a view for explaining the occurrence of the worst forgetting problem according to an embodiment.
2 is a view for explaining the learning structure of the EWC according to an embodiment.
3 is a flowchart illustrating a method for alleviating the worst forgetting phenomenon in a CNN for image recognition according to an embodiment.
4 is a block diagram illustrating an apparatus for alleviating the worst forgetting phenomenon in a CNN for image recognition according to an embodiment.
5 is a diagram schematically illustrating a learning structure of a predictable EWC according to an embodiment.
6 is a view for explaining a prediction process for generating a new task according to an embodiment.

Hereinafter, embodiments will be described with reference to the accompanying drawings. However, the described embodiments may be modified in various other forms, and the scope of the present invention is not limited by the embodiments described below. In addition, various embodiments are provided to more fully describe the present invention to those skilled in the art. The shape and size of elements in the drawings may be exaggerated for a more clear description.

In neural networks, as the process of learning a new data set is repeated, a phenomenon of losing the ability to analyze features obtained from the previously learned data set appears. This is called the Catastrophic Forgetting Problem (CFP). When classifying images using CNN (Convolutional Neural Networks), the representative feature appearing in each category has a significant effect on the network determining the classification of the predicted image, so the worst forgetting problem (CFP) is very It is fatal.

The predictable EWC (Predictive Elastic Weight Consolidation) proposed in this embodiment can sample and train only images that are difficult to classify among the images to be trained. At this time, the criterion for extracting the sample is the difference between the predicted value of the network and the actual annotation value. Through this, the size of the task to be learned can be reduced while efficient learning can be performed by repeating this process for all tasks.

Hereinafter, a method and apparatus for alleviating the worst forgetting phenomenon in a CNN for image recognition will be described in detail.

The good performance of CNN is due to the use of mid-level features rather than low-level features. For this, a large number of annotation images are required because the number of weights to be adjusted increases. Therefore, in most cases, instead of learning a large image set every time, the network is initialized with the weights already completed and additional learning is performed. Through this, the new weight to be generated can start new learning while maintaining the ability to recognize the image feature of the existing weight.

This is called transfer learning, and its implementation methods include feature extractor and finetuning. The feature extractor uses the feature recognition ability without modifying the imported weights, and trains only the fully-connected classifier in the last stage that classifies the analyzed features into desired annotations. The finetuning is done in the same way, but all the weights including the classifier are modified according to the characteristics of the newly learned image, and the learning is progressed. The feature extractor has a high efficiency in terms of memory or learning time due to a small amount of parameters to be modified, but in general, finetuning tends to fit better to an image on which training has been performed.

The study to solve the CFP is divided into five types: regularization, ensemble, rehearsal, dual-memory, and sparse-coding. Can be classified. Regularization added the condition to control the variability to the loss function, which is the object to be minimized in the learning process. The ensemble creates a new network for each learning set, that is, for each task, and makes the synthesized prediction the final result. When rehearsal techniques give a network a new set of learning, the rehearsal technique brings together some of the set that has already been learned. Dual-memory (dual-memory) separates the network with high stability and the network with high variability, and combines the results of the two after learning. Sparse-coding is intended to reduce the likelihood that each feature will affect each other by not overlapping the range of numerical values expressed. Each technique used a different idea, but when trying to learn images in common, it was intended to preserve the characteristics of expressing the already learned task well.

The ensemble technique has been widely used to improve performance since the beginning of deep learning. As the structure of the network becomes complicated and the number of weights to be adjusted increases, the accuracy of prediction increases, but the amount of calculation increases rapidly. The ensemble trains multiple simple networks independently, instead of creating one complex network with a large number of weights for the same set of images. And by making final predictions by synthesizing the results of each network, it is possible to make accurate predictions with less computing power. Ren used the idea of ensembles to learn the new and the new tasks independently, and then synthesize the predictions of each network to alleviate the worst-case problem (CFP). In addition, an algorithm for determining whether the generated new weight is to be reflected in the existing weight is additionally proposed, not just for learning for a new task. However, when dealing with high-resolution images, the cost is high in the part of performing convolution. Fine-tuning multiple networks into high-resolution images is not suitable in terms of computational complexity, since the overhead obtained by increasing the resolution of the input image is greater than the computational savings achieved by reducing the depth and width of the network.

The rehearsal technique does not change the structure of the network, but reuses the training dataset. After learning one task, the next task is learned, but a part of the previously learned task is sampled and added to a new task before learning. By doing this review together, we avoid the CFP. However, this technique has a disadvantage in that all images used in the last step training must be kept in memory for sampling. To improve this inefficient resource utilization problem, Robins proposed a pseudo-rehearsal technique. This technique does not reuse the image itself, but adds a pseudo pattern obtained by analyzing an existing image to a new task. Since the pattern fits in place of the image, the memory overhead is reduced.

2 is a view for explaining the learning structure of the EWC according to an embodiment.

Referring to FIG. 2, the existing EWC (Elastic Weight Consolidation) introduces the concept of fisher information used in statistics to determine how much variability each weight value has when learning a new task. Here, Fisher information means the amount of information obtained through observation of an event when an event occurs. Since an event with a smaller probability of occurrence has more information, the value of Fisher information has a larger value with a smaller probability of occurrence. Fisher information can be applied by looking at one task as the background of an event, and by viewing each value of the weights that have been learned about it as an independent event. Accordingly, the smaller the probability that the currently calculated weight is from the corresponding task, the greater the value of the Fisher information.

EWC performed loss regularization by adding the loss reflecting the Fisher information to the loss function. Accordingly, the larger the value of the Fisher information among the weight values, the more updates. That is, the variability becomes high.

Therefore, the degree to which the weight has been influenced by the already learned task can be considered as the Weight of Weight (WoW). By changing the variability of each weight through the process of suppressing variability, the value reflecting the main characteristic produces the result that one network ensembles a network with high stability and a network with high variability. Since only one set of weights is stored, it is more efficient than an ensemble in terms of memory, and it is possible to design a structure close to the brain pursued by the artificial neural network in that the importance of each weight is different.

2 shows the learning structure of the EWC, and each weight of the network has Fisher information. Since the variability of the weight depends on the Fisher information, the Fisher information can be considered as the weight of the weight. As the new task D enters, calculation of the fisher information and backpropagation are performed, so that the weight set W and the fisher information set WoW can be updated.

3 is a flowchart illustrating a method for alleviating the worst forgetting phenomenon in a CNN for image recognition according to an embodiment.

Referring to FIG. 3, a method for alleviating the worst forgetting phenomenon in a CNN for image recognition according to an embodiment includes: (S110) sampling data to be learned about a task to be learned through a prediction process, and a new sampled task It may be made by including the step of progressing the learning using (S120).

Here, the step (S110) of sampling the data to be learned about the task to be learned through the prediction process is to sort the images in the task through the prediction result for each image of the task to be learned and the annotation of the image, and to be aligned. And generating a new task by extracting a specific one of the images.

In addition, the step of aligning the images in the task through the annotation of the prediction result and the image for each image of the task to be learned (S120), the difference between the prediction value, which is the prediction result for each image of the current network, and the actual annotation value of the image. The method may include determining one real value per image through and sorting images in descending order according to the real value.

Hereinafter, a method for alleviating the worst forgetting phenomenon in the CNN for image recognition according to an embodiment will be described as an example.

The method for alleviating the worst oblivion phenomenon in the CNN for image recognition according to an embodiment may be described in more detail by using the apparatus for alleviating the worst oblivion in the CNN for image recognition according to an embodiment.

4 is a block diagram illustrating an apparatus for alleviating the worst forgetting phenomenon in a CNN for image recognition according to an embodiment.

Referring to FIG. 4, the apparatus 100 for alleviating the worst forgetting phenomenon in the CNN for image recognition according to an embodiment may include a prediction unit 110 and a learning unit 120. According to an embodiment, the prediction unit 110 may further include a real value calculation unit 111, an alignment unit 112, and a sampling unit 113. Also, the learning unit 120 may further include a gradient calculation unit 121, a Fisher information extraction unit 122, and an update unit 123.

In step S110, the prediction unit 110 may sample data to be learned about the task to be learned through a prediction process.

More specifically, the prediction unit 110 may generate a new task by extracting a specific image from the sorted images after sorting the images in the task through the prediction result and the annotation of the image for each image of the task to be learned. have.

Here, the prediction unit 110 may further include a real value calculation unit 111, an alignment unit 112, and a sampling unit 113.

The real value calculating unit 111 may determine one real value per image through a difference between a prediction value that is a prediction result for each image of the current network and an actual annotation value of the image.

The sorting unit 112 may sort the images in descending order according to the real value.

In addition, the sampling unit 113 may generate a new task by extracting and sampling an image having a large difference between a prediction value, which is a prediction result for each image of the current network, and an actual annotation value of the image.

In step S120, the learning unit 120 may progress learning using the new sampled task.

More specifically, the learning unit 120 may update the weight set and the fisher information set by calculating gradient and fisher information as new tasks are delivered.

The learning unit 120 may further include a gradient calculation unit 121, a Fisher information extraction unit 122, and an update unit 123.

The gradient calculator 121 may calculate the gradient as a new task is delivered.

The fisher information extraction unit 122 may calculate fisher information as new tasks are delivered.

The update unit 123 may receive the calculation results from the gradient calculation unit 121 and the Fisher information extraction unit 122 to update the weight set and the Fisher information set.

As described above, the embodiments extend EWC, a transfer learning technique through loss regularization, to alleviate the worst forgetting problem (CFP). I can provide a technique for learning using only images.

EWC, for each weight, the greater the degree of involvement in recognizing representative features, the lower the variability. This makes it possible to adjust the weights that are less important to the important features in image classification, thereby alleviating the worst-case forgetting problem (CFP) and preserving the network's perception. All newly input image sets are trained while each weight has the degree of involvement in the image sets already learned.

The apparatus 100 for alleviating the worst forgetting phenomenon in the CNN for image recognition according to an embodiment (hereinafter, simply referred to as "predictable EWC 100") adopts a process of adjusting variability by weight. , Unlike EWC, it does not learn as soon as it receives a new set of images. Instead, among the images of the task to be learned, images with many features that are difficult for the network to recognize at that time are extracted and this sampling set is determined as the task to be learned. Since EWC was used at this time, the weight heavily involved in image classification maintains stability.

Additionally, the predictable EWC according to an embodiment samples the image to be trained and replaces the task, so that the degree of involvement in image classification is small, so that the highly variable weight is more intensively learned. In addition, unlike the existing EWC, since not all images are used, the number of images to learn decreases, thereby reducing the learning time.

Therefore, the predictable EWC according to an embodiment may improve the image classification ability of the network by increasing the variability of other weights while maintaining the stability of the main weights by learning only difficult images.

As a result of the experiment, the time of one time of EWC learning without sampling and the time of three times of predictable EWC learning with sampling rate of 0.5 were similar. As a result of AUROC measurement in both experiments, the lowest AUROC value was about 0.3, whereas the predictable EWC was about 0.6. In addition, it was confirmed that the highest AUROC in the learning of fine-tuning all images without dividing the task was about 0.6. As a result, it was confirmed through sampling that the recognition accuracy was doubled while using the same amount of memory resources.

Below, the structure of the proposed predictable EWC will be described, and specific elements of the experiment and its results will be described.

The predictable EWC according to an embodiment may use sampling concepts of rehearsal and pseudo-rehearsal and Fisher information of the EWC technique. Conventionally, the technique using sampling had to have in memory a pseudo-pattern generated by reflecting the images themselves used for learning or characteristics of the images. Despite the inefficiencies in terms of memory, the performance of the network is improved, but there is no improvement over other techniques in alleviating the worst-case forgetting problem (CFP). On the other hand, the predictable EWC according to an embodiment does not sample from a task that has already been used for learning, such as a rehearsal technique.

5 is a diagram schematically illustrating a learning structure of a predictable EWC according to an embodiment.

Referring to FIG. 5, in the predictable EWC 100 according to an embodiment, each set D denotes each task, and D ′ denotes a new task made through network prediction and sampling in the corresponding task D. According to an embodiment, the learning set reduced through the prediction process in the prediction unit 110, and the gradient and Fisher information may be calculated by the learning unit 120 to perform an update.

The predictable EWC 100 according to an embodiment does not need to store an image in a memory because it performs a process of sampling data to be learned for a task to be learned in the future. To this end, before the task to be learned in the future is learned, the prediction unit 110 may assume the task as a data set for inspection and perform a prediction process first. And after aligning the images in the task through the prediction result for each image and the annotation of the image, a new task is generated, and the learning part 120 replaces the original task with this new task to progress the learning. At this time, the alignment criterion is L1-norm of the difference between the predicted value of the network and the actual annotation value for each image, and can be expressed as the following equation.

[Equation 1]

[Equation 2]

[Equation 3]

[Equation 4]

[Equation 5]

Here, [Equation 2] means a task, and one element constituting it is a pair of images [Equation 3] and a corresponding annotation [Equation 4]. n is the number of elements in the task [Equation 2], that is, the size of each task. C denotes how many classes are given in the learning network [Equation 5]. Since the sigmoid function is applied at the last stage of the classifier, the predicted value of the image passing through the network is a vector of real values between 0 and 1. The annotation value is a vector of length C consisting of 0 or 1, depending on whether it is negative or positive for each classification.

Therefore, according to [Equation 1], the absolute values of the difference between the predicted value and the annotated value of a network for each classification are calculated for one image, and the sum of them calculates one positive real value per image. At this time, it can be interpreted that the larger the real value, the greater the difference between the predicted value of the network and the actual annotation value, so that the larger the norm value, the more difficult the network is to recognize.

6 is a view for explaining a prediction process for generating a new task according to an embodiment.

Referring to FIG. 6, a process for generating a new task in the prediction unit 110 is shown, and an image in D is predicted by a network and 1-norm is sorted by annotation corresponding to it, and values are sorted in order of highest value. Images can be extracted.

More specifically, after obtaining the L1-norm for each image, the images in the task can be sorted in descending order, and a new task can be generated by sampling the upper image according to a predetermined sampling rate. The resulting new task is smaller in size than the original task, and it is the set of the most difficult images to classify to the network because only L1-norm values are extracted.

The absolute value of the difference between the predicted value of the classified network and the annotated value of the classified network is always 0 to 1. Therefore, when L2-norm is taken, the norm value is smaller than the absolute value of the original difference through the square operation. Here, since the role of norm is to determine the difficulty of the corresponding image by measuring the distance between the prediction and the annotation, the sum of the absolute values is a clearer criterion than the sum of squares. Therefore, alignment and sampling can be performed based on L1-norm, not L2-norm.

In order to perform this process, the process of creating a new task requires the network to predict the image and the additional cost of aligning the image. However, the prediction process is a linear operation between the input image and the weight and the activation function passing process, and the alignment process is a simple descending order of positive real values and does not require a complicated operation. Therefore, even if all images of the new task are used by setting the sampling rate to 1, there is no significant difference in the total learning time from the existing EWC.

On the other hand, when sampling is performed, the number of images decreases, and thus, the computational amount of gradients for weight update is significantly reduced. Therefore, there is a cost saving effect greater than the additional cost due to the prediction and alignment process. Therefore, the predictable EWC according to an embodiment takes less total learning time than the existing EWC.

Replacing the new task input for learning with a smaller task reduces the number of features the network learns. As shown in FIG. 1 (a), the basic form that does not consider the worst-case forgetting problem (CFP) decreases the number of features to learn, and not only the learning of new images, but also the worst-case forgetting problem (CFP) Occurs and loses the ability of the network to classify images.

However, the predictable EWC according to an embodiment may use the existing EWC to preserve the classification ability already possessed and further learn only features that are not yet sufficiently learned through task creation. Fisher information, which has been adopted from EWC, plays a role in increasing the stability of weights, which played a major role in recognizing the characteristics of images already learned, and slowing the learning speed of the weights. Therefore, when the network using the fisher information learns a new task, if a certain weight is high in variability, the weight is a relatively low weight associated with an image characteristic well recognized at the time. The predictable EWC according to an embodiment configures a new task to be learned into an image having a distance between prediction and annotation.

Accordingly, the predictable EWC according to an embodiment may train an image rich in features that the network lacks in ability to recognize, with a weight that has a small degree of contribution to the recognition ability. Thus, while preserving the classification ability before learning new tasks as much as the existing EWC, it is possible to learn efficiently through only difficult images.

As described above, the artificial neural network suffers from catastrophic forgetting, which is a problem that the network loses the ability to recognize features of existing data due to new data. Rehearsal eased the worst forgetting problem (CFP) by reusing learning materials, and EWC solved the worst forgetting problem (CFP) by adjusting the update rate for each internal node of weight through loss constraint I tried to.

These embodiments are based on the rehearsal sampling method and the EWC loss constraint technique, but the method of sampling and reusing the learning data from the already learned task causes memory overload, so the task that was learned It was conducted in a task to be learned in the future, not in Esau. At this time, by sampling the image with a large difference between the predicted value and the actual annotation on the current network image, the study focused on the image rich in features with poor network classification.

The experiment was conducted in four cases: fine-tuning of the entire set, transfer learning for each task, EWC, and predictable EWC. The worst AUROC (Area Under Receiver Operating Characteristic) of the other cases, except for fine tuning, was improved to about 0.3, and the worst AUROC of the proposed technique was improved to about 0.6. That is, the proposed predictable EWC technique has a worst AUROC of about 0.6, which is improved over 0.3 of the existing EWC.

The device described above may be implemented with hardware components, software components, and / or combinations of hardware components and software components. For example, the devices and components described in the embodiments include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor (micro signal processor), a microcomputer, a field programmable array (FPA), It may be implemented using one or more general purpose computers or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may run an operating system (OS) and one or more software applications running on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of understanding, a processing device may be described as one being used, but a person having ordinary skill in the art, the processing device may include a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that may include. For example, a processing device may include a plurality of processors or a processor and a controller. In addition, other processing configurations, such as parallel processors, are possible.

The software may include a computer program, code, instruction, or a combination of one or more of these, and configure the processing device to operate as desired, or process independently or collectively You can command the device. Software and / or data may be interpreted by a processing device, or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. Can be embodied in The software may be distributed on networked computer systems, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer-readable medium may include program instructions, data files, data structures, or the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiments or may be known and usable by those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, and magnetic media such as floptical disks. -Hardware devices specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language code that can be executed by a computer using an interpreter, etc., as well as machine language codes produced by a compiler.

As described above, although the embodiments have been described by a limited embodiment and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques are performed in a different order than the described method, and / or the components of the described system, structure, device, circuit, etc. are combined or combined in a different form from the described method, or other components Alternatively, even if replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (7)

In the method of mitigating the worst oblivion in the image recognition CNN using the apparatus for alleviating the worst oblivion in the CNN for image recognition,
Sampling data to be learned about a task to be learned by a prediction unit of the apparatus for alleviating the worst forgetting phenomenon in the CNN for image recognition through a prediction process; And
Learning by using the new task sampled by the learning unit of the apparatus for alleviating the worst forgetting phenomenon in the CNN for image recognition.
Including,
The step of sampling the data to be learned about the task to be learned through a prediction process,
Sorting the images in the task through the prediction result for each image of the task to be learned and the annotation of the image; And
Generating a new task by extracting a specific image from the aligned images
It includes,
The step of progressing the learning using the sampled new task,
Calculating gradient and fisher information as the new task is delivered to update the weight set and the fisher information set
Characterized in that, the method of mitigating the worst oblivion phenomenon in CNN for image recognition. delete According to claim 1,
The step of arranging the images in the task through the prediction result for each image of the task to be learned and the annotation of the image,
Determining one real value per image through a difference between a prediction value that is a prediction result for each image in the current network and an actual annotation value of the image; And
Sorting the images in descending order according to the real value
A method of alleviating the worst oblivion phenomenon in a CNN for image recognition, comprising a. According to claim 1,
The step of generating a new task by extracting a specific image from the aligned images,
Generating the new task by sampling an image with a large difference between a prediction value that is a prediction result for each image in the current network and an actual annotation value of the image
Characterized in that, the method of mitigating the worst oblivion phenomenon in CNN for image recognition. In the apparatus for alleviating the worst forgetting phenomenon in the image recognition CNN,
A prediction unit that samples the data to be learned about the task to be learned through a prediction process; And
A learning department that progresses learning using new sampled tasks
Including,
The prediction unit,
After sorting the images in the task through the prediction result for each image of the task to be learned and the annotation of the image, a specific task is extracted from the aligned images to generate a new task,
The learning unit,
Calculating gradient and fisher information as the new task is delivered to update the weight set and the fisher information set
Characterized in that, the apparatus for mitigating the worst forgetting phenomenon in the CNN for image recognition. delete The method of claim 5,
The prediction unit,
A real value calculation unit for determining one real value per image through a difference between a prediction value that is a prediction result for each image in the current network and an actual annotation value of the image;
An alignment unit to sort the images in descending order according to the real value; And
A sampling unit that generates the new task by sampling an image with a large difference between a prediction value that is a prediction result for each image of the current network and an actual annotation value of the image.
Including, including, the apparatus for mitigating the worst forgetting phenomenon in the CNN for image recognition.

Images