WO2024090600A1

WO2024090600A1 - Deep learning model training method and deep learning computation apparatus applied with same

Info

Publication number: WO2024090600A1
Application number: PCT/KR2022/016397
Authority: WO
Inventors: 이상설; 장성준; 김경호
Original assignee: 한국전자기술연구원
Priority date: 2022-10-26
Filing date: 2022-10-26
Publication date: 2024-05-02
Also published as: KR20240058252A

Abstract

Provided are a deep learning model training method and a deep learning computation apparatus applied with same. The deep learning model training method according to an embodiment of the present invention comprises training a deep learning model, pruning some weights in the trained deep learning model, and loading specific weights to the pruned weights. Accordingly, the deep learning computation apparatus in which resources are limited can quickly perform training while quickly improving prediction accuracy by applying pre-learned weights to weights pruned during deep learning model training by an additional dataset.

Description

Deep learning model learning method and deep learning computing device to which it is applied

The present invention relates to image-based deep learning processing and system SoC (System on chip) technology, and more specifically, to a method of learning a deep learning model at high speed with high accuracy in a lightweight deep learning computing device.

The best way to learn deep learning is to learn a deep learning model using many training datasets. However, in the case of deep learning computing devices (deep learning accelerators) with limited resources, such as SoC, learning from many learning datasets is impossible.

Accordingly, a method of additional learning using only a small amount of training data set for a transfer learned deep learning model is widely used.

However, in this case, there is a problem that the accuracy of the deep learning model is lowered, and due to resource limitations, additional learning takes a lot of time, so the learning speed is very slow.

The present invention was created to solve the above problems, and the purpose of the present invention is to quickly learn a deep learning model using additional datasets in a deep learning computing device with limited resources, while maintaining a high level of prediction accuracy. To provide a deep learning model learning method that can be maintained and a deep learning computing device to which it is applied.

A deep learning model learning method according to an embodiment of the present invention to achieve the above object includes a first learning step of training a deep learning model; A first pruning step of pruning some weights in the learned deep learning model; It includes a first loading step of loading specific weights into the pruned weights.

The first loading step may load the weights of a previously learned deep learning model. In the first learning step, the deep learning model to which the weights of the previously learned deep learning model are transferred may be fine-tuned to the first data set.

A deep learning model learning method according to an embodiment of the present invention includes a second learning step of fine tuning the deep learning model on which the first loading step has been performed with a second data set; A second pruning step of pruning some weights in the fine-tuned deep learning model; It may further include a second loading step of loading specific weights into the pruned weights.

The second loading step may load the weights of a previously learned deep learning model. Some weights pruned in the second pruning step may be some of the weights pruned in the first pruning step.

The first pruning step and the second pruning step may prune weights on a channel basis. The first pruning step and the second pruning step may prune weights of different channels for each layer.

Deep learning models can be mounted on lightweight, low-power deep learning computing devices.

A deep learning computing device according to another embodiment of the present invention trains a deep learning model. An operator that prunes some weights from the learned deep learning model and loads specific weights into the pruned weights; and a memory that provides storage space required for the calculator.

A deep learning model learning method according to another embodiment of the present invention includes a first pruning step of pruning some weights in the deep learning model; A first loading step of loading specific weights into the pruned weights; a second pruning step of pruning some weights in the deep learning model in which the first loading step was performed; It includes a second loading step of loading specific weights into the pruned weights.

A deep learning computing device according to another embodiment of the present invention prunes some weights in a deep learning model, loads specific weights on the pruned weights, and prunes some weights in the deep learning model loaded with specific weights. an operator for pruning weights and loading specific weights into the pruned weights; and a memory that provides storage space required for the calculator.

As described above, according to embodiments of the present invention, by applying pre-learned weights to pruned weights when learning a deep learning model using an additional dataset in a deep learning computing device with limited resources. , prediction accuracy can be quickly improved while learning progresses quickly.

1 is a diagram conceptually showing a deep learning model learning method in a deep learning computing device;

Figure 2 shows test results for the transfer learned deep learning model,

3 to 5 are diagrams provided to explain a deep learning model learning method according to an embodiment of the present invention;

Figure 6 is a diagram showing the configuration of a deep learning computing device according to another embodiment of the present invention.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

Figure 1 is a diagram conceptually showing a deep learning model learning method in a deep learning computing device (deep learning accelerator). As shown in the upper part of FIG. 1, a deep learning computing device that cannot learn from many learning datasets provides additional data for the deep learning model transfer learned by the server as shown in the lower part of FIG. 1. It is carried out by learning three.

Figure 2 shows test results for the transfer learned deep learning model. As shown, when a transfer-learned deep learning model is additionally trained, learning performance quickly increases compared to a deep learning model without transfer learning.

However, in the case of transfer learning, a catastrophic forgetting phenomenon occurs, that is, a phenomenon in which the accuracy of the previous dataset that is additionally learned from another consecutive dataset decreases.

As a way to solve this, it is possible to independently apply the FC layer (Fully Connected Layer) for the additional dataset. However, this is also difficult to apply to deep learning computing devices with limited resources. This is because the increase in FC layers increases each time a new dataset increases and at the same time causes the problem of deterioration of existing learning performance.

An embodiment of the present invention presents a deep learning model learning method that can quickly train a deep learning model using an additional dataset in a deep learning computing device with limited resources while maintaining high prediction accuracy.

3 to 5 are diagrams provided to explain a deep learning model learning method according to an embodiment of the present invention. The deep learning model learning method according to an embodiment of the present invention is suitable for learning a deep learning model mounted on a lightweight deep learning accelerator, but is not necessarily limited to this and can also be applied in other environments/methods.

To learn a deep learning model, first, weights are transferred to the deep learning model as shown in Figure 3. This is a process of securing the weights of the deep learning model acquired through pre-training using a large amount of learning data sets at the server side and loading them into the deep learning model to be learned.

In Figure 3, the weights shown on the left are the weights of the first layer, and the weights shown on the right are the weights of the second layer. According to this, the deep learning model trained in the embodiment of the present invention consists of two layers, but this is only an example for convenience of explanation. There is no limit to the number of layers of a deep learning model to which embodiments of the present invention can be applied.

Meanwhile, as shown in Figure 3, the deep learning model has a structure in which images are input through multi-channels, and feature maps of the images are also generated through multi-channels, and are divided by weights for each channel.

Next, as shown in the upper part of FIG. 4, the deep learning accelerator uses dataset #1 to fine-tune the deep learning model to which the weights have been transferred, and to select weights subject to pruning.

And, as shown in the center of Figure 4, some weights are pruned in the fine-tuned deep learning model. In Figure 4, the weights subject to pruning are those displayed in white.

As shown, weight pruning is performed on a channel basis. That is, the weights for some channels are pruned and the weights for the remaining channels are left. Meanwhile, weight pruning can prune the weights of different channels for each layer. As shown, the weight pruning target channels in the first layer shown on the left and the weight pruning target channels in the second layer shown on the right are different from each other.

Thereafter, as shown in the lower part of FIG. 4, the weights of the previously learned deep learning model are loaded for the pruned weights. Previously, 0 was loaded into pruned waddles or randomly generated weights were loaded. In an embodiment of the present invention, the prediction accuracy of the deep learning model was improved by loading the weights of the previously learned deep learning model into the pruned weights.

For the deep learning model for which training has been completed using dataset #1, additional learning can be performed using dataset #2, and this process is shown in FIG. 5.

First, as shown in the upper part of FIG. 5, the deep learning accelerator uses dataset #2 to fine-tune the deep learning model that has been learned through the process shown in FIG. 4, and select weights subject to pruning. .

In this process, weights that were not subject to pruning in FIG. 4 can be excluded from the pruning subject. That is, among the weights that were the pruning target in FIG. 4, some weights are selected as the pruning target. Weights that were not subject to pruning in Figure 4 are not selected for pruning even in learning using dataset #2.

Furthermore, by excluding weights that were not subject to pruning in FIG. 4 from fine tuning, it is possible to prevent the weights from being changed by fine tuning.

Next, as shown in the center of FIG. 5, some weights selected from the fine-tuned deep learning model are pruned. In Figure 5, the weights subject to pruning are those displayed in white.

Similar to learning with dataset #1 shown in FIG. 4, in learning with dataset #2 shown in FIG. 5, weight pruning is performed on a channel basis, and the pruning target channel for each layer may be different. .

Thereafter, as shown in the lower part of FIG. 5, the weights of the previously learned deep learning model are loaded onto the pruned weights.

Figure 6 is a diagram showing the configuration of a deep learning computing device according to another embodiment of the present invention. As shown, the deep learning computing device according to an embodiment of the present invention includes a communication interface 110, a deep learning calculator 120, and a memory 130.

The communication interface 110 communicates with an external host system and receives data sets, parameters (weight, bias) of previously learned deep learning models, etc. The deep learning calculator 120 trains the mounted deep learning model using the method shown in FIGS. 3 to 5 described above. The memory 130 provides storage space necessary for the deep learning calculator 120 to perform calculations.

So far, the deep learning model learning method and the deep learning computing device to which it is applied have been described in detail with preferred embodiments.

In the above example, a method of applying pre-learned weights to pruned weights during a deep learning model using an additional dataset in a deep learning computing device with limited resources was presented.

As a result, the deep learning processing unit does not perform calculations on the pruned weights, allowing high-speed learning with low power while maintaining prediction accuracy at a high level.

Meanwhile, of course, the technical idea of the present invention can be applied to a computer-readable recording medium containing a computer program that performs the functions of the device and method according to this embodiment. Additionally, the technical ideas according to various embodiments of the present invention may be implemented in the form of computer-readable code recorded on a computer-readable recording medium. A computer-readable recording medium can be any data storage device that can be read by a computer and store data. For example, of course, computer-readable recording media can be ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical disk, hard disk drive, etc. Additionally, computer-readable codes or programs stored on a computer-readable recording medium may be transmitted through a network connected between computers.

In addition, although preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the invention pertains without departing from the gist of the present invention as claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be understood individually from the technical idea or perspective of the present invention.

Claims

A first learning step of training a deep learning model;

A first pruning step of pruning some weights in the learned deep learning model;

A deep learning model learning method comprising a first loading step of loading specific weights into the pruned weights.
In claim 1,

The first loading step is,

A deep learning model learning method characterized by loading the weights of a previously learned deep learning model.
In claim 2,

The first learning stage is,

A deep learning model learning method characterized by fine tuning a deep learning model to which the weights of a previously learned deep learning model are transferred to a first data set.
In claim 3,

A second learning step of fine tuning the deep learning model on which the first loading step was performed with a second data set;

A second pruning step of pruning some weights in the fine-tuned deep learning model;

A deep learning model learning method further comprising a second loading step of loading specific weights into the pruned weights.
In claim 4,

The second loading step is,

A deep learning model learning method characterized by loading the weights of a previously learned deep learning model.
In claim 4,

Some weights that are pruned in the second pruning step are:

A deep learning model learning method, characterized in that some weights are among the weights pruned in the first pruning step.
In claim 4,

The first pruning step and the second pruning step are,

A deep learning model learning method characterized by pruning weights on a channel basis.
In claim 7,

The first pruning step and the second pruning step are,

A deep learning model learning method characterized by pruning the weights of different channels for each layer.
In claim 1,

The deep learning model is,

A deep learning model learning method characterized by being mounted on a lightweight, low-power deep learning computing device.
Train a deep learning model. An operator that prunes some weights from the learned deep learning model and loads specific weights into the pruned weights; and

A deep learning computing device comprising a memory that provides storage space required for the computing device.
A first pruning step of pruning some weights in the deep learning model;

A first loading step of loading specific weights into the pruned weights;

a second pruning step of pruning some weights in the deep learning model in which the first loading step was performed;

A deep learning model learning method comprising a second loading step of loading specific weights into the pruned weights.
Pruning some weights in a deep learning model and loading specific weights into the pruned weights. Pruning some weights from a deep learning model loaded with specific weights and loading specific weights into the pruned weights. a calculator that does; and

A deep learning computing device comprising a memory that provides storage space required for the computing device.