KR20230068989A - Method and electronic device for performing learning of multi-task model - Google Patents

Abstract

A method and electronic device for performing learning of a multi-task model of the present disclosure set a task weight corresponding to each task of a loss function of a multi-task model based on an inherent accuracy of each task of the multi-task model and an accuracy according to a learning degree of each task, and perform learning of the multi-task model using the loss function of the multi-task model.

Description

METHOD AND ELECTRONIC DEVICE FOR PERFORMING LEARNING OF MULTI-TASK MODEL

A method and electronic device for performing multi-task model learning.

With the rapid development of the computer vision field, neural network models that perform each of the various vision tasks have an excellent level of accuracy. However, when various neural network models are used together to observe a scene from various viewpoints to understand a scene, problems may occur depending on the performance and resources of an embedded device. Accordingly, interest in a multi-task model that can efficiently and quickly perform multi-tasks is growing.

According to an embodiment of the present disclosure, a method for performing learning of a multi-task model is provided. A method for performing learning of the multi-task model is based on the accuracy of each task of the multi-task model and the accuracy according to the learning degree of each task, and the task corresponding to each task corresponds to the loss of each task of the multi-task model. and setting a task weight corresponding to each task of a loss function of the multi-task model corresponding to a weight sum to which weights have been applied. Also, the method of performing learning of the multi-task model includes performing learning of the multi-task model by using a loss function of the multi-task model.

According to one embodiment of the present disclosure, it is possible to provide a computer-readable recording medium on which a program for executing the above-described method is recorded.

According to an embodiment of the present disclosure, an electronic device performing learning of a multi-task model is provided. An electronic device that performs learning of the multi-task model includes a memory storing one or more instructions and a processor executing the one or more instructions stored in the memory. By executing one or more instructions, the processor sets the task weight corresponding to each task to the loss of each task of the multi-task model, based on the unique accuracy of each task of the multi-task model and the accuracy according to the learning degree of each task. Set the task weight corresponding to each task of the loss function of the multi-task model corresponding to the applied weight sum. Also, the processor performs learning of the multi-task model using the loss function of the multi-task model by executing the one or more instructions.

1 is a diagram for explaining a multi-task model.
2 is a flowchart illustrating a method of performing learning of a multi-task model according to an embodiment of the present disclosure.
3 is a detailed flowchart of a step of setting a task weight corresponding to each task of a loss function of a multi-task model according to an embodiment of the present disclosure.
4 is a detailed flowchart of a step of obtaining accuracy of a single-task teaching model corresponding to each task of a multi-task model according to an embodiment of the present disclosure.
5 is a diagram for explaining a process of performing learning of a single-task teaching model corresponding to each task of a multi-task model according to an embodiment of the present disclosure.
6 is a detailed flowchart of a step of performing learning of a multi-task model using a loss function of a multi-task model according to an embodiment of the present disclosure.
7 is a diagram for explaining an example of loss of each task in a process of performing self-learning of a multi-task model according to an embodiment of the present disclosure.
8 illustrates an example of loss of each task when an arbitrary task having a ground-truth is included in a process of performing self-learning of a multi-task model according to an embodiment of the present disclosure. It is a drawing for
9 is a diagram for explaining a result of learning a vision model that performs a multi-task according to a method for learning a multi-task model according to an embodiment of the present disclosure.
10 is a block diagram illustrating the configuration of an electronic device according to an embodiment of the present disclosure.
11 is a diagram illustrating an example of an electronic device according to an embodiment of the present disclosure.

Terms used in this specification will be briefly described, and the present disclosure will be described in detail. In this disclosure, the expression “at least one of a, b, or c” means “a”, “b”, “c”, “a and b”, “a and c”, “b and c”, “a, b” and c”, or variations thereof.

The terms used in the present disclosure have been selected from general terms that are currently widely used as much as possible while considering the functions in the present disclosure, but they may vary according to the intention or precedent of a person skilled in the art, the emergence of new technologies, and the like. In addition, in a specific case, there are also terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the corresponding description. Therefore, terms used in the present disclosure should be defined based on the meaning of the term and the general content of the present disclosure, not simply the name of the term.

Singular expressions may include plural expressions unless the context clearly dictates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by one of ordinary skill in the art described herein. Also, terms including ordinal numbers such as 'first' or 'second' used in this specification may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another.

When it is said that a certain part "includes" a certain component throughout the specification, it means that it may further include other components without excluding other components unless otherwise stated. In addition, terms such as “unit” and “module” described in the specification refer to a unit that processes at least one function or operation, and may be implemented as hardware or software or a combination of hardware and software.

Hereinafter, with reference to the accompanying drawings, embodiments of the present disclosure will be described in detail so that those skilled in the art can easily carry out the present invention. However, the present disclosure may be embodied in many different forms and is not limited to the embodiments described herein.

Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

1 is a diagram for explaining a multi-task model.

In the present disclosure, the electronic device 100 may include a mobile device such as a smart phone, smart glasses, a wearable device, a personal digital assistant (PDA) or a mobile scanner, but is not limited thereto.

The electronic device 100 may be loaded with various types of neural network models. For example, the electronic device 100 may include at least one model such as a deep neural network (DNN), a convolution neural network (CNN), a recurrent neural network (RNN), and a bidirectional recurrent deep neural network (BRDNN). These may be used in combination.

When the electronic device 100 is an embedded device loaded with various neural network models, problems may occur depending on resources and performance of the electronic device 100 . Accordingly, as shown in FIG. 1 , the multi-task model can be loaded into the electronic device 100 so that tasks processed by various neural network models can be efficiently and quickly performed.

The multi-task model may be in the form of a neural network composed of a plurality of layers. The multi-task model may be composed of shared backbone layers and additional layers for each task, each of which takes an output of the shared backbone as an input. As shown in FIG. 1 , the multi-task model may include a feature extractor that extracts features necessary for prediction and a predictor that performs prediction using the features extracted therefrom.

For example, in the case of a vision model that performs tasks in various vision fields, most operations are performed in the part of extracting features from an input image, so as shown in FIG. 1, one feature extractor for each task Using a multi-task model in the form of connecting prediction heads can be very effective.

For convenience of description, in some descriptions, the multi-task model is a vision model that performs a task in the computer vision field as an example, but the type or function of the multi-task model described in the present disclosure is not limited thereto, and voice It can be an artificial intelligence model that can be used in various fields such as recognition and natural language processing.

Since the multi-task model must be trained so that all tasks can have uniform performance without being biased toward a specific task, it is very important which loss function and which data set to train the multi-task model with.

The loss function of the multi-task model may be defined as a weight sum obtained by applying task weights corresponding to each task to each task loss, as shown in Equation 1 below.

[Equation 1]

In the multi-task model, the accuracy of each task is greatly influenced by how the task weight corresponding to each task is set. When the multi-task model can be learned in consideration of both the characteristics and the degree of learning of each task of the multi-task model, all tasks are learned in a balanced way, and even performance of the multi-task model can be expected.

Since the multi-task data set for learning the multi-task model requires a large amount of inputs (eg, input images) and annotations for various tasks, large-scale data that satisfies these requirements A set is difficult to secure realistically. To this end, by using learned single-task teacher models, pseudo labels are generated for inputs of tasks for which labels do not exist in partially labeled inputs, By securing a multi-task data set, learning of a multi-task model may be performed.

Hereinafter, a method of learning a multi-task model using a loss function of the multi-task model and a multi-task data set will be described in detail.

2 is a flowchart illustrating a method of performing learning of a multi-task model according to an embodiment of the present disclosure.

The electronic device 100 sets a task weight corresponding to each task of the loss function of the multi-task model based on the unique accuracy of each task of the multi-task model and the accuracy according to the learning degree of each task. S210) The loss function of the multi-task model corresponds to a weighted sum obtained by applying a task weight corresponding to each task to the loss of each task of the multi-task model.

The inherent accuracy of each task of the multi-task model means the expected accuracy of each task when the multi-task model is sufficiently trained. The inherent accuracy of each task in the multi-task model may be different for each task. Even if the learning process is sufficiently performed for each task, the learning level may be different depending on the type of task. For example, when a model corresponding to the first task undergoes a sufficient learning process, a highly accurate prediction value for the first task may be output. On the other hand, even if the model corresponding to the second task undergoes a sufficient learning process, if learning is difficult or the difficulty of the task is high, the accuracy of the predicted value output for the second task may not be high.

The accuracy according to the learning level of each task of the multi-task model means the current accuracy of each task according to the current learning level in the learning process of the multi-task model. Since the degree of influence by learning may be different for each task, the accuracy according to the learning degree of each task may be different for each task. For example, when the difficulty of the task is low and learning is successful, the accuracy according to the learning degree of the task rapidly converges to the intrinsic accuracy. On the other hand, when learning is not performed well due to the high difficulty of the task, the accuracy according to the learning degree of the task slowly converges to the intrinsic accuracy.

Although the intrinsic accuracy of each task in the multi-task model is different, considering only the current accuracy of each task, if the task weight is given higher as the current accuracy of each task is lower, all tasks in the multi-task model Balanced learning cannot be achieved. For example, in the case of a task with low intrinsic accuracy, even if the current accuracy is close to the intrinsic accuracy, if the current accuracy is lower than other tasks and the task weight is given high, only the specific task is learned, and all tasks are learned. A problem may occur that is not learned quickly and in a balanced way.

In the process of learning the multi-task model according to an embodiment of the present disclosure, based on the unique accuracy of each task of the multi-task model as well as the accuracy according to the learning degree of each task of the multi-task model, Set the task weight corresponding to each task of the loss function of the multi-task model.

The electronic device 100 corresponding to an embodiment of the present disclosure may acquire the unique accuracy of each task in advance before learning the multi-task model. For example, the unique accuracy of each task may be set to a predetermined value by a manager in consideration of characteristics of each task. For another example, the unique accuracy of each task may be obtained through a predetermined process.

Hereinafter, with reference to FIGS. 3 to 5 , an example of setting the task weight corresponding to each task by obtaining the unique accuracy of each task of the multi-task model in advance will be described in detail.

3 is a detailed flowchart of a step of setting a task weight corresponding to each task of a loss function of a multi-task model according to an embodiment of the present disclosure.

The electronic device 100 is a single-task teaching model corresponding to each task of the multi-task model. Accuracy may be obtained. (S310) Each single-task teaching model may correspond to each task of the multi-task model. The accuracy of the single-task teaching model means the accuracy of the single-task teaching model upon completion of learning. When the loss calculated in the learning process of the single-task teaching model is minimized, the single-task teaching model can be regarded as completed learning. Alternatively, when the accuracy of the learned single-task teaching model satisfies a predetermined condition, for example, when the accuracy of the learned single-task teaching model converges to a specific value, the single-task teaching model may be regarded as completed learning.

The electronic device 100 may communicate with at least one external device equipped with a single-task teaching model corresponding to each task of the multi-task model. The electronic device 100 may obtain the accuracy of the single-task teaching model corresponding to each task by receiving the accuracy of the single-task teaching model from at least one external device.

The electronic device 100 may load a single-task teaching model corresponding to each task of the multi-task model. The electronic device 100 may obtain the accuracy of the single-task teaching model corresponding to each task by calculating the accuracy of the single-task teaching model corresponding to each task. In this regard, it will be described in detail with reference to FIG. 4 .

4 is a detailed flowchart of a step of obtaining accuracy of a single-task teaching model corresponding to each task of a multi-task model according to an embodiment of the present disclosure.

The electronic device 100 may perform learning of a single-task instructional model corresponding to each task by using a dedicated learning data set corresponding to each task (S410). The single-task instructional model corresponding to each task. For each, a dedicated training data set may be prepared. For example, when the multi-task model is a vision model, each dedicated training data set for each task such as object detection, segmentation, depth estimation, and optical flow there may be Each dedicated training data set is a data set specialized for a task and may be a labeled data set having a much smaller size than a data set required to train a multi-task model. A process of performing learning of a single-task teaching model corresponding to each task will be described in detail with reference to FIG. 5 .

5 is a diagram for explaining a process of performing learning of a single-task teaching model corresponding to each task of a multi-task model according to an embodiment of the present disclosure.

Referring to FIG. 5 , a single-task instructional model A corresponding to task A of the multi-task model, a single-task instructional model B corresponding to task B of the multi-task model, and a single-task instructional model corresponding to task C of the multi-task model Task teaching model C shows the process of performing each learning. The number of single-task teaching models is not limited to the example shown in FIG. 5 and may be determined according to the number of tasks of the multi-task model.

The single-task teaching model A may be learned using a dedicated learning data set A corresponding to task A. Dedicated training data set A contains labeled input data small enough to learn task A. The electronic device 100 may calculate a loss corresponding to a difference between the actual value and the output A for the single-task teaching model A. The electronic device 100 may perform learning of the single-task teaching model A in a direction in which loss is reduced through backpropagation.

The single-task teaching model B may be learned using a dedicated learning data set B corresponding to task B. Dedicated training data set B contains labeled input data small enough to train task B. The electronic device 100 may calculate a loss corresponding to a difference between the actual value and the output B for the single-task teaching model B. The electronic device 100 may perform learning of the single-task teaching model B in a direction in which loss is reduced through backpropagation.

The single-task teaching model C may be learned using a dedicated learning data set C corresponding to task C. A dedicated training dataset C contains labeled input data small enough to learn task C. The electronic device 100 may calculate a loss corresponding to a difference between the actual value and the output C for the single-task teaching model C. The electronic device 100 may perform learning of the single-task teaching model C in a direction in which loss is reduced through backpropagation.

For example, if there is a multi-task model that performs object detection (corresponding to task A), segmentation (corresponding to task B), and depth estimation (corresponding to task C), the single-task teaching model A is an object detection model. , single-task instruction model B can be a segmentation model, and single-task instruction model C can be a depth estimation model. A single-task teaching model A (object detection model) can be learned using labeled images including label A for object detection. A single-task teaching model B (segmentation model) can be trained using labeled images including label B for segmentation. A single-task teaching model C (depth estimation model) can be learned using labeled images including labels C for depth estimation.

Referring back to FIG. 4 , the electronic device 100 may determine whether learning of the single-task teaching model corresponding to each task is completed (S420). Learning of the single-task teaching model corresponding to each task is completed. If it is determined that the learning is not completed, the electronic device 100 may repeatedly perform the single-task teaching model until learning is completed.

The electronic device 100 may consider that learning of the single-task instruction model A is completed when the loss calculated in the learning process of the single-task instruction model A is minimized. The electronic device 100 may consider that learning of the single-task instruction model B is completed when the loss calculated in the learning process of the single-task instruction model B is minimized. The electronic device 100 may consider that learning of the single-task instruction model C is completed when the loss calculated in the learning process of the single-task instruction model C is minimized.

The electronic device 100 may measure accuracy when learning of the learned single-task teaching model corresponding to each task is completed (S430). Upon completion of learning of the single-task teaching model A, the electronic device 100 may measure accuracy. , the accuracy of single-task instruction model B upon completion of learning, and the accuracy of single-task instruction model C upon completion of learning can be measured.

Referring back to FIG. 3 , the electronic device 100 provides a single-task teaching model corresponding to each task. Based on the accuracy and the accuracy according to the learning degree of each task, a task weight corresponding to each task of the loss function of the multi-task model may be set (S320). task teaching model Accuracy is obtained, and the obtained single-task instruction model corresponding to each task Accuracy can be used as the specific accuracy of each task.

Since the accuracy of the single-task teaching model can be viewed as the potential accuracy that can be expected when the task is performed in the multi-task model, the upper bound of the accuracy according to the learning degree of the multi-task model can be defined

According to an embodiment of the present disclosure, a task weight corresponding to each task in the loss function of the multi-task model of Equation 1 may be defined as Equation 3 as follows.

[Equation 2]

-- ①

-- ①

-- ②

-- ②

-- ③

-- ③

)와 각 태스크의 학습 정도에 따른 정확도(

) 간의 정확도 차이에 기초하여, 각 태스크에 대응되는 태스크 가중치를 설정할 수 있다. 이때, 각 태스크의 고유의 정확도(

)는 각 태스크에 대응되는 싱글-태스크 교수 모델의 정확도에 해당할 수 있다. 전자 장치(100)는 정확도 차이에 비례하여 각 태스크에 대응되는 태스크 가중치를 조정할 수 있다.According to the first equation (①) of Equation 2, the electronic device 100 has a unique accuracy of each task (

) and the accuracy according to the learning degree of each task (

), it is possible to set a task weight corresponding to each task. At this time, the unique accuracy of each task (

) is the single-task teaching model corresponding to each task. may be related to accuracy. The electronic device 100 may adjust the task weight corresponding to each task in proportion to the difference in accuracy.

)와 각 태스크의 학습 정도에 따른 정확도(

) 간의 정확도 차이는 감소하는 방향으로 이동하기 때문에, 각 태스의 태스크 가중치가 감소하게 되고, 이것은 학습률(learning rate)이 감소하는 것과 같은 효과를 가져온다. 각 태스크의 태스크 가중치가 감소함에 따라, 멀티-태스크 모델의 업데이트 양이 줄어들어, 멀티-태스 모델이 충분히 학습되지 못하고, 조기에 미세 조정(fine-tuning)되는 것을 방지할 필요가 있다. 이를 위해, 수학식 2의 두 번째 식(②)과 같이, 각 태스크에 대응되는 태스크 가중치의 합이 태스크의 개수가 되도록, 수학식 2의 첫 번째 식(①)을 정규화할 수 있다. The first equation (①) of Equation 2 can be normalized as the second equation (②) of Equation 2 to avoid underfitting. As learning of each task progresses, the inherent accuracy of each task (

) and the accuracy according to the learning degree of each task (

), the task weight of each task decreases, which has the same effect as reducing the learning rate. As the task weight of each task decreases, the update amount of the multi-task model decreases, so that it is necessary to prevent the multi-task model from not being sufficiently trained and fine-tuned at an early stage. To this end, like the second equation (②) of Equation 2, the first equation (①) of Equation 2 can be normalized such that the sum of task weights corresponding to each task becomes the number of tasks.

The second equation (②) of Equation 2 is in the form of adding a slight margin, like the third equation (③) of Equation 2, so that the multi-task model can outperform the single-task model. can transform

According to an embodiment of the present disclosure, a task weight corresponding to each task in the loss function of the multi-task model of Equation 1 may be defined as Equation 3 as follows.

[Equation 3]

-- ①

-- ①

-- ②

-- ②

)에 대한 각 태스크의 학습 정도에 따른 정확도(

)의 비율에 기초하여, 각 태스크에 대응되는 태스크 가중치를 설정할 수 있다. 이때, 각 태스크의 고유의 정확도(

)는 각 태스크에 대응되는 싱글-태스크 교수 모델의 정확도에 해당할 수 있다. 각 태스크의 고유의 정확도(

)에 대한 각 태스크의 학습 정도에 따른 정확도(

)의 비율은 각 태스크의 학습 정도에 따른 상대적인 학습률을 의미한다. 전자 장치(100)는 각 태스크의 고유의 정확도(

)에 대한 각 태스크의 학습 정도에 따른 정확도(

)의 비율에 반비례하여 각 태스크에 대응되는 태스크 가중치를 조정할 수 있다. 전자 장치(100)는 태스크의 학습 정도에 따라 태스크의 상대적인 학습률이 낮으면, 해당 태스크의 태스크 가중치를 높이고, 태스크의 상대적인 학습률이 높으면, 해당 태스크의 태스크 가중치를 낮출 수 있다. According to the first equation (①) of Equation 3, the electronic device 100 has a unique accuracy of each task (

Accuracy according to the learning degree of each task for ) (

), a task weight corresponding to each task may be set. At this time, the unique accuracy of each task (

) is the single-task teaching model corresponding to each task. may be related to accuracy. Inherent accuracy of each task (

Accuracy according to the learning degree of each task for ) (

) means the relative learning rate according to the learning level of each task. The electronic device 100 has a unique accuracy of each task (

Accuracy according to the learning degree of each task for ) (

), the task weight corresponding to each task can be adjusted in inverse proportion to the ratio. According to the learning degree of the task, the electronic device 100 may increase the task weight of the corresponding task when the relative learning rate of the task is low, and decrease the task weight of the corresponding task when the relative learning rate of the task is high.

The first equation (①) of Equation 3 can be normalized as the second equation (②) of Equation 3. In the present disclosure, the task weight of each task is not limited to the equations shown in Equations 2 and 3, and can be modified in various forms.

Referring back to FIG. 2 , the electronic device 100 performs learning of the multi-task model using the loss function of the multi-task model (S220). To learn the multi-task model, a large-scale multi-task It is necessary to use a training data set, but it is very difficult in terms of time and cost to perform annotations on a large amount of input data and a plurality of tasks. According to an embodiment of the present disclosure, an example of performing learning of a multi-task model using a partially labeled multi-task learning data set will be described in detail with reference to FIGS. 6 to 8 .

6 is a detailed flowchart of a step of performing learning of a multi-task model using a loss function of a multi-task model according to an embodiment of the present disclosure.

The electronic device 100 may input the augmented multi-task learning data set to a single-task teaching model and a multi-task model corresponding to each task (S610). The multi-task learning data set is partially labeled. It can be an input data set. The electronic device 100 may input the augmented multi-task learning data set to not only the multi-task model but also pre-learned single-task teaching models. For example, when the multi-task learning data set is images, augmentation of the images may be performed through transformation such as rotation, flipping, or shifting. Augmented images may be equally input to a single-task teaching model and a multi-task model corresponding to each task.

The single-task teaching model corresponding to each task can generate pseudo labels for tasks for which labels do not exist. In an embodiment of the present disclosure, the output of the single-task teaching model corresponding to each task may be generated as a pseudo label represented in the form of a probability value according to a soft label method. If the hard label method is followed, a problem may occur in which the ground-truth of the output of the single-task teaching model is incorrectly generated. Since such erroneously generated real values can greatly affect the self-learning and accuracy of the multi-task instructional model, in order to minimize the influence, in one embodiment of the present disclosure, the output of the single-task instructional model is set as a soft label. follow the method For example, when the multi-task model is a vision model, when each task is image classification, object detection, and segmentation, pseudo labels are generated according to the soft label method for not only image classification but also object detection and segmentation.

In an embodiment of the present disclosure, in order to more effectively generate pseudo labels according to the soft label method, rather than performing training of a multi-task model by generating pseudo labels in advance, augmented multi-task learning data is multi-tasked. -Pseudo-levels can be created dynamically by inputting not only the task model but also the single-task teaching model corresponding to each task.

The electronic device 100 assigns a task weight corresponding to each task to each task loss corresponding to the difference between the output of the single-task teaching model corresponding to each task and the output of each task of the multi-task model corresponding to the output. Self-learning of the multi-task model may be performed so that the sum of applied weights is minimized (S620).

Each task loss may be defined differently depending on whether there is an actual value of the task. In the case of a task having no actual value, task loss may be determined based on a difference between a pseudo label output from a single-task teaching model corresponding to the task and an output of the multi-task model corresponding to the task. In the case of a task having an actual value, the task loss may be determined based on the difference between the actual value corresponding to the task and each of the pseudo labels output by the single-task teaching model and the output of the multi-task model corresponding to the task. .

7 is a diagram for explaining an example of loss of each task in a process of performing self-learning of a multi-task model according to an embodiment of the present disclosure.

Referring to FIG. 7 , the electronic device 100 augments a partially labeled multi-task learning data set and inputs the augmented multi-task learning data into a multi-task model and a single-task model corresponding to each task. can As a result, the multi-task model outputs output A, output B, and output C as predicted values corresponding to each task, and the single-task model corresponding to each task produces pseudo label A, pseudo label B, and pseudo label. output C.

)과 각 태스크의 태스크 가중치(

)를 획득하여, 멀티-태스크 모델의 손실 함수(

)를 계산할 수 있다. 전자 장치(100)는 역전파(backpropagation)를 통해, 손실 함수(

)가 최소화 되도록 멀티-태스크 모델의 셀프 학습을 수행할 수 있다. The electronic device 100 provides a task loss corresponding to the difference between the output of the single-task model corresponding to each task and the output of each task of the multi-task model respectively corresponding to the output (

) and the task weight of each task (

), the loss function of the multi-task model (

) can be calculated. The electronic device 100 through backpropagation, the loss function (

) can be performed by self-learning of the multi-task model so that

)에서 각 태스크의 태스크 가중치(

)는 앞서 도 3 내지 도 5의 설명에서 살펴본 바와 같이, 각 태스크에 대응되는 싱글-태스크 교수 모델의 정확도에 기초하여 정의될 수 있다. 본 개시의 일 실시예에 따르면, 다음과 같은 수학식 4의 멀티-태스크 모델의 손실 함수(

)에서 각 태스크에 대응되는 태스크 손실은 다음과 같이 정의될 수 있다.According to an embodiment of the present disclosure, the loss function of the multi-task model (

), the task weight of each task (

) may be defined based on the accuracy of the single-task teaching model corresponding to each task, as discussed above in the description of FIGS. 3 to 5. According to an embodiment of the present disclosure, the loss function of the multi-task model of Equation 4 as follows (

), the task loss corresponding to each task can be defined as follows.

[Equation 4]

로 정의될 수 있다.

는 각 태스크에 대응되는 싱글-태스크 교수 모델이 출력한 의사 레이블과 해당 태스크에 대응되는 멀티-태스크 모델의 출력 간의 차이이다. If the actual value of each task is unknown, the task loss corresponding to each task is as shown in Equation 4 above.

can be defined as

is the difference between the pseudo label output by the single-task teaching model corresponding to each task and the output of the multi-task model corresponding to the corresponding task.

8 illustrates an example of loss of each task when an arbitrary task having a ground-truth is included in a process of performing self-learning of a multi-task model according to an embodiment of the present disclosure. It is a drawing for

Referring to FIG. 8 , the electronic device 100 augments a partially labeled multi-task learning data set and inputs the augmented multi-task learning data into a multi-task model and a single-task model corresponding to each task. can Task A and task B have actual values of label A and label C, respectively, and task B has no actual value. The multi-task model outputs output A, output B, and output C as predicted values corresponding to each task, and the single-task model outputs pseudo label A, pseudo label B, and pseudo label C corresponding to each task. do.

)와 각 태스크에 대응되는 실제 값과 이에 각각 대응되는, 멀티-태스크 모델의 출력 간의 차이(

)의 합에 해당하는 태스크 손실(

)과 각 태스크의 태스크 가중치(

)를 획득하여, 멀티-태스크 모델의 손실 함수(

)를 계산할 수 있다. 전자 장치(100)는 역전파(backpropagation)를 통해, 손실 함수(

)가 최소화 되도록 멀티-태스크 모델의 셀프 학습을 수행할 수 있다. The electronic device 100 determines the difference between the output of the single-task model corresponding to each task and the output of each task of the multi-task model respectively corresponding to the output (

) and the difference between the actual value corresponding to each task and the corresponding output of the multi-task model (

) task loss corresponding to the sum of (

) and the task weight of each task (

), the loss function of the multi-task model (

) can be calculated. The electronic device 100 through backpropagation, the loss function (

) can be performed by self-learning of the multi-task model so that

)에서 각 태스크의 태스크 가중치(

)는 앞서 도 3 내지 도 5의 설명에서 살펴본 바와 같이, 각 태스크에 대응되는 싱글-태스크 교수 모델의 정확도에 기초하여 정의될 수 있다. 본 개시의 일 실시예에 따르면, 다음과 같은 수학식 5의 멀티-태스크 모델의 손실 함수(

)에서 각 태스크에 대응되는 태스크 손실은 다음과 같이 표시될 수 있다.According to an embodiment of the present disclosure, the loss function of the multi-task model (

), the task weight of each task (

) may be defined based on the accuracy of the single-task teaching model corresponding to each task, as discussed above in the description of FIGS. 3 to 5. According to an embodiment of the present disclosure, the loss function of the multi-task model of Equation 5 as follows (

), the task loss corresponding to each task can be expressed as follows.

[Equation 5]

로 정의될 수 있다. 실제 값이 없는 태스크의 경우, 해당 태스크에 대응되는 싱글-태스크 교수 모델이 출력한 의사 레이블과 해당 태스크에 대응되는 멀티-태스크 모델의 출력 간의 차이(

)로 해당 태스크의 태스크 손실이 결정될 수 있다. 실제 값이 있는 태스크의 경우, 해당 태스크에 대응되는 실제 값과 해당 태스크에 대응되는 멀티-태스크 모델의 출력 간의 차이(

)와 싱글-태스크 교수 모델이 출력한 의사 레이블과 해당 태스크에 대응되는 멀티-태스크 모델의 출력 간의 차이(

)의 합으로 해당 태스크의 태스크 손실이 결정될 수 있다.If there is a task that knows the actual value among the tasks of the multi-task model, the task loss corresponding to each task is as shown in Equation 5 above.

can be defined as In the case of a task without an actual value, the difference between the pseudo label output by the single-task teaching model corresponding to the task and the output of the multi-task model corresponding to the task (

), the task loss of the corresponding task may be determined. In the case of a task with an actual value, the difference between the actual value corresponding to the task and the output of the multi-task model corresponding to the task (

) and the difference between the pseudo label output by the single-task teaching model and the output of the multi-task model corresponding to the task (

), the task loss of the corresponding task may be determined.

)에 해당하는 해당 임의의 태스크의 태스크 손실에 해당 임의의 태스크에 대응되는 태스크 가중치(

)를 적용한 가중치 합도 최소화되도록, 멀티-태스크 모델의 셀프 학습을 수행하는 것임을 알 수 있다.Comparing Equation 4 and Equation 5, in the case of an arbitrary task having an actual value among each task of the multi-task model, the difference between the actual value and the output of the corresponding arbitrary task (

), the task weight corresponding to the arbitrary task corresponding to the task loss of the corresponding arbitrary task (

), it can be seen that self-learning of the multi-task model is performed so that the sum of the weights applied is also minimized.

9 is a diagram for explaining a result of learning a vision model that performs a multi-task according to a method for learning a multi-task model according to an embodiment of the present disclosure.

Referring to FIG. 9, various methods (weighted sum, GLS, GradNorm, IMTL_G, IMTL, DTP) for learning a multi-task model of a vision model performing object detection, segmentation, and depth estimation, one of the present disclosure A method for learning a multi-task model (PAL, potential-aware multi-task loss) according to an embodiment, and a method for learning a multi-task model according to an embodiment of the present disclosure include a soft label method. It shows the result of learning according to the method (PAL-ST, potential-aware multi-task loss with self training) in which self-learning of the multi-task model is further performed using the pseudo-labels according to the method.

Looking at the total metric, it can be seen that PAL and PAL-ST according to an embodiment of the present disclosure have relatively high accuracy because the entire task is learned in a balanced manner, compared to other methods. It can be seen that PAL-ST shows better learning results by performing more self-learning of the multi-task model than PAL. Compared to the method using each single teaching model for each task, PAL-ST achieves a high accuracy of 97.83%, but has the effect of significantly reducing the amount of computation.

10 is a block diagram illustrating the configuration of an electronic device 100 according to an embodiment of the present disclosure.

Referring to FIG. 10 , an electronic device 100 according to an embodiment may include a memory 110 and a processor 120.

The memory 110 may store instructions, data structures, and program codes readable by the processor 120 . Operations performed by the processor 120 may be implemented by executing instructions or codes of a program stored in the memory 110 .

The memory 110 includes a flash memory type, a hard disk type, a multimedia card micro type, and a card type memory (eg SD or XD memory). Non-volatile memory including at least one of ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk, and optical disk and volatile memory such as RAM (Random Access Memory) or SRAM (Static Random Access Memory).

The memory 110 according to an embodiment may store one or more instructions and/or programs for controlling the electronic device 100 to perform learning of the multi-task model. For example, a model learning module may be stored in the memory 110 .

The processor 120 may control operations or functions performed by the electronic device 100 by executing instructions stored in the memory 110 or programmed software modules. The processor 120 may be composed of hardware components that perform arithmetic, logic and input/output operations and signal processing. For example, the processor 120 may call and execute a model learning module stored in the memory 110 .

The processor 120 may include, for example, a central processing unit, a microprocessor, a graphic processing unit, application specific integrated circuits (ASICs), digital signal processors (DSPs), DSPDs ( Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), Application Processors, Neural Processing Units, or artificial intelligence designed with hardware structures specialized in processing AI models. It may be configured with at least one of dedicated processors, but is not limited thereto. Each processor constituting the processor 120 may be a dedicated processor for performing a predetermined function.

The processor 120 executes one or more instructions stored in the memory 110, so that the electronic device 100 performs learning of the multi-task model or overall operation for processing a task using the multi-task model. can control them.

The processor 120 according to an embodiment of the present disclosure executes one or more instructions stored in the memory 110, based on the unique accuracy of each task of the multi-task model and the accuracy according to the learning degree of each task, Set the task weight corresponding to each task of the loss function of the multi-task model. The loss function of the multi-task model may correspond to a weighted sum obtained by applying a task weight corresponding to each task to a loss of each task of the multi-task model.

The processor 120 according to an embodiment may set a task weight corresponding to each task based on an accuracy difference between an accuracy inherent in each task and an accuracy according to a learning degree of each task. The processor 120 may adjust a task weight corresponding to each task in proportion to the difference in accuracy. The processor 120 according to an embodiment may set a task weight corresponding to each task based on a ratio of accuracy according to the learning degree of each task to the unique accuracy of each task. The processor 120 may adjust the task weight corresponding to each task in inverse proportion to the ratio of accuracy according to the learning degree of each task to the unique accuracy of each task.

Processor 120 according to an embodiment of the single-task teaching model corresponding to each task Accuracy is obtained, and the obtained single-task instruction model corresponding to each task Accuracy can be used as the specific accuracy of each task. The processor 120 may perform learning of a single-task teaching model corresponding to each task by using a dedicated learning data set corresponding to each task. The processor 120 may obtain the accuracy of the single-task teaching model corresponding to each task by measuring the accuracy of the learned single-task teaching model corresponding to each task upon completion of learning.

The processor 120 according to an embodiment performs learning of the multi-task model by using a loss function of the multi-task model by executing one or more instructions stored in the memory 110 . The processor 120 may input the augmented multi-task learning data set to a single-task teaching model and a multi-task model respectively corresponding to each task of the multi-task model. The processor 120 assigns each task loss corresponding to the difference between the output of the single-task teaching model corresponding to each task and the output of each task of the multi-task model, respectively, corresponding to the output of the single-task teaching model. Self-learning of the multi-task model may be performed so that the sum of weights to which task weights corresponding to is applied is minimized. An output of the single-task teaching model corresponding to each task may be a pseudo label represented in the form of a probability value according to a soft label method.

In the case of an arbitrary task having an actual value among tasks of the multi-task model, the processor 120 determines the task loss of the arbitrary task corresponding to the difference between the actual value and the output of the arbitrary task and the task corresponding to the arbitrary task. Self-learning of the multi-task model may be performed so that the sum of the weights to which the weights are applied is also minimized. In the case of a task for which the actual value is known, task loss may be determined by comparing the actual value corresponding to the task and each of the pseudo labels output by the single-task teaching model with the output of the multi-task model corresponding to the task. In the case of a task whose actual value is unknown, task loss may be determined by comparing a pseudo label output from a single-task teaching model corresponding to the task with an output of the multi-task model corresponding to the task.

11 is a diagram illustrating an example of an electronic device 100 according to an embodiment of the present disclosure.

When the multi-task model loaded in the electronic device 100 is a vision model, as shown in FIG. 11 , the electronic device 100 according to an embodiment includes a memory 110, a processor 120, and a camera 130. ), a display 140, and a communication interface 150.

Regarding the memory 110 and the processor 120, descriptions of overlapping contents with those described in FIG. 10 are excluded below. A model learning module and a model execution module may be stored in the memory 110 . The processor 120 may load and execute a model learning module or a model execution module from the memory 110 .

The camera 130 is a hardware module that acquires images. The camera 130 may capture images such as photos or videos. The camera 130 may include at least one camera module, and may support macro, depth-of-field, telephoto, wide-angle, and ultra-wide-angle functions according to specifications of the electronic device 100 .

The display 140 includes an output unit for providing information or images, and may further include an input unit for receiving an input. The output unit may include a display panel and a controller controlling the display panel, and may include a variety of display devices such as OLED (Organic Light Emitting Diodes) displays, AM-OLED (Active-Matrix Organic Light-Emitting Diodes) displays, LCD (Liquid Crystal Display) displays, and the like. can be implemented in this way. The input unit may receive various types of inputs from a user, and may include at least one of a touch panel, a keypad, and a pen recognition panel. The display 140 may be provided in the form of a touch screen in which a display panel and a touch panel are combined, and may be implemented in a flexible or foldable manner.

The communication interface 150 may perform wired/wireless communication with other devices or networks. The communication interface 150 may include a communication circuit or communication module supporting at least one of various wired/wireless communication methods. For example, the communication interface 150 may be, for example, wired LAN, wireless LAN, Wi-Fi, Bluetooth, ZigBee, Wi-Fi Direct (WFD), infrared Communication (IrDA, Infrared Data Association), BLE (Bluetooth Low Energy), NFC (Near Field Communication), WiBro (Wireless Broadband Internet, Wibro), WiMAX (World Interoperability for Microwave Access, WiMAX), SWAP (Shared Wireless Access Protocol) Data communication between the electronic device 100 and other devices may be performed using at least one of data communication methods including WiGig, Wireless Gigabit Alliances (RF), and RF communication.

The communication interface 150 according to an embodiment transmits and receives an artificial intelligence model (eg, a multi-task model or a single-task model) used to learn a multi-task model or a data set for learning with an external device. can do. The communication interface 150 may transmit an image captured through the camera 130 of the electronic device 100 to a server (not shown) or receive an artificial intelligence model or data set learned by the server (not shown). .

The processor 120 or a first processor constituting the processor 120 may execute a model learning module to learn a multi-task model. Learning of the multi-task model may be performed using multi-task learning data. The processor 120 or the first processor may execute a model learning module to learn a single task module corresponding to each task of the multi-task model. The learned single task module can be used to obtain unique accuracy corresponding to each task, and can also be used for self-learning of a multi-task model.

The processor 120 or the first processor may train a multi-task model or a single-task model using the training data set. The processor 120 or the first processor may learn a multi-task model or a single-task model by using learning data related to criteria used for processing each task. The training data set may be classified according to data attribute information such as data generation time or place, size, creator, and data type, and may be used for model learning.

For example, when the multi-task model or the single-task model is a vision model, the processor 120 or the first processor may acquire a plurality of images for model learning. The processor 120 or the first processor may acquire an image through the camera 130 of the electronic device 100 or acquire an image from an external device through the communication interface 150 . The processor 120 or the first processor may perform a pre-processing process on the obtained image or select input data for learning a model from a training data set.

The processor 120 or the first processor may train the multi-task model or the single-task model using a learning algorithm including error back-propagation or gradient descent.

The processor 120 or a second processor constituting the processor 120 may execute a model execution module to process each task of the multi-task model. The processor 120 or the second processor may obtain input data necessary for processing each task. The processor 120 or the second processor may input input data to the multi-task model and output a result corresponding to each task. The learned multi-task model may output prediction or inference values corresponding to each task in response to inputs corresponding to each task. The processor 120 or the second processor may provide a predetermined service through an additional processing operation on an output value.

Unlike shown in FIG. 11 , one of the model learning module and the model execution module may be installed in the electronic device 100 and the other may be installed in the server. In this case, the electronic device 100 and the server may provide information about a model built in a model learning module loaded in one device to a model execution module loaded in another device through communication. Alternatively, the electronic device 100 and the server may provide a data set input to a model execution module loaded in one device to a model learning module loaded in another device through communication.

Meanwhile, the embodiments of the present disclosure may be implemented in the form of a recording medium including instructions executable by a computer, such as program modules executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer readable media may include computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media may typically include computer readable instructions, data structures, or other data in a modulated data signal such as program modules.

Also, the computer-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-temporary storage medium' only means that it is a tangible device and does not contain signals (e.g., electromagnetic waves), and this term refers to the case where data is stored semi-permanently in the storage medium and temporary It does not discriminate if it is saved as . For example, a 'non-temporary storage medium' may include a buffer in which data is temporarily stored.

According to one embodiment, the method according to various embodiments disclosed in this document may be provided by being included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. A computer program product is distributed in the form of a device-readable storage medium (eg compact disc read only memory (CD-ROM)), or through an application store or between two user devices (eg smartphones). It can be distributed (e.g., downloaded or uploaded) directly or online. In the case of online distribution, at least a part of a computer program product (eg, a downloadable app) is stored on a device-readable storage medium such as a memory of a manufacturer's server, an application store server, or a relay server. It can be temporarily stored or created temporarily.

According to an embodiment of the present disclosure, a method for performing learning of a multi-task model is provided. A method for performing learning of the multi-task model is based on the accuracy of each task of the multi-task model and the accuracy according to the learning degree of each task, and the task corresponding to each task corresponds to the loss of each task of the multi-task model. and setting a task weight corresponding to each task of a loss function of the multi-task model corresponding to a weight sum to which weights have been applied. Also, the method of performing learning of the multi-task model includes performing learning of the multi-task model by using a loss function of the multi-task model.

In addition, according to an embodiment of the present disclosure, the step of setting a task weight corresponding to each task may include corresponding to each task based on an accuracy difference between an accuracy inherent in each task and an accuracy according to a learning degree of each task. Set the task weight to be

In addition, the step of setting the task weight corresponding to each task adjusts the task weight corresponding to each task in proportion to a difference in accuracy between the unique accuracy of each task and the accuracy according to the learning degree of each task.

In addition, according to an embodiment of the present disclosure, setting a task weight corresponding to each task may include corresponding to each task based on a ratio of accuracy according to a learning degree of each task to unique accuracy of each task. Set the task weight to be

In the step of setting the task weight corresponding to each task, the task weight corresponding to each task is adjusted in inverse proportion to the ratio of the accuracy according to the learning degree of each task to the unique accuracy of each task.

In addition, according to an embodiment of the present disclosure, the step of setting task weights corresponding to each task may include a single-task teaching model corresponding to each task. and obtaining accuracy. In addition, the step of setting the task weight corresponding to each task may include the obtained single-task teaching model corresponding to each task. and setting a task weight corresponding to each task of the loss function of the multi-task model based on the accuracy and the accuracy according to the learning degree of each task.

In addition, the step of acquiring the accuracy of the single-task teaching model corresponding to each task includes the step of learning the single-task teaching model corresponding to each task using a dedicated training data set corresponding to each task. include Also, acquiring the accuracy of the single-task teaching model corresponding to each task includes measuring the accuracy of the learned single-task teaching model corresponding to each task when learning is completed.

In addition, according to an embodiment of the present disclosure, the step of learning the multi-task model includes inputting the augmented multi-task learning data set to a single-task teaching model and a multi-task model corresponding to each task, respectively. It includes steps to In addition, the step of performing learning of the multi-task model may include a difference between the output of the single-task teaching model corresponding to each task and the output of each task of the multi-task model corresponding to the output of the single-task teaching model. and performing self-learning of the multi-task model so that a weight sum obtained by applying a task weight corresponding to each task to each task loss corresponding to is minimized. In addition, the output of the single-task teaching model corresponding to each task is a pseudo label represented in the form of a probability value according to the soft label method.

In addition, the step of performing self-learning of the multi-task model may include, in the case of an arbitrary task having an actual value among tasks of the multi-task model, a random task corresponding to a difference between the actual value and the output of the arbitrary task. Self-learning of the multi-task model is performed so that the weight sum obtained by applying the task weight corresponding to an arbitrary task to the task loss is also minimized.

According to one embodiment of the present disclosure, it is possible to provide a computer-readable recording medium on which a program for executing the above-described method is recorded.

According to an embodiment of the present disclosure, an electronic device 100 performing learning of a multi-task model is provided. The electronic device 100 that performs learning of the multi-task model includes a memory 110 that stores one or more instructions and a processor 120 that executes one or more instructions stored in the memory 110 . By executing one or more instructions, the processor 120 calculates a loss corresponding to each task of the multi-task model based on the unique accuracy of each task of the multi-task model and the accuracy according to the learning degree of each task. Set the task weight corresponding to each task of the loss function of the multi-task model corresponding to the weight sum to which the task weights are applied. Also, the processor 120 performs learning of the multi-task model by using the loss function of the multi-task model by executing one or more instructions.

In addition, according to an embodiment of the present disclosure, the processor 120 executes one or more instructions, and based on the difference in accuracy between the unique accuracy of each task and the accuracy according to the learning degree of each task, the processor 120 corresponds to each task. Set task weight.

Also, by executing one or more instructions, the processor 120 adjusts a task weight corresponding to each task in proportion to a difference in accuracy between the unique accuracy of each task and the accuracy according to the learning degree of each task.

In addition, according to an embodiment of the present disclosure, the processor 120 executes one or more instructions, and based on the ratio of the accuracy according to the learning degree of each task to the unique accuracy of each task, the processor 120 corresponds to each task. Set task weight.

Also, by executing one or more instructions, the processor 120 adjusts the task weight corresponding to each task in inverse proportion to the ratio of the accuracy according to the learning degree of each task to the unique accuracy of each task.

In addition, according to an embodiment of the present disclosure, the processor 120 executes one or more instructions to generate a single-task instructional model corresponding to each task. Accuracy is obtained, and the obtained single-task instruction model corresponding to each task Accuracy is used as the intrinsic accuracy of each task.

In addition, the processor 120 performs learning of a single-task teaching model corresponding to each task by using a dedicated training data set corresponding to each task by executing one or more instructions. In addition, the processor 120 measures the accuracy of the learned single-task teaching model corresponding to each task upon completion of learning by executing one or more instructions, thereby determining the accuracy of the single-task teaching model corresponding to each task. Acquire

In addition, according to an embodiment of the present disclosure, the processor 120 executes one or more instructions to input the augmented multi-task learning data set to a single-task teaching model and a multi-task model corresponding to each task, respectively. do. In addition, the processor 120 executes one or more instructions, so that the difference between the output of the single-task instruction model corresponding to each task and the output of each task of the multi-task model corresponding to the output of the single-task instruction model. Self-learning of the multi-task model is performed so that the weight sum obtained by applying the task weight corresponding to each task to each task loss corresponding to is minimized.

In addition, the output of the single-task teaching model corresponding to each task is a pseudo label represented in the form of a probability value according to the soft label method.

In addition, the processor 120 executes one or more instructions, so that, in the case of an arbitrary task having an actual value among each task of the multi-task model, the task of the arbitrary task corresponding to the difference between the actual value and the output of the arbitrary task. Self-learning of the multi-task model is performed so that the weight sum obtained by applying the task weight corresponding to an arbitrary task to the loss is also minimized.

The description of the present disclosure described above is for illustrative purposes, and those skilled in the art can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present disclosure. will be. Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present disclosure is indicated by the following claims rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts should be interpreted as being included in the scope of the present disclosure. do.

Claims (20)

In the method of performing the learning of the multi-task model,
Based on the unique accuracy of each task of the multi-task model and the accuracy according to the learning degree of each task, corresponding to the weight sum obtained by applying the task weight corresponding to each task to the loss of each task of the multi-task model. Setting the task weight corresponding to each task of the loss function of the multi-task model (S210); and
Performing learning of the multi-task model using the loss function (S220);
Including, method. According to claim 1,
In the step of setting the task weight corresponding to each task (S210),
The method of setting the task weight corresponding to each task based on an accuracy difference between the unique accuracy of each task and the accuracy according to the learning degree of each task. According to claim 2,
In the step of setting the task weight corresponding to each task (S210),
Adjusting the task weight corresponding to each task in proportion to the accuracy difference. According to claim 1,
In the step of setting the task weight corresponding to each task (S210),
The method of setting the task weight corresponding to each task based on the ratio of the accuracy according to the learning degree of each task to the unique accuracy of each task. According to claim 4,
In the step of setting the task weight corresponding to each task (S210),
Adjusting the task weight corresponding to each task in inverse proportion to the ratio. According to any one of claims 1 to 5,
In the step of setting the task weight corresponding to each task (S210),
of the single-task teaching model corresponding to each task Acquiring accuracy (S310); and
of the obtained single-task teaching model corresponding to each task and setting the task weight corresponding to each task of the loss function of the multi-task model based on the accuracy and the accuracy according to the learning degree of each task (S320). According to claim 6,
Acquiring the accuracy of the single-task teaching model corresponding to each task (S310),
performing learning of the single-task teaching model corresponding to each task by using a dedicated learning data set corresponding to each task (S410); and
and measuring an accuracy of the learned single-task teaching model corresponding to each task when learning is completed (S420, S430). According to any one of claims 1 to 7,
In the step of performing the learning of the multi-task model (S220),
inputting the augmented multi-task learning data set to the single-task teaching model and the multi-task model corresponding to each task (S610); and
The loss of each task corresponding to the difference between the output of the single-task instruction model corresponding to each task and the output of each task of the multi-task model, respectively, corresponding to the output of the single-task instruction model. Performing self-learning of the multi-task model so that the sum of the weights to which the task weights corresponding to each task is applied is minimized (S620);
The output of the single-task teaching model corresponding to each task is a pseudo label represented in the form of a probability value according to a soft label method. According to claim 8,
Performing self-learning of the multi-task model (S620),
In the case of an arbitrary task having a true value (ground-truth) among the respective tasks of the multi-task model, the task loss of the arbitrary task corresponding to the difference between the actual value and the output of the arbitrary task corresponds to the ground-truth. A method for performing self-learning of the multi-task model so that the sum of weights obtained by applying task weights corresponding to the tasks of is also minimized. A computer-readable recording medium recording a program for executing the method of any one of claims 1 to 9 on a computer. In the electronic device 100 that performs learning of the multi-task model,
memory 110 for storing one or more instructions; and
a processor (120) to execute the one or more instructions stored in the memory (110);
The processor 120, by executing the one or more instructions,
Based on the unique accuracy of each task of the multi-task model and the accuracy according to the learning degree of each task, corresponding to the weight sum obtained by applying the task weight corresponding to each task to the loss of each task of the multi-task model. The electronic device 100 configured to set the task weight corresponding to each task of the loss function of the multi-task model and perform learning of the multi-task model using the loss function. According to claim 11,
The processor 120, by executing the one or more instructions,
The electronic device 100 that sets the task weight corresponding to each task based on an accuracy difference between accuracy inherent in each task and accuracy according to a learning degree of each task. According to claim 12,
The processor 120, by executing the one or more instructions,
The electronic device 100 that adjusts the task weight corresponding to each task in proportion to the accuracy difference. According to claim 11,
The processor 120, by executing the one or more instructions,
The electronic device 100 that sets the task weight corresponding to each task based on a ratio of accuracy according to the learning degree of each task to the unique accuracy of each task. According to claim 14,
By executing the one or more instructions, the processor:
The electronic device 100 that adjusts the task weight corresponding to each task in inverse proportion to the ratio. According to any one of claims 11 to 15,
The processor 120, by executing the one or more instructions,
of the single-task teaching model corresponding to each task Accuracy is obtained, and the obtained single-task instruction model corresponding to each task The electronic device 100, using accuracy as the unique accuracy of each task. According to claim 16,
The processor 120, by executing the one or more instructions,
Learning of the single-task teaching model corresponding to each task is performed using a dedicated learning data set corresponding to each task, and learning of the learned single-task teaching model corresponding to each task The electronic device 100, which obtains the accuracy of the single-task teaching model corresponding to each task by measuring the accuracy upon completion. According to any one of claims 11 to 17,
The processor 120, by executing the one or more instructions,
The augmented multi-task learning data set is input to the single-task teaching model and the multi-task model corresponding to each task, and the output of the single-task teaching model corresponding to each task and the single-task The multi-task model minimizes the sum of the weights obtained by applying the task weight corresponding to each task to the task loss corresponding to the difference between the outputs of the respective tasks of the multi-task model, respectively corresponding to the outputs of the teaching model. - The electronic device 100, which performs self-learning of the task model. According to claim 18,
The electronic device 100, wherein the output of the single-task teaching model corresponding to each task is a pseudo label represented in the form of a probability value according to a soft label method. According to claim 18,
The processor 120, by executing the one or more instructions,
In the case of an arbitrary task having an actual value among the respective tasks of the multi-task model, the task loss of the arbitrary task corresponding to the difference between the actual value and the output of the arbitrary task corresponds to the arbitrary task The electronic device 100 that performs self-learning of the multi-task model so that the sum of weights to which task weights are applied is also minimized.

Images