KR20230032690A - Method for training multi-task model - Google Patents

Abstract

According to several embodiments of the present disclosure, a method for training a multi-task model, which is performed by a computing device comprising at least one processor, is disclosed. The training method for the multi-task model comprises: acquiring a step of a training data set; and a step of, based on the training data set, training a neural network model which outputs a prediction result for an input value and estimates the uncertainty of the prediction. A loss function for training the neural network model may include a first loss function for quantifying the prediction result and the uncertainty of the prediction, and a second loss function for improving the prediction accuracy of the neural network model. Therefore, the training method for the multi-task model can perform accurate predictions and inferences and effectively quantify the uncertainty of predictions.

Description

Learning method of multi-task model {METHOD FOR TRAINING MULTI-TASK MODEL}

The present disclosure relates to a method for learning a multi-task model, and more specifically, to a method for training a neural network model that outputs a prediction result for an input value and estimates the uncertainty of the prediction.

In addition to making accurate predictions and inferences, quantifying uncertainty can be an essential task for safety-critical systems. Factors contributing to uncertainty in deep learning can be classified into two types: irreducible observational noise (i.e. metaphorical uncertainty) and uncertainty in model parameters (i.e. perceptual uncertainty). In particular, quantification of perceptual uncertainty presents a problem in that it is accompanied by many difficulties in expressing the uncertainty of model parameters and requires considerable cost.

Existing methods for estimating perceptual uncertainty include an ensemble-based method and a method using Bayesian neural networks. Ensemble-based methods and Bayesian network-based methods are known to produce impressive results both in estimation accuracy and robustness. However, since the ensemble-based method requires a large number of network models, it has a disadvantage that many resources are inevitably required for calculation. In addition, since the Bayesian neural network network requires expansive approximations for a posterior distribution or probability, there is a disadvantage in that it is inefficient in terms of cost of operation.

Republic of Korea Patent No. 10-2213670

The present disclosure has been made in response to the above background art, and aims to provide a method for learning a multi-task model capable of performing accurate prediction and inference and at the same time effectively quantifying the uncertainty of prediction.

The technical problems of the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

According to one embodiment of the present disclosure for solving the above problems, a method for learning a multi-task model performed by a computing device including at least one processor is disclosed. The learning method of the multi-task model includes acquiring a learning data set; and training a neural network model that outputs a prediction result for an input value and estimates uncertainty of the prediction based on the learning data set. The loss function for learning the neural network model may include a first loss function for quantifying the prediction result and uncertainty of the prediction, and a second loss function for improving prediction accuracy of the neural network model. there is.

In addition, the neural network model may be a model based on an Evidential Regression Network (ENet).

In addition, the first loss function may include a negative log-likelihood (NLL) based loss function.

Also, the second loss function may include a mean squared error (MSE)-based loss function.

In addition, the second loss function may include a first sub-loss function based on a mean square error (MSE); and a second sub-loss function based on a Lipschitz-continuous function derived to mitigate a gradient conflict between the first loss function and the second loss function; can include

In addition, the first sub-loss function or the second sub-loss function may be obtained by comparing an operation value of the first sub-loss function with a threshold value based on a variance parameter of the neural network model related to the uncertainty of the prediction. Depending on the result, the second loss function may be selectively used for learning the neural network model.

In addition, when the calculated value of the first sub-loss function is less than a threshold value based on the variance parameter of the neural network model related to the uncertainty of the prediction, the first sub-loss function as the second loss function is used for learning of the neural network model. can be used

In addition, when the calculated value of the first sub-loss function is greater than or equal to a threshold value based on the variance parameter of the neural network model related to the uncertainty of the prediction, the second sub-loss function as the second loss function is used for learning of the neural network model. can be used

Also, the threshold value based on the variance parameter of the neural network model associated with the uncertainty of the prediction may be an average value of minimum values of the variance parameter.

In addition, the loss function for learning the neural network model may include a third loss function for regulating the first loss function by increasing the uncertainty of the prediction; may further include.

In addition, as a computer program stored on a computer-readable storage medium, the computer program, when executed on one or more processors, performs a method for learning a multi-task model, the method comprising: learning data obtaining a set; and training a neural network model that outputs a prediction result for an input value and estimates uncertainty of the prediction based on the learning data set. The loss function for learning the neural network model may include a first loss function for quantifying the prediction result and uncertainty of the prediction, and a second loss function for improving prediction accuracy of the neural network model. can

Also, as a computing device for learning a multi-task model, the communication unit for acquiring a learning data set; and a processor that trains a neural network model that outputs a prediction result for an input value and estimates an uncertainty of the prediction based on the training data set. The loss function for learning the neural network model may include a first loss function for quantifying the prediction result and uncertainty of the prediction, and a second loss function for improving prediction accuracy of the neural network model. can

The technical solutions obtainable in the present disclosure are not limited to the above-mentioned solutions, and other solutions not mentioned will become clear to those skilled in the art from the description below. You will be able to understand.

According to some embodiments of the present disclosure, it is possible to provide a method for learning a multi-task model capable of performing accurate prediction and inference and at the same time effectively quantifying the uncertainty of prediction.

Effects obtainable in the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below. .

Various aspects are now described with reference to the drawings, wherein like reference numbers are used to collectively refer to like elements. In the following embodiments, for explanation purposes, numerous specific details are set forth in order to provide a thorough understanding of one or more aspects. However, it will be apparent that such aspect(s) may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to facilitate describing one or more aspects.
1 is a block diagram illustrating an example of a computing device according to some embodiments of the present disclosure.
2 is a flowchart illustrating an example of a method for a computing device to learn a neural network model, according to some embodiments of the present disclosure.
3 is a diagram for explaining an example of tilt collision according to some embodiments of the present disclosure.
4 is a flowchart illustrating an example of a method of selectively using a first sub loss function or a second sub loss function as a second loss function by a computing device according to some embodiments of the present disclosure.
5 is a diagram for explaining an example of a parameter according to some embodiments of the present disclosure.
6 is a diagram for explaining the results of an experiment performed using a neural network model according to some embodiments of the present disclosure.
7 depicts a general schematic diagram of an example computing environment in which embodiments of the present disclosure may be implemented.

Various embodiments and/or aspects are now disclosed with reference to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth in order to facilitate a general understanding of one or more aspects. However, it will also be appreciated by those skilled in the art that such aspect(s) may be practiced without these specific details. The following description and accompanying drawings describe in detail certain illustrative aspects of one or more aspects. However, these aspects are exemplary and some of the various methods in principle of the various aspects may be used, and the described descriptions are intended to include all such aspects and their equivalents. Specifically, “embodiment,” “example,” “aspect,” “exemplary,” etc., as used herein, is not to be construed as indicating that any aspect or design described is superior to or advantageous over other aspects or designs. Maybe not.

Hereinafter, the same reference numerals are given to the same or similar components regardless of reference numerals, and overlapping descriptions thereof will be omitted. In addition, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the embodiment disclosed in this specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the accompanying drawings.

Although first, second, etc. are used to describe various elements or components, these elements or components are not limited by these terms, of course. These terms are only used to distinguish one element or component from another. Accordingly, it goes without saying that the first element or component mentioned below may also be the second element or component within the technical spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used in a meaning commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless explicitly specifically defined.

In addition, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or”. That is, unless otherwise specified or clear from the context, “X employs A or B” is intended to mean one of the natural inclusive substitutions. That is, X uses A; X uses B; Or, if X uses both A and B, "X uses either A or B" may apply to either of these cases. Also, the term "and/or" as used herein should be understood to refer to and include all possible combinations of one or more of the listed related items.

Also, the terms "comprises" and/or "comprising" mean that the feature and/or element is present, but excludes the presence or addition of one or more other features, elements and/or groups thereof. It should be understood that it does not. Also, unless otherwise specified or where the context clearly indicates that a singular form is indicated, the singular in this specification and claims should generally be construed to mean "one or more".

In addition, the terms "information" and "data" used herein may often be used interchangeably with each other.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when a component is referred to as “directly connected” or “directly connected” to another component, it should be understood that no other component exists in the middle.

The suffixes "module" and "unit" for components used in the following description are given or used interchangeably in consideration of ease of writing the specification, and do not have meanings or roles that are distinct from each other by themselves.

Objects and effects of the present disclosure, and technical configurations for achieving them will become clear with reference to embodiments described later in detail in conjunction with the accompanying drawings. In describing the present disclosure, if it is determined that a detailed description of a known function or configuration may unnecessarily obscure the gist of the present disclosure, the detailed description will be omitted. In addition, terms to be described below are terms defined in consideration of functions in the present disclosure, which may vary according to the intention or custom of a user or operator.

However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in a variety of different forms. These embodiments are provided only to make this disclosure complete and to completely inform those skilled in the art of the scope of the disclosure, and the disclosure is only defined by the scope of the claims. . Therefore, the definition should be made based on the contents throughout this specification.

를 파라미터

가 알려지지 않은 정규 분포(Normal distribution)로부터 추출한 표본으로 결정할 수 있다. 여기서, 파라미터

는 상기 정규 분포의 이전 켤레(conjugate)인 정규 역 감마 분포(Normal-Inverse-Gamma distribution, NIG distribution)로부터 추출될 수 있다. 타겟값

및 파라미터

는 하기의 수학식과 같이 정의될 수 있다.In the present disclosure, a computing device may learn a multi-task model that outputs a prediction result for an input value and estimates the uncertainty of the prediction. Here, the multi-task model may be a model in which multi-task learning is trained multiple times as a single neural network model. As an example, the neural network model of the present disclosure may be a model based on an Evidential Regression Network (ENet), and the Evidential Regression Network may output prediction results for input values and estimate uncertainty of prediction. For example, a neural network model based on an evidence regression network is a target value

parameter

can be determined by a sample drawn from an unknown normal distribution. Here, the parameter

may be extracted from a Normal-Inverse-Gamma distribution (NIG distribution), which is a previous conjugate of the normal distribution. target value

and parameters

Can be defined as the following equation.

는

을 포함할 수 있다.

는 입력일 수 있고,

는 입력 벡터의 차원일 수 있다.

은 타겟이고,

은 데이터 샘플의 수일 수 있다.

일 수 있고,

일 수 있다. 또한,

일 수 있고,

은 정규 역 감마 분포일 수 있다. 상기 수학식 1에서, 정규 역 감마 분포는 증거 회귀 네트워크의 출력인

에 의해 파라미터화 될 수 있다. 증거 회귀 네트워크의 출력은

과 같이 결정될 수 있다. 여기서,

는 학습가능한 파라미터일 수 있다.data set here

Is

can include

can be an input,

may be the dimension of the input vector.

is the target,

may be the number of data samples.

can be,

can be also,

can be,

may be an inverse normal gamma distribution. In Equation 1, the normal inverse gamma distribution is the output of the evidence regression network

can be parameterized by The output of the evidence regression network is

can be determined as here,

may be a learnable parameter.

에 대한 한계 가능도(marginal likelihood)일 수 있다. 한계 가능도

는 하기의 수학식 2를 통해 계산될 수 있다.The predictive distribution of an evidence regression network is an unknown parameter

may be the marginal likelihood for . marginal likelihood

Can be calculated through Equation 2 below.

은 위치 파라미터

, 스케일 파라미터

및 자유도

포함하는 학습-t 분포일 수 있다. 수학식 2를 통해 계산되는 예측 분포

를 이용하여, 모델 예측, 예측의 불확실성 및 의사(pseudo) 관찰 등이 결정될 수 있다. 일례로, 증거 회귀 네트워크의 모델 예측(또는 예측 결과)

, 비유적 불확실성

및 인식적 불확실성

이 하기의 수학식 3을 통해 결정될 수 있다.here,

is the positional parameter

, the scale parameter

and degrees of freedom

It can be a learning-t distribution that contains The predicted distribution calculated through Equation 2

Using , model prediction, uncertainty of prediction, and pseudo observation can be determined. As an example, model predictions (or prediction results) of evidence regression networks

, figurative uncertainty

and cognitive uncertainty

This can be determined through Equation 3 below.

일 수 있다. 그리고, 예측의 불확실성은

및

일 수 있다. 실시예에 따라, 의사 관찰은 베이지안 통계 문헌에서 사용되는 예측 불확실성의 대체 해석으로서 사용될 수도 있다. 일례로, 본 개시에서는

및

와 같이 의사 관찰을 정의할 수 있고, 비유적 불확실성 및 인식적 불확실성은 의사 관찰이 증가함에 따라 감소될 수 있다. 환언하자면, 비유적 불확실성은 의사 관찰의 증가와는 반비례

할 수 있고, 인식적 불확실성 또한 의사 관찰의 증가와는 반비례

할 수 있다.Based on Equation 3 above, the prediction result of the prediction of the input value by the neural network model in the present disclosure is

can be And, the uncertainty of the forecast is

and

can be Depending on the embodiment, pseudo-observation may be used as an alternative interpretation of predictive uncertainty used in the Bayesian statistical literature. For example, in this disclosure

and

We can define pseudo-observation as, and metaphorical and epistemic uncertainties can be reduced as pseudo-observation increases. In other words, metaphorical uncertainty is inversely proportional to the increase in physician observation.

and cognitive uncertainty is also inversely proportional to the increase in physician observation.

can do.

가 최소화될 수 있는 파라미터를 추정할 수 있다.On the other hand, the neural network model according to the present disclosure is a loss function (Negative Log-Likelihood loss function, NLL log function) through Equation 4 below (

A parameter that can be minimized can be estimated.

은 감마 함수일 수 있고,

일 수 있다. 본 개시의 신경망 모델은 손실함수를 이용하여, 예측 결과

를 계산할 수 있고 또한, 비유적 불확실성

및 인식적 불확실성을 정량화할 수 있다. 더하여, 본 개시에서는 수학식 4를 통해 결정되는 손실함수 외 추가적인 손실함수를 이용하여, 신경망 모델을 학습시킬 수 있다. 일례로, 본 개시에 따른 신경망 모델을 학습시키기 위한 손실함수는 상술한 예측 결과 및 예측의 불확실성을 정량화 하기 위한 제 1 손실함수 외에도 신경망 모델의 예측 정확도를 향상시키기 위한 제 2 손실함수를 포함할 수 있다. 이하, 본 개시에 따른 신경망 모델을 학습시키기 위한 손실함수에 대해 도 1 내지 도 6을 통해 설명한다.here,

may be a gamma function,

can be The neural network model of the present disclosure uses a loss function to predict the result

can be calculated and also, the figurative uncertainty

and quantify perceptual uncertainty. In addition, in the present disclosure, a neural network model may be trained using an additional loss function other than the loss function determined through Equation 4. For example, a loss function for training a neural network model according to the present disclosure may include a second loss function for improving the prediction accuracy of the neural network model in addition to the first loss function for quantifying the prediction result and uncertainty of the prediction described above. there is. Hereinafter, a loss function for learning a neural network model according to the present disclosure will be described with reference to FIGS. 1 to 6.

1 is a block diagram illustrating an example of a computing device according to some embodiments of the present disclosure.

Referring to FIG. 1 , a computing device 100 may include a processor 110 and a storage unit 120 . However, since the above-described components are not essential to implement the computing device 100, the computing device 100 may have more or fewer components than the components listed above.

Computing device 100 may include any type of computer system or computer device, such as, for example, a microprocessor, mainframe computer, digital processor, portable device or device controller, and the like.

The processor 110 may typically process overall operations of the computing device 100 . The processor 110 processes signals, data, information, etc. input or output through components included in the computing device 100 or runs an application program stored in the storage unit 120 to provide information or functions appropriate to the user. can be provided or processed.

In the present disclosure, the processor 110 may train a neural network model that outputs a prediction result for an input value and estimates the uncertainty of the prediction based on the training data set obtained through the communication unit 130 . Here, the loss function for training the neural network model may include a first loss function for quantifying prediction results and prediction uncertainty and a second loss function for improving prediction accuracy of the neural network model. Depending on embodiments, the first loss function may include a negative log-likelihood (NLL) based loss function. The second loss function may include a mean squared error (MSE) based loss function. Negative Log-Likelihood (NLL)-based loss function may cause gradient contraction or gradient collision problems in certain parts when the neural network model calculates the prediction result for the input value. Therefore, the loss function in the present disclosure may include a second loss function in addition to the first loss function. Hereinafter, information on the loss function according to the present disclosure will be described with reference to FIG. 2 .

The storage unit 120 may include a memory and/or a permanent storage medium. Memory is a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg SD or XD memory, etc.), RAM (Random Access Memory, RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory, ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk, optical disk At least one type of storage medium may be included.

In the present disclosure, the storage unit 120 may store the learning data set acquired through the communication unit 130 as a database. For example, the storage unit 120 may store information related to a plurality of drugs and a plurality of target substances in a database.

The communication unit 130 may include one or more modules enabling communication between the computing device 100 and a communication system or between the computing device 100 and a network (not shown). The communication unit 130 may include at least one of a mobile communication module, a wired Internet module, and a wireless Internet module.

Hereinafter, a method of learning a neural network model performed by the computing device 100 will be described.

2 is a flowchart illustrating an example of a method for a computing device to learn a neural network model, according to some embodiments of the present disclosure. 3 is a diagram for explaining an example of tilt collision according to some embodiments of the present disclosure.

Referring to FIG. 2 , the communication unit 130 of the computing device 100 may acquire a learning data set (S110). For example, the communication unit 130 may obtain a learning data set related to a plurality of drugs and a plurality of target substances. However, it is not limited thereto.

The processor 110 of the computing device 100 may train a neural network model that outputs a prediction result for an input value and estimates the uncertainty of the prediction based on the training data set (S120). According to embodiments, the neural network model may be a model based on an Evidential Regression Network (ENet). The prediction result may be, for example, a result of predicting affinity between a drug and a target substance. Uncertainty may be a value indicating how accurate or inaccurate the prediction result is.

According to some embodiments of the present disclosure, a loss function for training a neural network model may include a first loss function for quantifying prediction results and prediction uncertainty and a second loss function for improving prediction accuracy of the neural network model. there is. Depending on the embodiment, the first loss function may include a Negative Log-Likelihood (NLL) based loss function.

According to some embodiments of the present disclosure, the second loss function may include a mean squared error (MSE) based loss function.

및 신경망 모델을 통해 출력된 출력값 세트

가 고려될 수 있다. 여기서,

일 수 있다.

은

에서 역전파 되는 기울기 벡터일 수 있다. 여기서,

일 수 있다. 이하에서는 표기를 간결하기 위해

는 생략한다. 기울기 크기는

로 나타낼 수 있다. 한편, 모델의 예측값(예측 결과)의 기울기 크기가 신경망 모델의 예측 정확도에 관계없이 중요하지 않음을 확인할 필요가 있을 수 있다. 본 개시에 따른 입력값에 대한 예측 결과는 신경망 모델에 의해 생성된 한계 분포의 예측 평균(predictive mean)에 의해 결정될 수 있다. 따라서, 신경망 모델의 예측 평균이

로부터 획득되기 때문에, 신경망 모델의 예측 결과는

로 학습될 수 있다.

은

와 같은 도함수를 통해 참조될 수 있으며, 하기와 같이 재작성될 수 있다. Specifically, the processor 110 may train the neural network model so that the first loss function calculates a prediction result, but when only the first loss function is used, the prediction accuracy may be somewhat insufficient. Here, the prediction accuracy may be calculated through a method such as Mean Squared Error (MSE). In this case, one cause of poor prediction accuracy may be that the first loss function increases prediction uncertainty instead of correcting the prediction value. To explain this in more detail, the gradient vector and magnitude will be described. First, the loss function of the neural network model

and a set of outputs outputted through the neural network model.

may be considered. here,

can be

silver

may be a gradient vector that is back-propagated in here,

can be In order to simplify the notation below,

is omitted. the gradient magnitude is

can be expressed as On the other hand, it may be necessary to confirm that the magnitude of the gradient of the predicted value (prediction result) of the model is not important regardless of the prediction accuracy of the neural network model. A prediction result for an input value according to the present disclosure may be determined by a predictive mean of a marginal distribution generated by a neural network model. Therefore, the prediction average of the neural network model is

Since it is obtained from, the prediction result of the neural network model is

can be learned with

silver

It can be referenced through a derivative such as , and can be rewritten as follows.

일 수 있다. 다음으로, 예측 정확도가 저조한 한가지 원인을 설명하기 위해서

은 예측 결과와 실제 값의 차이를 나타내는

에 관계없이 급격히 떨어지거나 또는 0이 될 수 있다는 것을 증명할 필요가 있다. 구체적으로, 신경망 모델의 인식적 불확실성

이 무한히 높아 데이터의 희소성을 의미한다고 가정할 수 있다. 그리고, 가정된 무한히 큰 인식적 불확실성을 직접적으로 고려하는 대신 무한히 작은 의사 관찰

을 고려할 수 있다. 여기서,

는

을 통해 결정되었을 수 있다. 이 경우, 신경망 모델은

를 통해 예측 결과

를 훈련하는 것이 중지될 수 있다. 환언하자면, 정규 역 감마 분포에 있어서 의사 관찰이 0으로 수렴하는 경우(

), 제 1 손실함수의 기울기 크기(

)는

가 0이 되기 때문에, 거의 0이 될 수 있다. 이를 수학식으로 표현하면 하기와 같을 수 있다.here,

can be Next, in order to explain one cause of poor prediction accuracy,

represents the difference between the predicted result and the actual value

It is necessary to prove that it can drop rapidly or become zero regardless of Specifically, the perceptual uncertainty of neural network models

can be assumed to be infinitely high, indicating the sparsity of the data. and, instead of directly considering the assumed infinitely large epistemic uncertainty, infinitely small pseudo-observations

can be considered. here,

Is

may have been determined through In this case, the neural network model is

Predicted results via

training can be stopped. In other words, if the pseudo-observation converges to 0 for the inverse normal gamma distribution (

), the gradient of the first loss function (

)Is

becomes zero, so it can be almost zero. If this is expressed as a mathematical expression, it may be as follows.

를 예측하도록 학습시킬 수 없을 수 있다. 따라서, 본 개시에서의 신경망 모델을 학습시키기 위한 손실함수는 제 1 손실함수 뿐만 아니라, 신경망 모델의 예측 정확도를 향상시키기 위한 제 2 손실함수를 포함할 수 있다.As described above, the first loss function is a prediction result of the neural network model when the estimated perceptual uncertainty is high.

may not be trained to predict . Accordingly, the loss function for training the neural network model in the present disclosure may include not only the first loss function, but also a second loss function for improving prediction accuracy of the neural network model.

가 의사 관찰

와 독립적이므로, 신경망 모델이 예측 결과

를 산출하도록 지속적으로 훈련시킬 수 있다. 이에 따라, 제 2 손실함수는 높은 인식적 불확실성 조건에서 모델의 예측 결과를 향상시킬 수 있다.Specifically, the second loss function based on the mean square error is

Doctor Observation

Since it is independent of , the neural network model predicts the result

can be continuously trained to produce Accordingly, the second loss function may improve the prediction result of the model under a high perceptual uncertainty condition.

및 제 1 손실함수와 제 2 손실함수 간의 기울기 충돌(gradient conflict)를 완화시키기 위해 도출된 립시츠 연속　함수(Lipschitz-continuous function) 기반의 제 2 서브 손실함수를 포함할 수 있다.Meanwhile, according to some embodiments of the present disclosure, the second loss function is a first sub-loss function based on mean square error (MSE)

and a second sub-loss function based on a Lipschitz-continuous function derived to mitigate a gradient conflict between the first loss function and the second loss function.

로 나타낼 수 있다. 제 1 손실함수의 불확실성 기울기는

로 나타낼 수 있다. 이 경우, 제 1 손실함수의 총 기울기는

으로 나타낼 수 있다. 여기서,

일 수 있다. 이에 따라, 제 1 손실함수의 예측 결과 기울기는 제 1 서브 손실함수의 예측 결과 기울기와 충돌하지 않을 수 있다. 기울기의 충돌은 제 1 손실함수의 예측 결과 기울기 벡터 및 제 1 서브 손실함수의 예측 결과 기울기 벡터가 서로 다른 방향으로 향하는 것을 의미할 수 있다.Specifically, the prediction result gradient of the first loss function is

can be expressed as The uncertainty slope of the first loss function is

can be expressed as In this case, the total gradient of the first loss function is

can be expressed as here,

can be Accordingly, the predicted gradient of the first loss function may not collide with the predicted gradient of the first sub-loss function. The gradient collision may mean that the predicted gradient vector of the first loss function and the predicted gradient vector of the first sub-loss function are directed in different directions.

를 나타낼 수 있다. 제 2 선(320)은 제 1 손실함수의 불확실성 기울기

를 나타낼 수 있다. 제 3 선(330)은 제 1 서브 손실함수의 예측 결과 기울기

를 나타낼 수 있다. 제 1 선(310) 내지 제 3 선(330)을 참조하면, 제 1 선(310) 내지 제 3 선(330)은 기울기 벡터가 서로 같은 방향으로 향할 수 있다. 제 1 선(310) 내지 제 3 선(330)의 기울기 벡터가 서로 같은 방향으로 향한다는 것은, 제 1 선(310) 내지 제 3 선(330)의 기울기가 충돌이 발생하지 않았음을 나타낼 수 있다. 한편, 도 3의 (b)는 기울기가 적어도 일부 충돌하는 일례를 나타내기 위한 도면일 수 있다. 여기서, 제 1 선(311)은 제 1 손실함수의 예측 결과 기울기

를 나타낼 수 있다. 제 2 선(321)은 제 1 손실함수의 불확실성 기울기

를 나타낼 수 있다. 제 3 선(331)은 제 1 서브 손실함수의 예측 결과 기울기

를 나타낼 수 있다. 제 1 선(311) 및 제 3 선(331)을 참조하면, 제 1 선(311) 및 제 3 선(331)은 기울기 벡터가 서로 같은 방향으로 향하므로, 제 1 선(311) 및 제 3 선(331)의 기울기가 충돌이 발생하지 않았음을 나타낼 수 있다. 반면, 제 2 선(321)은 제 1 선(311) 및 제 3 선(331)과는 다른 방향으로 기울기 벡터가 향할 수 있다. 따라서, 제 2 선(331)은 제 1 선(311) 및 제 3 선(331)과 기울기의 적어도 일부가 충돌한 것을 나타낼 수 있다. 도 3의 (c)는 불확실성 추정에 따른 기울기가 충돌하는 일례를 나타내기 위한 도면일 수 있다. 여기서, 제 1 선(312)은 제 1 손실함수의 예측 결과 기울기

를 나타낼 수 있다. 제 2 선(322)은 제 1 손실함수의 불확실성 기울기

를 나타낼 수 있다. 제 3 선(332)은 제 1 서브 손실함수의 예측 결과 기울기

를 나타낼 수 있다. 제 1 선(312) 및 제 3 선(332)을 참조하면, 제 1 선(312) 및 제 3 선(332)은 기울기 벡터가 서로 같은 방향으로 향하므로, 제 1 선(312) 및 제 3 선(332)의 기울기가 충돌이 발생하지 않았음을 나타낼 수 있다. 반면, 제 2 선(322)은 제 1 선(312) 및 제 3 선(332)과는 다른 방향으로 기울기 벡터가 향할 수 있다. 따라서, 제 2 선(332)은 제 1 선(312) 및 제 3 선(332)과 기울기의 적어도 일부가 충돌한 것을 나타낼 수 있다.Reference may be made to FIG. 3 to describe an example of the gradient collision. (a) of FIG. 3 may be a diagram for illustrating an example in which the inclination does not collide. Here, the first line 310 is the predicted slope of the first loss function

can represent The second line 320 is the uncertainty slope of the first loss function.

can represent The third line 330 is the predicted slope of the first sub-loss function.

can represent Referring to the first line 310 to the third line 330 , the gradient vectors of the first line 310 to the third line 330 may be directed in the same direction. If the gradient vectors of the first line 310 to the third line 330 are directed in the same direction, the gradients of the first line 310 to the third line 330 may indicate that no collision has occurred. there is. Meanwhile, (b) of FIG. 3 may be a diagram illustrating an example in which slopes collide at least partially. Here, the first line 311 is the predicted slope of the first loss function

can represent The second line 321 is the uncertainty slope of the first loss function

can represent The third line 331 is the predicted slope of the first sub-loss function.

can represent Referring to the first line 311 and the third line 331, since the gradient vectors of the first line 311 and the third line 331 are directed in the same direction, the first line 311 and the third line 311 The slope of line 331 may indicate that no collision occurred. On the other hand, the gradient vector of the second line 321 may be directed in a direction different from that of the first line 311 and the third line 331 . Accordingly, the second line 331 may indicate that at least a part of the slope collides with the first line 311 and the third line 331 . FIG. 3(c) may be a diagram illustrating an example in which slopes according to uncertainty estimation collide. Here, the first line 312 is the predicted slope of the first loss function

can represent The second line 322 is the uncertainty slope of the first loss function.

can represent The third line 332 is the predicted slope of the first sub-loss function.

can represent Referring to the first line 312 and the third line 332, the gradient vectors of the first line 312 and the third line 332 are directed in the same direction. The slope of line 332 may indicate that no collision occurred. On the other hand, the gradient vector of the second line 322 may be directed in a direction different from that of the first line 312 and the third line 332 . Accordingly, the second line 332 may indicate that at least a part of the slope collides with the first line 312 and the third line 332 .

는 제 1 서브 손실함수의 예측 결과 기울기

와 충돌하지 않는 것을 확인할 수 있다. 이를 통해, 기울기가 충돌하지 않는 지점 및 이유 등을 추정해 볼 수 있다. 구체적으로, 제 1 서브 손실함수의 예측 결과 기울기

의 크기 및 제 1 손실함수의 예측 결과 기울기

의 크기가 0이 아닌 경우, 제 1 서브 손실함수의 예측 결과 기울기 및 제 1 손실함수의 예측 결과 기울기 간의 코사인 유사도

은 1일 수 있다. 따라서, 제 1 서브 손실함수의 예측 결과 기울기

는 하기의 수학식 6과 같이 제 1 손실함수의 예측 결과 기울기

와 비례 관계에 있을 수 있다.Referring to (a) to (c) of FIG. 3, the predicted slope of the first loss function

is the predicted result gradient of the first sub-loss function

You can check that it doesn't collide with . Through this, it is possible to estimate the point where the gradient does not collide and why. Specifically, the prediction result gradient of the first sub-loss function

The magnitude of and the slope of the predicted result of the first loss function

If the magnitude of is not 0, the cosine similarity between the predicted slope of the first sub-loss function and the predicted slope of the first loss function

may be 1. Therefore, the prediction result gradient of the first sub-loss function

Is the prediction result slope of the first loss function as shown in Equation 6 below

may be in a proportional relationship with

및 제 1 손실함수의 예측 결과 기울기

를 이용하여 파라미터를 업데이트 하는 경우, 제 1 서브 손실함수의 기울기 및 제 1 손실함수의 기울기는 충돌이 발생하지 않을 수 있다. 따라서, 제 1 손실함수의 총 기울기 및 제 1 서브 손실함수의 기울기가 같은 방향이 아닌 경우, 제 1 손실함수의 불확실성 기울기는 제 1 서브 손실함수의 기울기와 같은 방향이 아닐 수 있다. 이를 수식으로 표현하면,

일 수 있다. 그러므로, 상술한 바는 제 1 서브 손실함수 및 제 1 손실함수 간의 기울기 충돌의 원인은 제 1 손실함수의 불확실성 기울기일 수 있다는 것을 나타낼 수 있다. 따라서, 본 개시에서의 제 2 손실함수는 제 1 손실함수와 제 2 손실함수 간의 기울기 충돌을 완화시키기 위해 도출된 립시츠 연속　함수 기반의 제 2 서브 손실함수를 포함할 수 있다. 그리고, 컴퓨팅 장치(100)의 프로세서(110)는 제 1 서브 손실함수의 연산값과 예측의 불확실성과 관련된 신경망 모델의 분산 파라미터(variance parameter)에 기반한 임계값을 비교한 결과에 따라, 제 1 서브 손실함수 또는 제 2 서브 손실함수를 선택적으로 사용할 수 있다. 이하, 도 4 및 도 5를 통해 본 개시에 따른 프로세서(110)가 제 1 서브 손실함수 또는 제 2 서브 손실함수를 선택적으로 사용하는 방법에 대해 설명한다.In other words, the neural network model predicts the gradient of the first sub-loss function

and the predicted result gradient of the first loss function.

When the parameter is updated using , the gradient of the first sub-loss function and the gradient of the first loss function may not collide. Therefore, when the total slope of the first loss function and the slope of the first sub-loss function are not in the same direction, the uncertainty slope of the first loss function may not be in the same direction as the slope of the first sub-loss function. Expressing this as a formula,

can be Therefore, the above may indicate that the cause of the gradient collision between the first sub-loss function and the first loss function may be the uncertainty gradient of the first loss function. Accordingly, the second loss function in the present disclosure may include a second sub-loss function based on a Lipshitz continuous function derived to mitigate a gradient collision between the first loss function and the second loss function. And, the processor 110 of the computing device 100 compares the calculated value of the first sub loss function with a threshold value based on a variance parameter of the neural network model related to the uncertainty of the prediction, and the first sub loss function. A loss function or a second sub-loss function may be selectively used. Hereinafter, a method of selectively using the first sub loss function or the second sub loss function by the processor 110 according to the present disclosure will be described with reference to FIGS. 4 and 5 .

4 is a flowchart illustrating an example of a method of selectively using a first sub loss function or a second sub loss function as a second loss function by a computing device according to some embodiments of the present disclosure. 5 is a diagram for explaining an example of a parameter according to some embodiments of the present disclosure.

일 수 있으며,

으로 계산될 수 있다. 그리고, 제 1 서브 손실함수의 연산값은 제 1 손실함수에 의한 의사 관찰의 증가와 관련이 있을 수 있다. 일례로, 하기의 수학식 7과 같이, 신경망 모델의 제 1 서브 손실함수의 연산값

이 특정 임계값

및

보다 큰 경우,

및

에 대한 제 1 손실함수의 미분 부호는 양수일 수 있다.Prior to the description of FIG. 4, a threshold value based on a calculation value of the first sub-loss function and a variance parameter of a neural network model related to prediction uncertainty will be described. Here, the calculated value of the first sub-loss function is

can be,

can be calculated as Also, the calculated value of the first sub loss function may be related to an increase in pseudo observation by the first loss function. For example, as shown in Equation 7 below, the calculated value of the first sub-loss function of the neural network model

this particular threshold

and

If greater than

and

The differential sign of the first loss function for may be a positive number.

일 수 있다.

일 수 있다.

는 디감마 함수(digamma function)일 수 있다.here,

can be

can be

may be a digamma function.

및

모두 제 1 서브 손실함수의 연산값

에 대응되는 값을 가지기 때문에,

로부터

를 결정하고,

로부터

를 결정하여,

를 결정할 수 있다. 이는, 예측값과 실제값의 차이가 특정 임계값

또는

보다 크면 제 1 손실함수의 손실이 예측의 불확실성을 증가시키도록 신경망 모델이 훈련될 수 있다는 것을 나타낼 수 있다. 따라서, 본 개시에서의 제 2 손실함수는 제 1 손실함수와 제 2 손실함수 간의 기울기 충돌을 완화시키기 위해 립시츠 연속　함수 기반의 제 2 서브 손실함수를 포함할 수 있다.

and

All calculated values of the first sub-loss function

Since it has a value corresponding to

from

to decide,

from

By determining

can decide This means that the difference between the predicted value and the actual value is a certain threshold value.

or

If it is greater than , it may indicate that the neural network model can be trained such that the loss of the first loss function increases the uncertainty of the prediction. Accordingly, the second loss function in the present disclosure may include a second sub-loss function based on a Lipshitz continuous function in order to mitigate a gradient collision between the first loss function and the second loss function.

Referring to FIG. 4 , the processor 110 of the computing device 100 may compare an operation value of the first sub-loss function with a threshold value based on a variance parameter of a neural network model related to prediction uncertainty (S210). Here, the threshold value based on the variance parameter of the neural network model related to prediction uncertainty may be an average value of minimum variance parameter values.

The processor 110 converts the first sub-loss function as the second loss function of the neural network model when the calculated value of the first sub-loss function is less than the threshold value based on the variance parameter of the neural network model related to the uncertainty of prediction (S220, Yes). It can be used for learning (S230). Here, the use of the first sub-loss function for learning may be understood to mean that the output value of the first sub-loss function is determined as the output value of the second loss function when the value is less than the threshold value.

When the calculated value of the first sub-loss function is greater than or equal to the threshold value based on the variance parameter of the neural network model related to prediction uncertainty (S220, No), the processor 110 converts the second sub-loss function as the second loss function to the neural network model. It can be used for learning (S240). Here, the use of the first sub-loss function for learning may be understood as determining that the output value of the second sub-loss function is determined as the output value of the second sub-loss function when the value is equal to or greater than the threshold value. Steps S210 to S240 may be expressed as the following equation.

는 제 2 서브 손실함수를 포함하는 제 2 손실함수일 수 있다.

는 예측의 불확실성과 관련된 신경망 모델의 분산 파라미터에 기반한 임계값을 나타내며,

일 수 있다. 다시 말해, 임계값은 예측의 불확실성과 관련된 분산 파라미터

에 관한 임계값

과 예측의 불확실성과 관련된 분산 파라미터

에 관한 임계값

중 최소값의 평균값일 수 있다.

가 임의의 데이터 세트 D에서 추출된 미니-배치라고 하면, 신경망 모델

는

를 입력받아

를 생성할 수 있다. 따라서, 각

및

에 대해 제 2 서브 손실함수는 상술한 [수학식 8]과 같이 정의될 수 있다.here,

may be a second loss function including a second sub-loss function.

represents the threshold based on the variance parameter of the neural network model related to the uncertainty of the prediction,

can be In other words, the critical value is the variance parameter related to the uncertainty of the prediction.

Threshold value for

and the variance parameter related to the uncertainty of the prediction

Threshold value for

It may be the average value of the minimum value among them.

Let be a mini-batch extracted from a random data set D, then the neural network model

Is

input

can create Therefore, each

and

For , the second sub-loss function may be defined as in [Equation 8] described above.

를 제한하여 기울기 크기를 조절할 수 있다. 여기서, 제 2 서브 손실함수가 분산 파라미터

에 대한 기울기를 전파하는 것은 아닐 수 있다.The second loss function including the second sub-loss function is when the pseudo observation is reduced by the first loss function,

You can control the size of the gradient by limiting . Here, the second sub-loss function is the variance parameter

It may not propagate the gradient for .

(211),

(212) 및

(213)는 제 1 손실함수(210)와 관련되는 파라미터일 수 있다. 그리고, 바이어스(bias) 파라미터인

(221)는 제 2 손실함수(220)와 관련되는 파라미터일 수 있다. 다만, 이에 한정되는 것은 아니다.For example, referring to FIG. 5, the variance parameter

(211),

(212) and

(213) may be a parameter related to the first loss function (210). And, the bias parameter

(221) may be a parameter related to the second loss function (220). However, it is not limited thereto.

Meanwhile, according to some embodiments of the present disclosure, a loss function for training a neural network model may further include a third loss function for regulating the first loss function by increasing prediction uncertainty. In this case, the loss function may be determined by the following equation.

은 제 2 손실함수이고,

은 제 1 손실함수이며,

은 제 3 손실함수일 수 있다. 그리고,

은 계수일 수 있다. 제 3 손실함수는 제 1 손실함수 및 제 2 손실함수와 같이 예측 결과를 산출하기 위해 사용될 수도 있다. 다만, 제 3 손실함수는 예측의 불확실성을 증가시킴으로써 제 1 손실함수를 조절하는 것이 더욱 주된 목적인 함수일 수 있다.here,

is the second loss function,

is the first loss function,

may be a third loss function. and,

may be a coefficient. The third loss function may be used to calculate the prediction result like the first loss function and the second loss function. However, the third loss function may be a function whose main purpose is to adjust the first loss function by increasing the uncertainty of prediction.

According to the configuration described above, the loss function for learning the neural network model is a first loss function for quantifying the prediction result and uncertainty of prediction, a second loss function for improving the prediction accuracy of the neural network model, and increasing the uncertainty of prediction. A third loss function adjusting the first loss function may be included. Accordingly, the neural network model of the present disclosure can solve the gradient collision problem that can occur in a conventional Evidential Regression Network (ENet) and improve the accuracy of prediction for an input value.

Hereinafter, results of experiments performed using the neural network model according to the present disclosure will be described.

6 is a diagram for explaining the results of an experiment performed using a neural network model according to some embodiments of the present disclosure.

6(a) may be a result of an experiment performed using the first data set (e.g., the Davis data set, which is one of well-known data sets in literature related to drug-target affinity prediction). In (a) of FIG. 6, the x-axis may indicate the number of iterations, and the y-axis may indicate the average cosine similarity of the slope. The first line 410 in (a) of FIG. 6 may be a line representing the cosine similarity between the slopes of the first loss function and the second loss function when learning is performed using an example of the neural network model according to the present disclosure. . The second loss function may be a loss function including the first sub loss function and the second sub loss function. Depending on the embodiment, the first line 410 may be a line representing the cosine similarity between the slopes of the first loss function and the second loss function using the neural network model according to the present disclosure further including a third loss function. The second line 420 in (a) of FIG. 6 may be a line representing the cosine similarity between the slopes of the first loss function and the second loss function in performing learning using another example of the neural network model according to the present disclosure. there is. The second loss function may be a loss function that does not include the second sub loss function and includes only the first sub loss function. The third line 430 in (a) of FIG. 6 shows the first loss function and the second loss function in performing learning using a small neural network model partially reduced in scale of the neural network model according to the present disclosure. It may be a line representing the cosine similarity between the slopes of . Referring to the first line 410 to the third line 430 , the first line 410 may have higher cosine similarity than the second line 420 and the third line 430 . As described above, this may indicate that the gradient vectors of the first loss function and the second loss function are directed in the same direction. In other words, it may indicate that the gradient collision between the first loss function and the second loss function is avoided. That is, the experimental results shown in (a) of FIG. 6 show that when the second sub-loss function according to an embodiment of the present disclosure is used, better performance can be secured compared to the case in which the second sub-loss function is not used. can be checked through

Meanwhile, (b) of FIG. 6 may be a result of an experiment performed using a second data set (e.g. Kiba data set, which is one of well-known data sets in literature related to drug-target affinity prediction). In (b) of FIG. 6, the x-axis may indicate the number of iterations, and the y-axis may indicate the average cosine similarity of the slope. The first line 411 in (b) of FIG. 6 may be a line representing the cosine similarity between the slopes of the first loss function and the second loss function when learning is performed using an example of the neural network model according to the present disclosure. . The second loss function may be a loss function including the first sub loss function and the second sub loss function. Depending on the embodiment, the first line 411 may be a line representing the cosine similarity between the slopes of the first loss function and the second loss function using the neural network model according to the present disclosure further including a third loss function. The second line 421 in (b) of FIG. 6 may be a line representing the cosine similarity between the slopes of the first loss function and the second loss function in performing learning using another example of the neural network model according to the present disclosure. there is. The second loss function may be a loss function that does not include the second sub loss function and includes only the first sub loss function. The third line 4301 in (b) of FIG. 6 shows the values of the first loss function and the second loss function in performing learning using a small neural network model obtained by partially reducing the scale of the neural network model according to the present disclosure. Referring to the first line 411 to the third line 431, the first line 411 is the cosine of the second line 421 and the third line 431. Here, the meaning of high cosine similarity can be understood as a small collision between the first loss function and the second loss function. The value indicated by the y-axis of may have a proportional relationship with the performance of the model, that is, in the case of using the second sub-loss function according to an embodiment of the present disclosure, as in (b) of FIG. It can be confirmed through the experimental results shown in FIG. 6(b) that better performance can be ensured compared to the case in which the sub-loss function is not used.

7 depicts a general schematic diagram of an example computing environment in which embodiments of the present disclosure may be implemented.

Although the present disclosure has generally been described above in terms of computer-executable instructions that can be executed on one or more computers, those skilled in the art will appreciate that the present disclosure may be implemented in combination with other program modules and/or as a combination of hardware and software. will know

Generally, modules herein include routines, procedures, programs, components, data structures, etc. that perform particular tasks or implement particular abstract data types. It will also be appreciated by those skilled in the art that the methods of the present disclosure may be used in single-processor or multiprocessor computer systems, minicomputers, mainframe computers, as well as personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, and the like (each of which is It will be appreciated that other computer system configurations may be implemented, including those that may be operative in connection with one or more associated devices.

The described embodiments of the present disclosure may also be practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

A computer typically includes a variety of computer readable media. Media accessible by a computer includes volatile and nonvolatile media, transitory and non-transitory media, removable and non-removable media. By way of example, and not limitation, computer readable media may include computer readable storage media and computer readable transmission media.

Computer readable storage media are volatile and nonvolatile media, transitory and non-transitory, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. includes media Computer readable storage media may include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital video disk (DVD) or other optical disk storage device, magnetic cassette, magnetic tape, magnetic disk storage device or other magnetic storage device. device, or any other medium that can be accessed by a computer and used to store desired information.

A computer readable transmission medium typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism. Including all information delivery media. The term modulated data signal means a signal that has one or more of its characteristics set or changed so as to encode information within the signal. By way of example, and not limitation, computer readable transmission media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. Combinations of any of the above are also intended to be included within the scope of computer readable transmission media.

An exemplary environment 1100 implementing various aspects of the present disclosure is shown including a computer 1102, which includes a processing unit 1104, a system memory 1106, and a system bus 1108. do. System bus 1108 couples system components, including but not limited to system memory 1106 , to processing unit 1104 . Processing unit 1104 may be any of a variety of commercially available processors. Dual processor and other multiprocessor architectures may also be used as the processing unit 1104.

System bus 1108 may be any of several types of bus structures that may additionally be interconnected to a memory bus, a peripheral bus, and a local bus using any of a variety of commercial bus architectures. System memory 1106 includes read only memory (ROM) 1110 and random access memory (RAM) 1112 . A basic input/output system (BIOS) is stored in non-volatile memory 1110, such as ROM, EPROM, or EEPROM, and is a basic set of information that helps transfer information between components within computer 1102, such as during startup. contains routines. RAM 1112 may also include high-speed RAM, such as static RAM, for caching data.

The computer 1102 may also include an internal hard disk drive (HDD) 1114 (eg, EIDE, SATA) - the internal hard disk drive 1114 may also be configured for external use within a suitable chassis (not shown). Yes—a magnetic floppy disk drive (FDD) 1116 (e.g., for reading from or writing to removable diskette 1118), and an optical disk drive 1120 (e.g., CD-ROM) for reading disc 1122 or reading from or writing to other high capacity optical media such as DVDs). The hard disk drive 1114, magnetic disk drive 1116, and optical disk drive 1120 are connected to the system bus 1108 by a hard disk drive interface 1124, magnetic disk drive interface 1126, and optical drive interface 1128, respectively. ) can be connected to The interface 1124 for external drive implementation includes, for example, at least one or both of Universal Serial Bus (USB) and IEEE 1394 interface technologies.

These drives and their associated computer readable media provide non-volatile storage of data, data structures, computer executable instructions, and the like. In the case of computer 1102, drives and media correspond to storing any data in a suitable digital format. Although the description of computer-readable storage media above refers to HDDs, removable magnetic disks, and removable optical media such as CDs or DVDs, those skilled in the art can use zip drives, magnetic cassettes, flash memory cards, cartridges, It will be appreciated that other types of storage media readable by the computer may also be used in the exemplary operating environment and any such media may include computer executable instructions for performing the methods of the present disclosure. .

A number of program modules may be stored on the drive and RAM 1112, including an operating system 1130, one or more application programs 1132, other program modules 1134, and program data 1136. All or portions of the operating system, applications, modules and/or data may also be cached in RAM 1112. It will be appreciated that the present disclosure may be implemented in a variety of commercially available operating systems or combinations of operating systems.

A user may enter commands and information into the computer 1102 through one or more wired/wireless input devices, such as a keyboard 1138 and a pointing device such as a mouse 1140. Other input devices (not shown) may include a microphone, IR remote control, joystick, game pad, stylus pen, touch screen, and the like. Although these and other input devices are often connected to the processing unit 1104 through an input device interface 1142 that is connected to the system bus 1108, a parallel port, IEEE 1394 serial port, game port, USB port, IR interface, may be connected by other interfaces such as the like.

A monitor 1144 or other type of display device is also connected to the system bus 1108 through an interface such as a video adapter 1146. In addition to the monitor 1144, computers typically include other peripheral output devices (not shown) such as speakers, printers, and the like.

Computer 1102 may operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s) 1148 via wired and/or wireless communications. Remote computer(s) 1148 may be a workstation, server computer, router, personal computer, handheld computer, microprocessor-based entertainment device, peer device, or other common network node, and may generally refer to computer 1102. Although many or all of the described components are included, for brevity, only memory storage device 1150 is shown. The logical connections shown include wired/wireless connections to a local area network (LAN) 1152 and/or a larger network, such as a wide area network (WAN) 1154 . Such LAN and WAN networking environments are common in offices and corporations and facilitate enterprise-wide computer networks, such as intranets, all of which can be connected to worldwide computer networks, such as the Internet.

When used in a LAN networking environment, computer 1102 connects to local network 1152 through wired and/or wireless communication network interfaces or adapters 1156. Adapter 1156 may facilitate wired or wireless communications to LAN 1152, which also includes a wireless access point installed therein to communicate with wireless adapter 1156. When used in a WAN networking environment, computer 1102 may include a modem 1158, be connected to a communications server on WAN 1154, or other device that establishes communications over WAN 1154, such as over the Internet. have the means A modem 1158, which may be internal or external and a wired or wireless device, is connected to the system bus 1108 through a serial port interface 1142. In a networked environment, program modules described for computer 1102, or portions thereof, may be stored on remote memory/storage device 1150. It will be appreciated that the network connections shown are exemplary and other means of establishing a communication link between computers may be used.

Computer 1102 is any wireless device or entity that is deployed and operating in wireless communication, eg, printers, scanners, desktop and/or portable computers, portable data assistants (PDAs), communication satellites, wireless detectable tags associated with It operates to communicate with arbitrary equipment or places and telephones. This includes at least Wi-Fi and Bluetooth wireless technologies. Thus, the communication may be a predefined structure as in conventional networks or simply an ad hoc communication between at least two devices.

Wi-Fi (Wireless Fidelity) makes it possible to connect to the Internet without wires. Wi-Fi is a wireless technology, such as a cell phone, that allows such devices, eg, computers, to transmit and receive data both indoors and outdoors, i.e. anywhere within coverage of a base station. Wi-Fi networks use a radio technology called IEEE 802.11 (a,b,g, etc.) to provide secure, reliable, and high-speed wireless connections. Wi-Fi can be used to connect computers to each other, to the Internet, and to wired networks (using IEEE 802.3 or Ethernet). Wi-Fi networks can operate in the unlicensed 2.4 and 5 GHz radio bands, for example, at 11 Mbps (802.11a) or 54 Mbps (802.11b) data rates, or on products that include both bands (dual band). there is.

Those skilled in the art will understand that the various illustrative logical blocks, modules, processors, means, circuits, and algorithm steps described in connection with the embodiments disclosed herein are electronic hardware, (for convenience) , may be implemented by various forms of program or design code (referred to herein as “software”) or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and the design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

Various embodiments presented herein may be implemented as a method, apparatus, or article of manufacture using standard programming and/or engineering techniques. The term "article of manufacture" includes a computer program or media accessible from any computer-readable device. For example, computer-readable storage media include magnetic storage devices (eg, hard disks, floppy disks, magnetic strips, etc.), optical disks (eg, CDs, DVDs, etc.), smart cards, and flash memory devices (eg, EEPROM, cards, sticks, key drives, etc.), but are not limited thereto. The term “machine-readable medium” includes, but is not limited to, wireless channels and various other media that can store, hold, and/or convey instruction(s) and/or data.

The description of the presented embodiments is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art of this disclosure, and the general principles defined herein may be applied to other embodiments without departing from the scope of this disclosure. Thus, the present disclosure is not to be limited to the embodiments presented herein, but is to be interpreted in the widest scope consistent with the principles and novel features presented herein.

Claims (12)

A learning method of a multi-task model performed by a computing device including at least one processor, comprising:
obtaining a training data set; and
training a neural network model that outputs prediction results for input values and estimates uncertainty of prediction based on the learning data set;
including,
The loss function for learning the neural network model is,
A first loss function for quantifying the prediction result and uncertainty of the prediction and a second loss function for improving the prediction accuracy of the neural network model,
method.
According to claim 1,
The neural network model,
A model based on the Evidential Regression Network (ENet),
method.
According to claim 1,
The first loss function,
Including a loss function based on the negative log-likelihood (NLL),
method.
According to claim 1,
The second loss function,
Including a loss function based on the mean squared error (MSE),
method.
According to claim 1,
The second loss function,
a first sub-loss function based on mean square error (MSE); and
a second sub-loss function based on a Lipschitz-continuous function derived to mitigate a gradient conflict between the first loss function and the second loss function;
including,
method.
According to claim 5,
The first sub loss function or the second sub loss function,
According to a result of comparing the calculated value of the first sub-loss function with a threshold value based on a variance parameter of the neural network model related to the uncertainty of the prediction, as the second loss function, selective for learning the neural network model used as,
method.
According to claim 6,
When the calculated value of the first sub-loss function is less than a threshold value based on the variance parameter of the neural network model related to the uncertainty of the prediction, the first sub-loss function as the second loss function is used for learning the neural network model ,
method.
According to claim 6,
When the calculated value of the first sub-loss function is greater than or equal to a threshold value based on the variance parameter of the neural network model related to the prediction uncertainty, the second sub-loss function as the second loss function is used for learning the neural network model. ,
method.
According to claim 6,
The threshold based on the variance parameter of the neural network model related to the uncertainty of the prediction,
The average value of the minimum values of the variance parameter,
method.
According to claim 1,
The loss function for learning the neural network model is,
a third loss function for regulating the first loss function by increasing the uncertainty of the prediction;
Including more,
method.
A computer program stored on a computer readable storage medium which, when executed on one or more processors, causes a method for training a multi-task model, the method comprising:
obtaining a training data set; and
training a neural network model that outputs prediction results for input values and estimates uncertainty of prediction based on the learning data set;
including,
The loss function for learning the neural network model is,
A first loss function for quantifying the prediction result and uncertainty of the prediction and a second loss function for improving the prediction accuracy of the neural network model,
A computer program stored on a computer readable storage medium.
A computing device for learning a multi-task model,
a communication unit acquiring a learning data set; and
a processor for training a neural network model that outputs a prediction result for an input value and estimates an uncertainty of prediction based on the training data set;
including,
The loss function for learning the neural network model is,
A first loss function for quantifying the prediction result and uncertainty of the prediction and a second loss function for improving the prediction accuracy of the neural network model,
computing device.

Images