KR20220010419A - Electronice device and learning method for low complexity artificial intelligentce model learning based on selecting the dynamic prediction confidence thresholed - Google Patents

Abstract

An aspect of the present invention relates to an electronic device which comprises: at least one processor; and at least one memory operatively linked to the at least one processor to contain at least one instruction which causes the at least one processor to perform operations, wherein the processor performs a learning operation of an artificial intelligence (AI) model which predicts a label for input data stored in the at least one memory. In addition, through the learning operation, the at least one processor performs a forward propagation operation by unit of a mini-batch for the input data, wherein the mini-batch has B sub input data included in the input data and the B is a natural number. Then, the at least one processor obtains a prediction confidence value for the label, performs a backward propagation operation only for the sub input data having a prediction confidence value no less than a prediction confidence threshold value among the sub input data, and obtains an approximate weighted value gradient against the weighted values of the AI model to update the weighted value based on the approximate weighted value gradient. Accordingly, calculation of a change in a weighted value can be omitted to reduce a calculating amount of an overall learning and save energy necessary for learning.

Description

ELECTRONICE DEVICE AND LEARNING METHOD FOR LOW COMPLEXITY ARTIFICIAL INTELLIGENTCE MODEL LEARNING BASED ON SELECTING THE DYNAMIC PREDICTION CONFIDENCE THRESHOLED

The present disclosure relates to an electronic device and method for low-complexity artificial intelligence model learning based on dynamic prediction reliability threshold selection.

DNN (deep neural network) is showing the best performance in various applications such as image recognition/classification and object detection based on many parameters and computational amount. A large amount of computation is required to learn many parameters of the DNN, and a long time of days/weeks is required to learn the DNN. In order to reduce the learning time and learning energy consumption, it is effective to reduce the amount of computation required for learning itself.

Since mini-batch gradient descent, which is commonly used for DNN training, inherently includes noise, it is possible to approximate the operation required for training without necessarily calculating it accurately. Conventional DNN learning requires a very large amount of computation, so there is a problem that it consumes a long learning time and a lot of learning energy. If an effective approximation can be applied to insignificant operations, the amount of computation required for overall learning can be reduced.

Republic of Korea Patent Publication 10-2019-0098107 Republic of Korea Patent Publication 10-2019-0118387

Various examples of the present disclosure utilize the characteristic that learning through the mini-batch gradient descent method allows noise, and within the allowable noise range set by the user, the error due to omission of the operation is small (ie, the prediction reliability is high) To provide an electronic device and method for learning a low-complexity artificial intelligence model based on dynamic prediction reliability threshold selection that can reduce the total amount of learning computation by omitting the weight change calculation for have.

Another object of the present invention is to provide an electronic device and method for learning a low-complexity artificial intelligence model based on dynamic prediction reliability threshold selection in which a user can adjust the degree of omission of weight change through dynamic prediction reliability threshold selection.

Technical problems to be achieved in various examples of the present disclosure are not limited to those mentioned above, and other technical problems not mentioned are to those of ordinary skill in the art from various examples of the present disclosure to be described below. can be considered by

In one aspect of the present disclosure, at least one processor (processor); and at least one memory operatively coupled to the at least one processor to store at least one instructions for causing the at least one processor to perform operations, the operations comprising: Performs a learning operation of an artificial intelligence model for predicting a label with respect to the input data stored in at least one memory, wherein the learning operation is performed in a mini-batch with respect to the input data - wherein the mini-batch has B sub-input data included in the input data, wherein B is a natural number; A forward propagation operation is performed in units to obtain a prediction reliability value for the label, and only backward propagation is performed for low-order input data in which the prediction reliability value is greater than or equal to the prediction reliability threshold value among the low-level input data. An electronic device is configured to obtain an approximate weight gradient with respect to the weight of the artificial intelligence model by performing an operation, and update the weight based on the approximate weight gradient.

In the learning operation, the back-propagation operation may be omitted for lower-level input data in which the prediction reliability value is less than the prediction reliability threshold value among the lower-level input data.

The learning operation includes an operation of setting a prediction reliability threshold for setting the prediction reliability threshold, and the operation of setting the prediction reliability threshold is: (a) an arbitrary prediction reliability threshold and RSE _skip - The RSE _skip is the inverse is a weight gradient error value according to omission of propagation operation; an operation of initializing; (b) reducing the random prediction reliability threshold by a specific variation value unit, and reducing the random prediction reliability threshold by the specific variation value unit The operation of calculating the RSE _skip each time, (c) the _{operation of repeatedly performing the operation until the RSE skip first} becomes equal to or exceeds a preset error boundary value for the first time, (d) the operation of the RSE _skip and setting, as the prediction reliability threshold value, the random prediction reliability threshold value when the threshold value is equal to or exceeds a preset error boundary value for the first time.

The RSE _skip may be defined as a product of a preset scaling factor, the number of lower input data having the prediction reliability value equal to or greater than the prediction reliability threshold value among the lower input data, and an average absolute gradient of the prediction reliability value.

The average absolute gradient may be defined based on a linear interpolation method for the entire confidence interval of the prediction reliability value.

In another aspect of the present disclosure, there is provided a learning method of an artificial intelligence model for predicting a label with respect to input data stored in at least one memory performed by an electronic device, wherein the input data is mini- a mini-batch, wherein the mini-batch has B sub-input data included in the input data, wherein B is a natural number; obtaining a prediction reliability value for the label by performing a forward propagation operation in units; obtaining an approximate weight gradient for the weight of the artificial intelligence model by performing a backward propagation operation only on lower input data in which the prediction reliability value is equal to or greater than a prediction reliability threshold value among the lower input data; and updating the weight based on the approximate weight gradient.

The back-propagation operation of the lower-order input data for which the prediction reliability value is less than the prediction reliability threshold value among the lower-order input data may be omitted.

The method further comprises the step of setting a prediction reliability threshold for setting the prediction reliability threshold, wherein the step of setting the prediction reliability threshold includes: (a) an arbitrary prediction reliability threshold and RSE _skip - The RSE _skip is the value of the backpropagation operation. Initializing -; weight gradient error value due to omission; (b) reducing the random prediction reliability threshold by a specific variation value unit, and calculating _{the RSE skip whenever the random prediction reliability threshold decreases by the specific variation value unit;} (c) _{repeatedly performing the operation (b) until the RSE skip first} becomes equal to or exceeds a preset error boundary value; and (d) setting the _{random prediction reliability threshold value when the RSE skip first} becomes equal to or first exceeds a preset error boundary value as the prediction reliability threshold value.

The RSE _skip may be defined as a product of a preset scaling factor, the number of lower input data having the prediction reliability value equal to or greater than the prediction reliability threshold value among the lower input data, and an average absolute gradient of the prediction reliability value.

The average absolute gradient may be defined based on a linear interpolation method for the entire confidence interval of the prediction reliability value.

The various examples of the present disclosure described above are only some of the preferred examples of the present disclosure, and various examples in which the technical features of various examples of the present disclosure are reflected are detailed descriptions to be detailed below by those of ordinary skill in the art can be derived and understood based on

According to various examples of the present disclosure, the following effects are obtained.

According to various examples of the present disclosure, by utilizing the characteristic that learning through the mini-batch gradient descent method allows noise, errors due to omission of calculations are small (that is, prediction reliability is high within the allowable noise range set by the user) ) by omitting the weight change calculation for the image, it is possible to reduce the total amount of learning computation and reduce the learning energy to minimize the impact on learning.

In addition, the user can control the degree of omission of the weight change amount by selecting the dynamic prediction reliability threshold.

Effects that can be obtained from various examples of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned are clearly derived to those of ordinary skill in the art based on the detailed description below. and can be understood

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings are provided to help understanding of various examples of the present disclosure, and various examples of the present disclosure are provided together with the detailed description. However, technical features of various examples of the present disclosure are not limited to specific drawings, and features disclosed in each drawing may be combined with each other to constitute a new embodiment. Reference numerals in each drawing refer to structural elements.
1 is a block diagram of an electronic device according to an example of the present disclosure.
2 is a block diagram of modules included in an electronic device according to an example of the present disclosure.
3 is a conceptual diagram for explaining operations performed by modules included in an electronic device according to an example of the present disclosure.
4 is a block diagram of a threshold selection module according to an example of the present disclosure.
5 is a conceptual diagram for explaining an operation performed by a threshold value selection module according to an example of the present disclosure.
6 is a flowchart of a learning method according to an example of the present disclosure.
7 is a flowchart of a method of setting a prediction reliability threshold included in a learning method according to an example of the present disclosure.
8 is a graph illustrating a relationship between a prediction reliability value and an absolute value of a weight gradient according to an example of the present disclosure.
9 is a graph illustrating a relationship between an average prediction reliability value and an epoch according to an example of the present disclosure.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, implementations according to the present invention will be described in detail with reference to the accompanying drawings. DETAILED DESCRIPTION The detailed description set forth below in conjunction with the appended drawings is intended to describe exemplary implementations of the present invention, and is not intended to represent the only implementation forms in which the present invention may be practiced. The following detailed description includes specific details in order to provide a thorough understanding of the present invention. However, it will be apparent to one of ordinary skill in the art that the present disclosure may be practiced without these specific details.

In some cases, well-known structures and devices may be omitted or shown in block diagram form focusing on core functions of each structure and device in order to avoid obscuring the concepts of the present disclosure. In addition, the same reference numerals are used to describe the same components throughout the present disclosure.

Since various examples according to the concept of the present invention can have various changes and can have various forms, various examples are illustrated in the drawings and described in detail in the present disclosure. However, this is not intended to limit the various examples according to the concept of the present invention to specific disclosed forms, and includes changes, equivalents, or substitutes included in the spirit and scope of the present invention.

Terms such as first or second may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one element from other elements, for example, without departing from the scope of rights according to the concept of the present invention, a first element may be named as a second element, Similarly, the second component may also be referred to as the first component.

When an element is referred to as being “connected” or “connected” to another element, it is understood that it may be directly connected or connected to the other element, but other elements may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle. Expressions describing the relationship between elements, for example, “between” and “between” or “directly adjacent to”, etc. should be interpreted similarly.

In various examples of the present disclosure, “/” and “,” should be construed as indicating “and/or”. For example, “A/B” may mean “A and/or B”. Furthermore, “A, B” may mean “A and/or B”. Furthermore, “A/B/C” may mean “at least one of A, B, and/or C”. Furthermore, “A, B, and C” may mean “at least one of A, B and/or C”.

In various examples of the present disclosure, “or” should be construed as indicating “and/or.” For example, “A or B” may include “only A”, “only B”, and/or “both A and B”. In other words, “or” should be construed as indicating “additionally or alternatively”.

In various examples of the present disclosure, terms such as "...unit", "...group", "module", etc. mean a unit that processes at least one function or operation, which is hardware or software or hardware and software It can be implemented as a combination of

The terminology used in the present disclosure is used only to describe various specific examples, and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present disclosure, terms such as "comprise" or "have" are intended to designate that the described feature, number, step, operation, component, part, or combination thereof exists, and includes one or more other features or numbers, It should be understood that the possibility of the presence or addition of steps, operations, components, parts or combinations thereof is not precluded in advance.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present disclosure. does not Hereinafter, various examples of the present disclosure will be described in detail with reference to the accompanying drawings.

electronic device

In the present disclosure, the electronic device 100 may be for performing a learning operation of an artificial intelligence model for predicting a label with respect to input data. In the present disclosure, for convenience, deep structured learning (DNN), which is referred to as deep learning or deep learning, is described as an example of an artificial intelligence model, but the artificial intelligence model is not limited thereto, and may be a variety of machine learning models.

1 is a block diagram of an electronic device according to an example of the present disclosure.

Referring to FIG. 1 , an electronic device 100 according to an example of the present disclosure may include an input/output device 110 , a transceiver 120 , a processor 130 , and a memory 140 . Here, the input/output device 110 and/or the transceiver 120 may be omitted depending on the implementation method of the electronic device 100 .

The input/output device 110 may be various interfaces, auxiliary devices, connection ports, etc. for receiving a user input from or outputting information to the user of the electronic device 100 . For example, in the present disclosure, the user's input may be input data, and the information output to the user may be output data and/or an artificial intelligence model according to the input data. The input/output device 110 may include an input device and an output device. The input device may be a comprehensive concept including various types of input means for sensing or receiving a user input. The output device may be a comprehensive concept including various types of output means for providing various types of data according to a user's input to the user.

The transceiver 120 may be connected to the processor 130 and may transmit and/or receive wired/wireless signals. For example, the transceiver 120 may be connected to various user terminals through a wireless communication network. Here, the wired/wireless signal may be input data, output data according to the input data, and/or an artificial intelligence model, and the user terminal selects at least one of a terminal providing input data and a terminal receiving output data according to the input data. may include Here, the wireless communication network may include a mobile communication network, a wireless LAN, a local area wireless communication network, and the like. For example, the wireless communication network includes LTE, LTE Advance (LTE-A), code division multiple access (CDMA), wideband CDMA (WCDMA), universal mobile telecommunications system (UMTS), Wireless Broadband (WiBro), or Global System (GSM). for Mobile Communications) and the like. For example, the wireless communication network may include at least one of wireless fidelity (WiFi), Bluetooth, Bluetooth low energy (BLE), Zigbee, near field communication (NFC), and radio frequency (RF).

The transceiver 120 may include a transmitter and a receiver. The transceiver 120 may be used interchangeably with a radio frequency (RF) unit. The transceiver 120 may transmit/receive various signals to and from the user terminal under the control of the processor 130 .

The processor 130 controls the input/output device 110 , the memory 140 , and/or the transceiver 120 , and may be configured to implement the descriptions, functions, procedures, proposals, methods, and/or operational flowcharts of the present disclosure. can For example, the processor 130 may receive input data through the input/output device 110 and store the input data in the memory 140 . Also, the processor 130 may train an artificial intelligence model according to input data, and store the learned artificial intelligence model in the memory 140 . In addition, the processor 130 may receive a radio signal through the transceiver 120 , and store information included in the radio signal in the memory 140 . In addition, the processor 130 may generate a wireless signal by processing information stored in the memory 140 , and then transmit the generated wireless signal through the transceiver 120 .

The memory 140 may be connected to the processor 130 and may store various information related to the operation of the processor 130 . For example, the memory 140 provides instructions for performing some or all of the processes controlled by the processor 130 , or for performing the descriptions, functions, procedures, suggestions, methods, and/or operational flowcharts of the present disclosure. It may store software code including (instructions). Also, the memory 140 may receive input data from the input/output device 110 and store the input data.

Hereinafter, various operation examples of the electronic device 100 will be described. The following various operation examples may be included in the operation of the at least one processor 130 described above.

2 is a block diagram of modules included in an electronic device according to an example of the present disclosure, and FIG. 3 is a conceptual diagram illustrating operations performed by modules included in an electronic device according to an example of the present disclosure.

The plurality of modules shown in FIG. 2 may be computer code or one or more instructions implemented so that the physical configuration of the electronic device 100 (eg, the processor 130 and the memory 140, etc.) performs a specified operation. . That is, the plurality of modules may be functional components stored in the memory 140 in the form of computer code, and when executed, the processor 130 may perform a specified operation. In other words, various operations performed by the plurality of modules of FIG. 2 may be operations performed by the processor 130 .

Referring to FIG. 2 , the electronic device 100 includes a calculation module 210 , a weight update module 220 , a control module 230 , and a threshold value selection module 240 .

The operation module 210 may perform a forward propagation operation and a backward propagation operation to perform a learning operation of the artificial intelligence model for predicting a label with respect to input data.

)을 수행할 수 있다. 여기서, A_in은 입력 활성화, A_out은 출력 활성화, W는 가중치를 의미한다.Referring to FIG. 3 , the forward propagation operation may refer to an operation of calculating and storing variables in order from an input layer to which input data of an artificial intelligence model is input to an output layer. For example, when a plurality of layers are included in the artificial intelligence model, the forward propagation operation is performed by the first convolution operation (

) can be done. Here, A _in denotes input activation, A _out denotes output activation, and W denotes a weight.

According to the forward propagation operation, a predicted reliability value and a loss value for the input data may be calculated. For example, the prediction reliability value may be calculated for each label, and the loss value may be calculated based on the prediction reliability value and the ground truth.

)을 수행할 수 있다. 여기서, L_in은 입력 손실 그래디언트이고, L_out은 출력 손실 그래디언트를 의미할 수 있다.The backpropagation operation may refer to an operation of calculating a gradient while propagating a loss inversely with respect to parameters of the artificial intelligence model. For example, the back propagation operation is performed by the second convolution operation (

) can be done. Here, L _in may mean an input loss gradient, and L _out may mean an output loss gradient.

)이 가중치 업데이트 모듈(220)에 의해 수행되는 것으로 도시되어 있으나, 제3 컨볼루션 연산은 역전파 동작에서 수행될 수도 있다. 여기서, W_G는 가중치 그래디언트일 수 있다.Meanwhile, in FIG. 3, the third convolution operation (

) is shown to be performed by the weight update module 220, the third convolution operation may also be performed in a backpropagation operation. Here, W _G may be a weight gradient.

The operation module 210 may perform the aforementioned forward propagation operation and/or backpropagation operation on input data in mini-batch units. Here, a mini-batch is a unit having B sub-input data included in the input data, and B may be a natural number (eg, a multiplier of 2, etc.) as a batch size.

For example, the operation module 210 may obtain a prediction reliability value for a label by performing a forward propagation operation in units of mini-batch.

The calculation module 210 may compare the prediction reliability value of the lower input data of the mini-batch with the prediction reliability threshold value. According to the comparison result, the operation module 210 may perform the backpropagation operation only on lower-level input data whose prediction reliability value is equal to or greater than the prediction reliability threshold value among the lower-level input data. In other words, the operation module 210 may omit the backpropagation operation with respect to the lower input data whose prediction reliability value is less than the prediction reliability threshold value among the lower input data. In this disclosure, a weight gradient obtained when the backpropagation operation is omitted for some lower input data may be referred to as an approximate weight gradient. That is, the calculation module 210 may omit the backpropagation operation for some lower input data according to the comparison of the prediction reliability value and the prediction reliability threshold value, and obtain an approximate weight gradient with respect to the weight of the artificial intelligence model.

The weight update module 220 may update the weight based on the weight gradient obtained from the operation module 210 or the approximate weight gradient. For example, when the backpropagation operation is not omitted for some lower input data, the weights may be updated based on the weight gradient, and if the backpropagation operation is omitted, the weights may be updated based on the approximate weight gradient.

The weight update module 220 may update the weight based on a stochastic gradient descent (SGD) algorithm. For example, the weight update module 220 may update the weight by subtracting a value obtained by multiplying the weight gradient or the approximate weight gradient by the learning rate from the existing weight.

The weight update module 220 may obtain _{G high} and G _low used for calculating an average absolute gradient, which will be described later, through weight update. Here, G _high is the accumulation of absolute values of the weight gradient in the highest confidence interval, and G _low is the accumulation of absolute values of the weight gradient in the lowest confidence interval. _{At this time, the weight update module 220 acquires G high} only in the first epoch section, using that the calculation result of the absolute value accumulation operation of the weight gradient of the high reliability section is constant throughout the entire learning process, and calculates _{G high in the} remaining epoch section can be omitted.

The control module 230 controls at least one of various parameters used in the calculation module 210 and the weight update module 220, for example, a batch size, a prediction confidence interval for a prediction reliability value, an error boundary value, and an epoch. One may be set, or the calculation module 210 and the weight update module 220 may be controlled through the parameters. Here, the epoch is a weight update period.

The threshold selection module 240 dynamically sets a prediction reliability threshold that is a criterion for omitting the backpropagation operation. Hereinafter, the operation of the threshold value selection module 240 will be described in detail with reference to FIGS. 4 and 5 .

4 is a block diagram of a threshold selection module according to an example of the present disclosure.

Referring to FIG. 4 , the threshold value selection module 240 according to an example of the present disclosure includes a CF accumulation module 241 , a linear interpolation module 242 , and a threshold value determination module 243 .

The CF accumulation module 241 accumulates a confidence distribution for the confidence interval based on the predicted confidence value. For example, the confidence distribution may be accumulated for each confidence interval for all epochs with respect to the input data.

The linear interpolation module 242 may calculate an average absolute gradient based on the linear interpolation method. The absolute value of the weight gradient has a linear characteristic when it is accumulated for each confidence interval. The linear interpolation module 242 may calculate the average absolute gradient using only the accumulation of absolute values of the weight gradient in the lowest confidence interval and the highest confidence interval by using the linear characteristic of the absolute value of the weight gradient.

에 기초하여 평균 절대 그래디언트를 산출할 수 있다. 여기서, G_high는 가장 높은 신뢰도 구간에서의 가중치 그래디언트의 절대값의 누적, G_low는 가장 낮은 신뢰도 구간에서의 가중치 그래디언트의 절대값의 누적, N은 신뢰도 구간의 개수, Th는 예측 신뢰도 임계값일 수 있다.For example, the linear interpolation module 242 may

An average absolute gradient can be calculated based on . Here, G _high is the accumulation of absolute values of the weight gradient in the highest confidence interval, G _low is the accumulation of absolute values of the weight gradient in the lowest confidence interval, N is the number of confidence intervals, and Th is the prediction reliability threshold. have.

The threshold determination module 243 dynamically determines the threshold based on the cumulative and average absolute gradients of the confidence distribution.

에 의해 정의될 수 있으며, CF(k)는 신뢰도 분포의 누적일 수 있다.Specifically, the threshold determination module 243 initializes an _{arbitrary prediction confidence threshold and RSE skip .} Here, RSE _skip is a weight gradient error value according to the omission of the backpropagation operation, and is a preset scaling factor, the number of sub-input data whose prediction reliability value is greater than or equal to the prediction reliability threshold value among the preset scaling factors, and the average absolute gradient of the prediction reliability value. It can be defined as a product. Here, among the sub-input data, the number of sub-input data whose prediction reliability value is equal to or greater than the prediction reliability threshold is

may be defined by , and CF(k) may be the accumulation of the reliability distribution.

The threshold value determination module 243 decreases the random prediction reliability threshold by a specific variation value unit, and calculates _{RSE skip whenever the arbitrary prediction reliability threshold decreases by a specific variation value unit.} Here, the specific variation value may be a preset value.

The threshold value determination module 243 _{decreases the random prediction reliability threshold by a specific variation value unit until the RSE skip first} becomes equal to or exceeds the preset error boundary value for the first time, and the random prediction reliability threshold value becomes the specific variation value It is possible to repeatedly perform the operation of calculating the _{RSE skip} whenever the unit is decreased.

The threshold value determination module 243 _{finally sets an arbitrary prediction reliability threshold value when RSE skip first} becomes equal to or exceeds a preset error boundary value as the prediction reliability threshold value.

5 is a conceptual diagram for explaining an operation performed by a threshold value selection module according to an example of the present disclosure.

Referring to FIG. 5 , a reliability distribution may be accumulated for a reliability interval (eg, 0 to 9) by the CF accumulation module 241 . In this case, each reliability interval may have a range according to the number of reliability intervals.

Based on the accumulated confidence distribution, _{it may be determined whether the RSE skip} is equal to or exceeds a preset error boundary value by the threshold value determination module 243 . For example, when the prediction reliability threshold value is 0.9, RSE _skip is calculated _{for the reliability distribution accumulation of 0.9 or more, and it may be determined whether the calculated RSE skip} is equal to or exceeds a preset error boundary value.

In this case, _{the average absolute gradient used to obtain the RSE skip} may be obtained by the linear interpolation module 242 . For example, the linear interpolation module 242 may obtain an average absolute gradient by applying a linear interpolation method to the entire confidence interval (eg, 0 to 9).

According to the electronic device 100 of the present disclosure described above, when the prediction reliability value is greater than the prediction reliability threshold value, it is determined that the weight gradient obtained through the backpropagation operation (and/or weight update) does not significantly contribute to learning. , omitting the backpropagation operation (and/or weight update) can save learning time and learning energy. In addition, the prediction reliability threshold may be dynamically set according to the allowable noise level.

learning method

Hereinafter, a learning method performed by the electronic device 100 according to various examples of the present disclosure will be described. A detailed description of the parts overlapping with those described above will be omitted.

6 is a flowchart of a learning method according to an example of the present disclosure.

Referring to FIG. 6 , in S110 , the electronic device 100 performs a forward propagation operation on input data in mini-batch units to obtain a predicted reliability value for the label. Here, the mini-batch has B sub-input data included in the input data, and B is a natural number.

In S120 , the electronic device 100 performs a backward propagation operation only on lower input data in which the prediction reliability value is equal to or greater than the prediction reliability threshold value among the lower input data to obtain an approximate weight gradient for the weight of the artificial intelligence model. ) is obtained. Here, the backpropagation operation for lower input data whose prediction reliability value is less than the prediction reliability threshold value among the lower input data may be omitted.

In S130, the electronic device 100 updates the weight based on the approximate weight gradient.

7 is a flowchart of a method of setting a prediction reliability threshold included in a learning method according to an example of the present disclosure.

In S210 , the electronic device 100 initializes an _{arbitrary prediction reliability threshold and RSE skip .} Here, RSE _skip is a weight gradient error value according to the omission of the backpropagation operation, and is a preset scaling factor, the number of sub-input data whose prediction reliability value is greater than or equal to the prediction reliability threshold value among the preset scaling factors, and the average absolute gradient of the prediction reliability value. It can be defined as a product. Here, the average absolute gradient may be defined based on a linear interpolation method for the entire confidence interval of the prediction reliability value.

In S220 , the electronic device 100 reduces the random prediction reliability threshold by a specific variation value unit, and calculates _{RSE skip whenever the random prediction reliability threshold decreases by a specific variation value unit.}

In S230, when the electronic device 100 RSE _skip the group or by the same first error threshold value is set, or determining whether the first time out and, RSE _skip that or equal first error threshold preset or not greater than the first , _{S220 may be repeated until the RSE skip first} becomes equal to or exceeds the preset error boundary value for the first time.

In S240 , the electronic device 100 sets _{an arbitrary prediction reliability threshold value when RSE skip first} becomes equal to or first exceeds a preset error boundary value as the prediction reliability threshold value.

Experimental example

Hereinafter, experimental examples for parameters used in various examples of the present disclosure described above will be described.

8 is a graph illustrating a relationship between a prediction reliability value and an absolute value of a weight gradient according to an example of the present disclosure, and FIG. 9 is a graph illustrating a relationship between an average prediction reliability value and an epoch according to an example of the present disclosure. .

8 and 9 , the absolute value of the weight gradient decreases linearly as the prediction reliability value increases regardless of the batch size or epoch, and the average prediction reliability value decreases irrespective of the type of input data (CIFAR) epoch increases as the In this case, the absolute value of the weight gradient rarely occurs when the prediction reliability value is high. _{Accordingly, the above-described weight update module 220 obtains G high} only in the first epoch period, using that the calculation result of the absolute value accumulation operation of the weight gradient in the high reliability interval is constant throughout the entire learning process, and in the remaining epoch intervals The G _high operation can be omitted.

Since examples of the proposed method in the above description may also be included as one of the implementation methods of the present disclosure, it is clear that they may be regarded as a kind of proposed method. In addition, the above-described proposed methods may be implemented independently, but may also be implemented in the form of a combination (or merge) of some of the proposed methods.

The examples of the present disclosure disclosed as described above are provided to enable those skilled in the art to implement and practice the present disclosure. Although the above has been described with reference to examples of the present disclosure, those skilled in the art may variously modify and change the examples of the present disclosure. Accordingly, this disclosure is not intended to be limited to the examples set forth herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

100: electronic device
110: input/output device 120: transceiver
130: processor 140: memory
210: calculation module 220: weight update module
230: control module 240: threshold selection module

Claims (10)

at least one processor; and
at least one memory operatively coupled to the at least one processor to store at least one instructions for causing the at least one processor to perform operations;
The actions are
performing a learning operation of an artificial intelligence model for predicting a label with respect to the input data stored in the at least one memory;
The learning operation is
a mini-batch for the input data, wherein the mini-batch has B sub-input data included in the input data, wherein B is a natural number; To obtain a prediction reliability value for the label by performing a forward propagation operation as a unit,
Obtaining an approximate weight gradient with respect to the weight of the artificial intelligence model by performing a backward propagation operation only on lower input data in which the prediction reliability value is greater than or equal to the prediction reliability threshold value among the lower input data,
updating the weights based on the approximate weight gradient;
electronic device.
According to claim 1,
The learning operation is
Omitting the backpropagation operation with respect to lower-order input data in which the prediction reliability value is less than the prediction reliability threshold value among the lower-level input data,
electronic device.
3. The method of claim 2,
The learning operation is
and a prediction reliability threshold setting operation of setting the prediction reliability threshold,
The prediction reliability threshold setting operation is,
(a) an arbitrary prediction reliability threshold and RSE _skip, wherein the RSE _skip is a weight gradient error value according to the omission of the backpropagation operation; an operation of initializing;
(b) reducing the random prediction reliability threshold by a specific variation value unit, and calculating _{the RSE skip whenever the random prediction reliability threshold value is decreased by the specific variation value unit;}
(c) _{repeatedly performing the operation (b) until the RSE skip first} becomes equal to or exceeds the preset error boundary value for the first time;
(d) setting the _{random prediction reliability threshold value when the RSE skip first} becomes equal to or first exceeds a preset error boundary value as the prediction reliability threshold value,
electronic device.
4. The method of claim 3,
The RSE _skip is defined as a product of a preset scaling factor, the number of lower input data in which the prediction reliability value is greater than or equal to the prediction reliability threshold value among the lower input data, and the average absolute gradient of the prediction reliability value,
electronic device.
5. The method of claim 4,
wherein the mean absolute gradient is defined based on linear interpolation over the entire confidence interval of the prediction confidence value;
electronic device.
A learning method of an artificial intelligence model for predicting a label with respect to input data stored in at least one memory performed by an electronic device, the method comprising:
a mini-batch for the input data, wherein the mini-batch has B sub-input data included in the input data, wherein B is a natural number; obtaining a prediction reliability value for the label by performing a forward propagation operation in units;
obtaining an approximate weight gradient for the weight of the artificial intelligence model by performing a backward propagation operation only on lower input data in which the prediction reliability value is greater than or equal to a prediction reliability threshold value among the lower input data; and
updating the weights based on the approximate weight gradient;
How to learn.
7. The method of claim 6,
The back-propagation operation of the lower input data for which the prediction reliability value is less than the prediction reliability threshold value among the lower input data is omitted,
How to learn.
8. The method of claim 7,
Further comprising the step of setting a prediction reliability threshold for setting the prediction reliability threshold,
The step of setting the prediction reliability threshold is:
(a) initializing a random prediction reliability threshold and RSE _skip, wherein the RSE _skip is a weighted gradient error value according to the omission of the backpropagation operation;
(b) reducing the random prediction reliability threshold by a specific variation value unit, and calculating _{the RSE skip whenever the random prediction reliability threshold decreases by the specific variation value unit;}
(c) _{repeatedly performing the operation (b) until the RSE skip first} becomes equal to or exceeds a preset error boundary value; and
(d) setting the _{random prediction reliability threshold value when the RSE skip first} becomes equal to or first exceeds a preset error boundary value as the prediction reliability threshold value,
How to learn.
9. The method of claim 8,
The RSE _skip is defined as a product of a preset scaling factor, the number of lower input data in which the prediction reliability value is greater than or equal to the prediction reliability threshold value among the lower input data, and the average absolute gradient of the prediction reliability value,
How to learn.
10. The method of claim 9,
wherein the mean absolute gradient is defined based on linear interpolation over the entire confidence interval of the prediction confidence value;
How to learn.

Images