KR20220060374A - Learning method, apparatus, computer readable recording medium for imputating missing data and electronic device using the method - Google Patents

Abstract

Disclosed is a loss inclusion learning device that executes an on-board artificial intelligence (AI) algorithm and/or a machine learning algorithm. The present device may perform learning for loss inclusion using a generative adversarial network (GAN) and a recursive coherent network. Therefore, the loss data can be included with high accuracy. An electronic device comprises: a memory; and a controller.

Description

LEARNING METHOD, APPARATUS, COMPUTER READABLE RECORDING MEDIUM FOR IMPUTATING MISSING DATA AND ELECTRONIC DEVICE USING THE METHOD

The present disclosure relates to machine learning and artificial intelligence technology, and more particularly, a loss-inclusion learning method for generating a loss-inclusion model, a learning apparatus for this, a computer-readable recording medium storing the learning method, and an electronic device to which the learning method is actually applied It's about devices.

Deep learning models are widely used in technology fields such as data classification and prediction.

In order for a deep learning model to perform well, a sufficient amount of data is required, and in reality, it is difficult to obtain complete data without loss.

Accordingly, there is a need for a method of imputation of the generated loss values as accurately as possible.

Prior art: Patent Publication No. 10-2018-0120478 (published date: 2018,11.06)

An object of the present invention is to provide a loss-included learning method and apparatus using both complete data in which no loss has occurred and loss data having a predetermined loss pattern in the learning step.

Another object of the present invention is to provide a loss inclusion learning method and apparatus for performing inclusion of various loss types using a network that generates a loss.

Another object of the present invention is to provide a loss inclusion learning method and apparatus that well reflects the actual conditional probability distribution for a non-lossy part.

Another object of the present invention is to provide a method and apparatus for learning loss calculation for improving loss calculation accuracy by using a cyclically consistent loss function.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the description below. will be able

In order to achieve the above object, the electronic device according to an embodiment of the present invention has a memory and loss storing a loss inclusion model learned in advance using a first Generative Adversarial Network (GAN), a second GAN, and a cyclic coherent network. The controller may include a controller that inputs a lossy image for testing including data into a loss inclusion model and generates a lossless image for testing in which the loss data is included as replacement data.

In addition, the loss inclusion learning method performed by the processor according to an embodiment of the present invention includes the steps of receiving a lossy reference image including one or more lossless reference images and loss data prepared in advance and corresponding to the lossless reference image; and updating the parameters of the first GAN and the second GAN for each batch period such that the sum of the first loss function of the first GAN, the second loss function of the second GAN, and the third loss function of the cyclically coherent network are minimized. may include

In addition, the loss-included learning method according to an embodiment of the present invention may be stored in a computer-readable recording medium in which a program to be executed on a computer is recorded.

When the program is executed by the processor, the processor receives one or more lossless reference images and one or more lossless reference images corresponding to the lossless reference images, including one or more previously prepared lossless reference images, and a first loss of the first GAN. function, the second loss function of the second GAN, and the third loss function of the recursive coherent network to minimize the sum of the parameters of the first GAN and the second GAN for each batch period. may include

The means of solving the technical problems to be achieved in the present invention are not limited to the solutions mentioned above, and other solutions not mentioned are clear to those of ordinary skill in the art to which the present invention belongs from the description below. can be understood clearly.

According to various embodiments of the present disclosure, since complete data in which no loss has occurred and loss data having a predetermined loss pattern are both used in the learning step, loss calculation accuracy may be improved.

In addition, the inclusion of various loss types can be performed with high accuracy, and the actual conditional probability distribution for the non-lossed part can be well reflected.

In addition, loss calculation accuracy can be improved by using the loss function of the recursive coherent network.

1 is a diagram for schematically explaining an operation of an electronic device to which a learned loss inclusion model is applied according to an embodiment of the present invention;
2 is a relative block diagram for explaining the configuration of an electronic device to which a loss inclusion learning apparatus and a learned loss inclusion model are applied according to an embodiment of the present invention;
3 is a view for explaining in detail the operation of the loss inclusion learning apparatus according to an embodiment of the present invention, and,
4 is a sequence diagram illustrating a loss inclusion learning method according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described with reference to the accompanying drawings. The present invention may be implemented in a variety of different forms, and should not be construed as being limited to the embodiments described herein. The relative sizes of components, layers, and regions in the drawings may be exaggerated for clarity of description.

1 is a diagram schematically illustrating an operation of an electronic device 200 to which a learned loss inclusion model 155 is applied according to an embodiment of the present invention.

First, before explaining in detail, some terms will be summarized. “Loss” means that there is lost data in the image data, and outputting the image data to a display may cause problems. In an optional embodiment, the loss data may form a predetermined pattern, and may be masked (TMAa) and output on the display. Imputation is inputting lossy data as replacement data for a lossy image including lossy data, and the imputation performance may be determined based on matching accuracy between the incorporated image and the lossless original image.

The electronic device 200 may be an apparatus (eg, a laptop, a smart phone, a television, etc.) using the trained loss inclusion model 155 . Specifically, the electronic device 200 inputs the lossy test image TMI including the loss data TMAa into the loss inclusion model 155, and the loss data TMAa is included as the replacement data. TNMI) can be output.

Here, the loss inclusion model 155 is a model generated by the loss inclusion learning apparatus ( 100 in FIG. 2 ). In an embodiment, the electronic device 200 may be implemented with the same device as the loss inclusion training apparatus 100 in which the loss inclusion model 155 is trained.

FIG. 2 is a relative block diagram for explaining the configuration of the apparatus 100 for learning loss inclusion according to an embodiment of the present invention and the electronic device 200 to which the learned loss inclusion model 155A is applied.

The loss calculation learning apparatus 100 is an apparatus for training the loss calculation model 155 , and may include a communication unit 110 , a memory 150 , and a processor 190 , and the electronic device 200 includes a communication unit 210 . , a memory 250 and a controller 290 may be included. The components shown in FIG. 2 are not essential in implementing the loss inclusion learning apparatus 100 and the electronic device 200, so the loss inclusion learning apparatus 100 and the electronic device 200 described herein are described above. It may have more or fewer elements than those listed.

The loss inclusion learning apparatus 100 may include a first Generative Adversarial Network (GAN) and a second GAN each including a generator and an identifier (Discriminator). In addition, the loss inclusion learning apparatus 100 may train the loss inclusion model 155 using a cyclic coherent network.

Here, the GAN may be a network that performs class classification with high accuracy by performing adversarial learning between a generator and an identifier as a method of unsupervised learning.

The loss inclusion learning apparatus 100 may improve loss inclusion performance by using one or more previously prepared lossless reference images and lossy reference images. A reference image (lossless or lossy) refers to an image that is prepared in advance and used as a reference without being generated by a GAN or a circular coherent network.

Here, the lossless reference image is an original image without lossy data, an image prepared in advance before learning, and an image not generated by the generator of the GAN. The lossy reference image is an original image including loss data, is an image prepared in advance before learning, is an image corresponding to the lossless reference image, and is an image not generated by the generator of the GAN.

In an optional embodiment, the lossless reference image and the lossy reference image may be images of a Modified National Institute of Standards and Technology Database (MNIST).

The loss inclusion learning apparatus 100 may improve inclusion accuracy by using both a lossless reference image, which is complete data in which no loss has occurred, and a lossless reference image, which is lossy data having a predetermined loss pattern, in the learning step. This overcomes the limitations of the prior art, in which the counting accuracy is not very high by performing learning only with lossy images or by performing learning only with lossless images.

The processor 190 of the loss inclusion training apparatus 100 may train the loss inclusion model 155 using the first GAN, the second GAN, and the cyclic coherent network.

The first GAN may include a first generator and a first identifier, and the second GAN may include a second generator and a second identifier. The circular coherent network may use both the first generator of the first GAN and the second generator of the second GAN.

Specifically, the first GAN generates a lossy reference image prepared in advance as a first lossless image through the first generator, and identifies the first lossless image and the lossless reference image based on the first loss function through the first identifier. can

That is, after the first GAN generates a lossy reference image including lossy data having a predetermined pattern as a lossless image, the first GAN may determine a difference between the generated lossless image and the lossless reference image. The discrimination degree of the first GAN may be determined by the value of the first loss function.

When the first generator of the first GAN generates a lossless image, it is configured to learn so that there is no difference from the lossless reference image, and the first identifier learns to precisely distinguish the lossless reference image from the lossless image generated by the first generator. is the composition The first generator and the first identifier may allow the first generator to generate a lossless image closer to the lossless reference image through adversarial learning.

Next, the second GAN may generate a lossless reference image prepared in advance through the second generator as a loss image, and identify the generated lossy image and the lossy reference image based on the second loss function.

That is, after the second GAN makes a lossless reference image without loss data into a lossy image, the second GAN may determine the difference between the lossy image and the lossy reference image generated by the GAN. The discrimination degree of the second GAN may be determined by the value of the second loss function.

The second generator of the second GAN is configured to learn so that there is no difference from the loss reference image when generating the loss image, and the second identifier is a configuration to learn to precisely distinguish the loss image and the loss reference image. The second generator and the second identifier may allow the second generator to generate a lossy image closer to the lost reference image through adversarial learning.

Finally, the processor 190 inputs the lossy image generated by the second generator of the second GAN to the first generator of the first GAN using the cyclic coherent network to obtain a second lossless image (the first described above). After the identifier is generated as a first lossless image (differentiated from the generated first lossless image), the second lossless image and the lossless reference image may be identified based on the third loss function. Discriminant information of the cyclically coherent network may be determined by a value of the third loss function.

The processor 190 may reset the parameters of the first loss function and the second loss function so that the sum of the first to third loss functions is minimized for each batch period. That is, the processor 190 may increase the counting accuracy through the two GANs, and finally, the counting accuracy may be further increased by applying the cyclic coherent network.

The processor 190 may train the loss inclusion model 155 for each batch period until data input is completed. When the learned loss inclusion model 155 receives the lossy image TMI in the application step as shown in FIG. 1 , the electronic device 200 may output the lossless image TNMI.

The loss inclusion learning apparatus 100 may provide the learned loss inclusion model 155 to the communication unit 210 of the electronic device 200 through the communication unit 110 , and the electronic device 200 provides the provided loss inclusion model 155A may be stored in memory 250 .

The controller 290 of the electronic device 200 may generate a test lossless image in which the loss data is included as replacement data by inputting the test loss image including the loss data into the learned loss inclusion model 155A. there is.

3 is a diagram for explaining in detail the operation of the loss inclusion learning apparatus 100 according to an embodiment of the present invention.

The processor 190 considers the loss data as replacement data for the loss image including the loss data using the first GAN (N1), the second GAN (N2), and the cyclic coherent network (N3) to optimize the lossless A loss inclusion model 155 for generating an image may be generated.

The processor 190 may perform on all input data, and may control each network N1 to N3 to perform learning in units of a batch period.

First, looking at the first GAN N1 , the processor 190 may generate the first lossless image NMIa by inputting the lossy reference image RMI to the first generator N1G.

At this time, the processor 190 prepares the lost reference image RMI in advance, but the lost reference image RMI may be represented by a pair of the first mask M1 indicating the observed portion and the lost position of the lost real data. there is. The first generator N1G may also receive a latent variable (Z, Latent Vector) randomly generated from a normal distribution as an input, and may stochastically calculate an output value.

The processor 190 provides the generated first lossless image NMIa as a first identifier N1D, and the first identifier N1D compares the lossless reference image RNMI and the first lossless image NMIa. The first identifier NID may be controlled.

Specifically, the processor 190 may calculate the value of the first loss function (V ₁ ), which is the discriminant of the GAN, according to [Equation 1] below.

[Equation 1]

V ₁ = log(D _θ (x)) + log(1-D _θ (x'))

Here, x is data of the lossless reference image RNMI, x' is data of the first lossless image NMIa generated by the first generator N1G, and θ is a parameter of the first loss function.

Next, looking at the second GAN, the processor 190 may generate the lossy image MI by inputting the lossless reference image RNMI to the second generator N2G.

In this case, the processor 190 may prepare the lossless reference image RNMI in advance and set the loss data as the second mask M2 by using a latent vector (Z, Latent Vector) following a normalized distribution.

The processor 190 may generate the lossy image MI by applying the generated second mask M2 to the lossless reference image RNMI using the second generator N2G.

The processor 190 provides the generated lossy image MI as a second identifier N2D, and the second identifier N2D compares the lossy reference image RMI and the lossy image MI with the second identifier N2D. ) can be controlled.

Specifically, the processor 190 may calculate the value of the second loss function (V ₂ ), which is the discriminant of the GAN algorithm, based on [Equation 2] below.

[Equation 2]

V ₂ = log(D _φ (y)) + log(1-D _φ (y'))

Here, y is the data of the loss reference image RMI reflecting the first mask M1, y' is the loss image MI reflecting the second mask M2, and φ is the second loss function V ₂ ) is a parameter of

Next, the processor 190 may provide the loss image MI generated by the second GAN N2 to the first generator N1G of the circular coherent network N3 . Here, the first generator N1G is the same as the first generator N1G of the first GAN N1 and may generate the second lossless image NMIb.

The cyclic coherent network N3 may compare the lossless reference image RNMI and the second lossless image RNMIb.

Specifically, the cyclic coherent network N3 may calculate the value of the third loss function V ₃ based on Equation 3 below.

[Equation 3]

V ₃ = ||x" - x||

Here, x" is data of the second lossless image RNMIb, and x is data of the lossless reference image RNMI. That is, the cyclic coherent network N3 converts the lossless reference image RNMI to the second GAN (N2). ) to output a lossy image (MI) by inputting the second generator (N2G) of The second lossless image NMIb and the lossless reference image RNMI may be compared by using the generated NMIb, so that counting accuracy may be further improved.

The processor 190 may sum the values of the first to third loss functions V ₁ to V ₃ and update the parameters θ and φ of the first GAN and the second GAN so that the sum is minimized. (S3).

The process of FIG. 3 describes a process performed during one batch period, and learning may be repeatedly performed on all data. Then, the loss inclusion model 155 may be finally generated.

4 is a sequence diagram illustrating a loss inclusion learning method performed by the processor 190 of the loss inclusion learning apparatus 100 according to an embodiment of the present invention.

First, the processor 190 receives a lossless reference image and a lossy reference image (S410).

Here, the lossless reference image and the lossy reference image may be prepared in advance during each deployment period. In an embodiment, the processor 190 may be implemented to receive both the lossless reference image and the lossy reference image at the beginning of learning.

Also, the lossy reference image corresponds to the lossless reference image and may include lossy data.

After step S410 (reference image input), the processor 190 performs the first loss function such that the sum of the first loss function of the first GAN, the second loss function of the second GAN, and the third loss function of the cyclic coherent network is minimized. The parameters of the first GAN and the second GAN may be updated for each deployment period (S450).

Each step ( S42 , S43 , S44 ) between steps S410 and S450 will be described below.

First, the processor 190 performs a first GAN after the reference image input step (step S410) (S42 and S420).

Specifically, the first generator of the first GAN generates the lossy reference image as the first lossless image (S421), and the first identifier of the first GAN is based on the first loss function, the first lossless image and the lossless reference image is identified (S423).

Also, the processor 190 performs a second GAN after the reference image input step (step S410) (S43 and S430).

Specifically, the second generator of the second GAN generates a lossless reference image as a loss image (S431), and the second identifier of the second GAN identifies the generated lossy image and the lossy reference image based on the second loss function. do (S433).

Here, steps S420 (including steps S421 and S423) and S430 (including steps S431 and S433) may be performed in a reverse order. That is, the processor 190 may perform the process in the order of steps S431, S433, S421, and S423.

On the other hand, the loss inclusion learning method according to an embodiment of the present invention may be stored in a computer-readable recording medium in which a program to be executed on a computer is recorded.

When the program is executed by a processor, the processor receives one or more lossless reference images and one or more lossless reference images corresponding to the lossless reference images including lossless data and a first loss function of the first GAN, and an executable instruction to update the parameters of the first GAN and the second GAN for each batch period so that the sum of the second loss function of the second GAN and the third loss function of the cyclically coherent network is minimized. can do.

In the foregoing, specific embodiments of the present invention have been described and illustrated, but the present invention is not limited to the described embodiments, and those of ordinary skill in the art may make various changes to other specific embodiments without departing from the spirit and scope of the present invention. It will be appreciated that modifications and variations are possible. Accordingly, the scope of the present invention should not be defined by the described embodiments, but should be defined by the technical idea described in the claims.

Claims (11)

An electronic device comprising:
a memory storing a loss inclusion model pre-trained using a first Generative Adversarial Network (GAN), a second GAN, and a recursive coherent network; and
and a controller configured to input a lossy test image including loss data into the loss inclusion model to generate a test lossless image in which the loss data is included as replacement data. According to claim 1,
The loss inclusion model is in the learning stage,
Receive one or more lossless reference images and lossy reference images prepared in advance, wherein the lossy reference image includes lossy data and corresponds to the lossless reference image,
The electronic device is trained such that the sum of the first loss function of the first GAN, the second loss function of the second GAN, and the third loss function of the cyclically coherent network is minimized for each deployment period. 3. The method of claim 2,
The first GAN includes a first generator (Generator) and a first identifier (Discriminator),
The first constructor,
generating the lossy reference image as a first lossless image;
The first identifier is
and identifying the first lossless image and the lossless reference image based on the first loss function. 4. The method of claim 3,
The second GAN includes a second generator and a second identifier;
The second constructor,
generating the lossless reference image as a lossy image;
The second identifier is
and identifying a generated lossy image and the lossy reference image based on the second loss function. 5. The method of claim 4,
The circular coherent network is
generating a second lossless image by inputting the lossy image generated by the second generator of the second GAN to the first generator of the first GAN;
and identifying the second lossless image and the lossless reference image based on the third loss function. A loss inclusion learning method performed by a processor, comprising:
receiving a lossy reference image including at least one lossless reference image and lossy data provided in advance and corresponding to the lossless reference image; and
The parameters of the first GAN and the second GAN are adjusted so that the sum of the first loss function of the first Generative Adversarial Network (GAN), the second loss function of the second GAN, and the third loss function of the cyclic coherent network is minimized. A method of learning with loss counting, comprising the step of updating every batch period. 7. The method of claim 6,
After the step of receiving the input,
a first step of generating, by a first generator of a first GAN, the lossy reference image as a first lossless image; and
The method further comprising a second step of identifying the first lossless image and the lossless reference image based on the first loss function by the first identifier of the first GAN. 8. The method of claim 7,
After receiving the input,
a third step of generating, by a second generator of a second GAN, the lossless reference image as a lossy image; and
According to the second identifier of the second GAN, based on the second loss function, the loss inclusion learning method further comprising a fourth step of identifying the generated loss image and the loss reference image. 9. The method of claim 8,
After the fourth step,
generating a second lossless image by inputting the lossy image generated by the second generator of the second GAN to the first generator of the first GAN; and
Based on the third loss function, the method further comprising: identifying the second lossless image and the lossless reference image. 9. The method of claim 8,
The process is performed in the order of the first to fourth steps, or
The process is performed in the order of the third step, the fourth step, the first step and the second step, the loss inclusion learning method. In a computer-readable recording medium recording a program for execution on a computer,
When the program is executed by a processor, the processor
an operation of receiving one or more lossless reference images including one or more lossless reference images and lossy data and corresponding to the lossless reference images; and
The parameters of the first GAN and the second GAN are adjusted so that the sum of the first loss function of the first Generative Adversarial Network (GAN), the second loss function of the second GAN, and the third loss function of the cyclic coherent network is minimized. A computer-readable recording medium comprising executable instructions for performing an operation of updating for each deployment period.

Images