KR102260039B1 - Method for training and testing obfuscation network capable of processing data to be concealment for privacy, and training device and testing device using the same - Google Patents

Abstract

The present invention relates to a method for learning a modulation network (Obfuscation Network) for processing original data by concealing for personal information protection, the learning apparatus allows the modulation network to modulate input learning data and modulated learning inputting data into the learning network to cause the learning network to apply a network operation to the modulated learning data to generate first characteristic information; causing the learning network to apply a network operation to the input learning data to generate second characteristic information; and an error obtained by referring to at least some of an error obtained by referring to the first and second characteristic information and an error obtained by referring to a task specific output and a corresponding ground truth (GT) to minimize the calculated error and learning the modulation network to maximize the error calculated by referring to the training data and the modulated training data.

Description

Method for learning and testing a tampering network that conceals original data for privacy protection, and a learning and testing device using the same AND TRAINING DEVICE AND TESTING DEVICE USING THE SAME}

The present invention provides a method for learning a modulation network for concealing original data for personal information protection, a learning apparatus using the same, and a modulation network trained to conceal original data for personal information protection. It relates to a test method and a test apparatus using the same.

Big data is not only standardized data used in existing corporate environments or public institutions, but also e-commerce data, metadata, web log data, Radio Frequency Identification (RFID) data, sensor network data, Data that includes unstructured or semi-structured data that has not been utilized before, such as social network data, Internet text and document data, and Internet search indexing data. It is called big data in the sense that it will have a level of data that is difficult to handle.

In addition, such big data may have no meaning by itself, but can be usefully used to create new data in various fields through machine learning on patterns, etc., or to make judgments or predictions.

In recent years, in order to trade or share such big data due to the strengthening of the Personal Information Protection Act, it is required to delete personally identifiable information of the data or to obtain the consent of the individual. However, it is difficult to check whether there is personally identifiable information among a considerable amount of big data, and it is impossible to obtain individual consent, so various technologies are emerging to support this.

As an example of the related art, the technology disclosed in Korean Patent Publication No. 1861520 can be exemplified. According to this, a detection step of detecting a human face region in an input image to be deformed, and a human control in the input image A first concealing step of transforming the detected face region into a distorted first image instead of a human face shape so as to be sealed, and a second image having a preset face shape is generated based on the first image, and the and transforming the first image into the second image in the input image, wherein the second image is generated to have a face shape different from that of the face region detected in the detection step. A sealing method is provided.

However, according to the prior art including the above technique, it is checked whether identification information such as a face or text is included on the data, and the part corresponding to the identification information is erased by masking or blurring. Because it was a method to hide or hide the original data, it was difficult to use in machine learning due to damage to the original data. In particular, the conventional security camera (Security Camera) performs de-identification processing by blurring all pixels having inter-frame changes in the video image. There was a problem that not only the main information such as facial expressions of the person was different from the information contained in the original video image, but also the personal identification information missing in the face detection process remained on the original video image. Also, the blurred video image may be restored to its original state using one of the conventional image deblurring techniques.

Accordingly, the inventor(s) is differently modulated from the original data to generate modulated data, but the original data is input to the machine learning model and output results, and the above modulated data is input to the machine learning model We propose a technique in which output results are the same or similar to each other.

An object of the present invention is to solve all the problems of the prior art described above.

In addition, another object of the present invention is to enable a simple and accurate concealing operation to be performed since a task of finding personal identification information in data is not required.

In addition, another object of the present invention is to be able to protect the security and privacy of the original data by irreversibly tampering with the original data to generate concealed data.

Another object of the present invention is to generate data that looks different to humans but looks identical or similar to each other to computers.

Another object of the present invention is to further activate the big data trading market.

A representative configuration of the present invention for achieving the above object is as follows.

According to an aspect of the present invention, in a method of learning a modulation network (Obfuscation Network) for processing original data by concealing for protection of personal information, (a) when learning data is obtained, the learning device, inputting training data into a modulation network to cause the modulation network to modulate the training data to generate modulated training data; (b) the learning device, (i) inputs the modulated training data to a learning network having at least one learned parameter to cause the learning network to (i-1) use the learned parameter By applying a network operation to the training data (i-2) to generate first characteristic information (Characteristic Information) corresponding to the modulated training data, (ii) by inputting the training data into the learning network (ii-1) causing a learning network to apply a network operation to the training data using the learned parameters (ii-2) to generate second characteristic information corresponding to the training data; and (c) at least one first_1 error obtained by referring to (i) (i-1) the first characteristic information and the second characteristic information, and (i-2) the first characteristic. At least one first calculated by referring to at least a portion of at least one first_2 error obtained by referring to a task specific output generated using the information and at least one ground truth (GT) corresponding thereto and (ii) learning the modulation network to minimize the error and to maximize the at least one second error calculated with reference to the training data and the modulated training data.

In one embodiment, the learning network comprises a first learning network to an n-th learning network having at least one first learned parameter to at least one n-th learned parameter, wherein when n is an integer greater than or equal to 1, the ( In step b), the learning device, (i) input the modulated training data to each of the first learning network to the n-th learning network to cause each of the first learning network to the n-th learning network (i) -1) to apply a network operation corresponding to each network to the modulated training data using the first learned parameter to the n-th learned parameter of each of the first learning network to the n-th learning network (i-2) to output each of 1_1 characteristic information to 1_n characteristic information for the modulated training data, (ii) input the training data to each of the first learning network to the n-th learning network, Let each of the first learning network to the n-th learning network perform (ii-1) a network operation corresponding to each network on the training data using each of the first learned parameter to the n-th learned parameter (ii-2) to output the respective 2_1 characteristic information to 2_n characteristic information for the learning data, and in the step (c), the learning apparatus, (i) (i-1) The (1_1)_1th error obtained by referring to the 1_1 characteristic information and the 2_1 characteristic information to the (1_1)_nth error obtained by referring to the 1_n characteristic information and the 2_n characteristic information 1_1 error, (i-2) (i-2a) at least one first task specific output generated using the first_1 characteristic information, and (i-2b) at least one first GT corresponding thereto At least one n-th task generated using the obtained (1_2)_1 error to (i-2a) the 1_n-th characteristic information Minimize the first error calculated by referring to at least a part of the 1_2 error, which is an average of the (1_2)_n-th error obtained by referring to a specific output and (i-2b) at least one n-th GT corresponding thereto, (ii) learning the modulation network so as to maximize the second error calculated by referring to the training data and the modulated training data;

In one embodiment, the learning network comprises a first learning network to an n-th learning network having at least one first learned parameter to at least one n-th learned parameter, wherein when n is an integer greater than or equal to 1, the ( In step a), the learning device inputs the training data to the modulation network to cause the modulation network to modulate the training data to generate first modulated training data, and in step (b), The learning device, (i) input the first modulated training data to the first learning network to cause the first learning network to (i-1) use the first learned parameter of the first learning network By applying a network operation to the first modulated training data (i-2) to output the first_1 characteristic information for the first modulated training data, (ii) the training data to the first learning network to (ii-1) the first learning network to apply a network operation to the training data using the first learned parameter, and (ii-2) output the second_1 characteristic information for the training data. In the step (c), the learning apparatus includes: (i) (i-1) at least one (1_1)_1 error obtained by referring to the first_1 characteristic information and the second_1 characteristic information, (i-2) at least a portion of at least one first task specific output generated using the first_1 characteristic information and at least one (1_2)_1 error obtained by referring to at least one first GT corresponding thereto learning the modulation network so as to minimize at least one first_1 error obtained by referring to (ii) and maximize at least one second_1 error calculated by referring to the training data and the first modulated training data. , such that the modulation network becomes a first learned modulation network, and the learning apparatus increases k, which is an integer, from 2 to n, (i) the input the training data into the k-1 th learned modulation network to cause the k-1 th learned modulation network to modulate the training data to generate k th modulated training data, (ii) (ii- 1) By inputting the k-th modulated training data into the k-th learning network, the k-th learning network uses at least one k-th learned parameter of the k-th learning network for the k-th modulated training data. operation is applied to output 1_k characteristic information for the k-th modulated training data, (ii-2) inputting the training data into the k-th learning network to cause the k-th learning network to The network operation is applied to the training data using the learned parameters to output the second_k characteristic information for the training data, (iii) (iii-1) (iii-1a) the first_k characteristic information and the At least one (1_1)_k error obtained by referring to the 2_k characteristic information, (iii-1b) at least one kth task specific output generated using the 1_k characteristic information, and at least one corresponding thereto At least one (1_2)_k error calculated by referring to at least a part of at least one (1_2)_k error obtained with further reference to the k-th GT is minimized (iii-2) by the training data and the k-th modulated learning The k-1 th learned modulation network is trained to maximize at least one 2_k error calculated with reference to data, so that the k-1 learned modulation network becomes a k-th learned modulation network.

In one embodiment, in the step (c), at least one modulated training data score corresponding to the modulated training data input to a discriminator for determining whether the input data is real or fake (Obfuscated Training) Data Score) as the second error, the learning device, (i) learning the modulation network to minimize the first error and maximize the second error, (ii) the At least one transform learning data score or at least one modulated transform learning data score corresponding to the transformed learning data or modulated transform learning data input to the discriminator is maximized and the modulated learning data score is minimized, and the discriminator is trained. and the transform learning data or the modulated transform learning data is data obtained by transforming the learning data or the modulated learning data, respectively.

In one embodiment, the learning network comprises a first learning network to an n-th learning network having at least one first learned parameter to at least one n-th learned parameter, wherein when n is an integer greater than or equal to 1, the ( In step b), the learning device, (i) input the modulated training data to each of the first learning network to the n-th learning network to cause each of the first learning network to the n-th learning network ( i-1) to apply a network operation corresponding to each network to the modulated training data using the first learned parameter to the n-th learned parameter of each of the first learning network to the n-th learning network to (i-2) output each of 1_1 characteristic information to 1_n characteristic information for the modulated training data, and (ii) input the training data into each of the first learning network to the n-th learning network. to cause each of the first learning network to the n-th learning network to (ii-1) a network corresponding to each network for the training data using each of the first learned parameter to the n-th learned parameter (ii-2) to output the respective 2_1 characteristic information to 2_n characteristic information for the learning data, in the step (c), the learning apparatus, (i) (i-1) ) the (1_1)_1th error obtained with reference to the 1_1 characteristic information and the 2_1 characteristic information to the (1_1)_nth error obtained with reference to the 1_n characteristic information and the 2_n characteristic information 1_1 error, (i-2) (i-2a) at least one first task specific output generated using the 1_1 characteristic information, and (i-2b) at least one first GT corresponding thereto are referenced At least one n-th task generated using the (1_2)_1th error to (i-2a) the 1_n-th characteristic information obtained by Minimizes the first error calculated by referring to at least a portion of the first_2 errors that is an average of the (1_2)_nth errors obtained by referring to a specific output and (i-2b) the nth GT corresponding thereto, learning the modulation network so as to maximize the second error, which is the modulated training data score corresponding to the input modulated training data, (ii) the transform training data input to the discriminator or the modulated transform learning The discriminator is trained so as to maximize the transformed learning data score or the modulated transformed learning data score corresponding to the data and minimize the modulated learning data score.

In one embodiment, the learning network includes a first learning network to an n-th learning network each having a first learned parameter to an n-th learned parameter, and when n is an integer greater than or equal to 1, in step (a), The learning apparatus inputs the learning data to the modulation network to cause the modulation network to modulate the learning data to generate first modulated learning data, and in step (b), the learning apparatus comprises: (i) input the first modulated training data to the first learning network to cause the first learning network to (i-1) use the first learned parameter of the first learning network to By applying a network operation to the training data (i-2) to output the 1_1 characteristic information for the first modulated training data, (ii) to input the training data to the first learning network to the first 1 causes the learning network to (ii-1) apply a network operation to the training data using the first learned parameter, (ii-2) to output the second_1 characteristic information for the training data, and ( In step c), the learning apparatus includes: (i) (i-1) at least one (1_1)_1 error obtained by referring to the first_1 characteristic information and the second_1 characteristic information, and (i-2) (i-2a) at least one first task specific output generated using the first_1 characteristic information and (i-2b) a (1_2)_1th error obtained by referring to at least one first GT corresponding thereto (ii) at least one second_1 that is at least one first modulated training data score corresponding to the first modulated training data input to the discriminator, and minimizes at least one first_1 error obtained with reference to at least a part To maximize the error, the modulation network is trained so that the modulation network becomes a first learned modulation network, and the learning apparatus comprises: (i) the input to the discriminator. Maximizing at least one first transform training data score or at least one first modulated transform training data score corresponding to the transform training data or the modulated transform training data, and (ii) the first modulated training data score learning the discriminator so as to minimize the discriminator so that the discriminator becomes the first learned discriminator, the learning apparatus increases k, which is an integer from 2 to n, (i) learning the k-1th learning data input to the modulated modulation network and cause the k-1th learned modulation network to modulate the training data to generate k-th modulated training data, (ii) (ii-1) the k-th modulated training data is input to the k-th learning network to cause the k-th learning network to apply a network operation to the k-th modulated training data using at least one k-th learned parameter of the k-th learning network, so that the k-th modulation output 1_k characteristic information for the acquired training data, and (ii-2) input the training data to the k-th learning network to cause the k-th learning network to apply the k-th learned parameters to the training data. to apply a network operation to output 2_k characteristic information for the learning data, and (iii) (iii-1) at least one (iii) obtained by referring to the first_k characteristic information and the second_k characteristic information. 1_1)_k error, (iii-2) at least one kth task specific output generated using the 1_k characteristic information, and at least one (1_2)th GT obtained by referring to at least one kth GT corresponding thereto At least one k-th modulated training data corresponding to the k-th modulated training data input to the k-1 learned discriminator while minimizing at least one 1 _ k error calculated by referring to at least some of ) _ k errors By learning the k-1 th learned modulation network to maximize at least one 2_k error that is the score, the k-1 learned edge Let the coarse network become a k-th learned modulation network, and (iv) score at least one k-th transform training data corresponding to the transformed training data input to the k-1 learned discriminator or the modulated transform training data. or at least one of the k-th learned discriminators is trained to maximize the k-th modulated transform training data score and minimize the k-th modulated training data score, so that the k-1 learned discriminator is the k-th learned discriminator. become a learned discriminator.

In one embodiment, the maximum value of the transform learning data score or the modulated transform learning data score corresponding to the transform learning data or the modulated transform learning data input to the discriminator is the transform learning data or the modulated transform learning data, respectively. 1, which determines that the transformed training data is genuine, and the minimum value of the modulated training data score corresponding to the modulated training data input to the discriminator is 0, which determines that the modulated training data is fake.

In an embodiment, in the step (c), the learning apparatus comprises: (i) a difference between the first characteristic information and the second characteristic information, and (ii) at least one between the task specific output and the GT. The first error is obtained by referring to at least a part of the difference between , and the second error is obtained by at least one difference between the training data and the modulated training data.

In an embodiment, the learning apparatus obtains the first error with reference to a norm or cosine similarity between the first characteristic information and the second characteristic information.

In one embodiment, in the step (c), the learning apparatus measures at least one quality (Quality) with reference to at least a part of entropy and noise level of the modulated learning data, and the measured quality with reference to obtain the first error.

According to another aspect of the present invention, in a test method for learning a modulation network (Obfuscation Network) for concealing original data for personal information protection, (a) a learning device, (i) acquires learning data and a process of inputting into a modulation network to cause the modulation network to modulate the training data to generate modulated training data, (ii) inputting the modulated training data into the learning network having at least one learned parameter; causing the learning network to (ii-1) apply a network operation to the modulated training data using the learned parameter (ii-2) to generate first characteristic information corresponding to the modulated training data; (iii) input the training data to the learning network to cause the learning network to (iii-1) apply a network operation to the training data using the learned parameters (iii-2) (iv) (iv-1) (iv-1a) at least one first_1 error obtained by referring to the first characteristic information and the second characteristic information and (iv-1b) at least a portion of at least one first_2 error obtained by referring to at least one task specific output generated using the first characteristic information and at least one GT (Ground Truth) corresponding thereto a process of learning the modulation network to minimize at least one first error calculated with reference to (iv-2) and maximize at least one second error calculated by referring to the training data and the modulated training data; in the performed state, performing, by the test device, a process of acquiring test data; and (b) the test device inputs the test data to the modulation network and causes the modulation network to modulate the test data using the learned parameters of the modulation network, thereby concealing the modulated test data. Output as test data.

In an embodiment, in the step (a), the learning network comprises a first learning network to an n-th learning network having at least one first learned parameter to at least one n-th learned parameter, wherein n is When it is an integer greater than or equal to 1, the learning device (i) input the modulated training data into each of the first learning network to the n-th learning network to cause each of the first learning network to the n-th learning network ( i-1) to apply a network operation corresponding to each network to the modulated training data using the first learned parameter to the n-th learned parameter of each of the first learning network to the n-th learning network to (i-2) a process of outputting each of 1_1 characteristic information to 1_n characteristic information for the modulated training data, (ii) the training data to each of the first learning network to the n-th learning network input to cause each of the first learning network to the n-th learning network to (ii-1) use the first learned parameter to the n-th learned parameter of each of the first learning network to the n-th learning network (ii-2) a process of outputting each of 2_1 characteristic information to 2_n characteristic information for the training data by applying a network operation corresponding to each network to the training data, (iii) (iii-) 1) (iii-1a) (1_1)_1th error obtained by referring to the 1_1 characteristic information and the 2_1 characteristic information to (1_1)th obtained by referring to the 1_n characteristic information and the 2_n characteristic information The first_1 error that is the average of _n errors, (iii-1b) the first task specific output generated using the first_1 characteristic information, and the at least one first GT corresponding thereto (1_2)_1 error to at least generated using the first_n characteristic information Minimizing the first error calculated by referring to at least a portion of the 1_2 errors, which is an average of (1_2)_n-th errors obtained by referring to one n-th task specific output and at least one n-th GT corresponding thereto, (iii-2) a process of learning the modulation network is performed to maximize the second error calculated with reference to the training data and the modulated training data;

In one embodiment, in the step (a), the learning device, (i) at least one modulated learning data corresponding to the modulated learning data input to a discriminator that determines whether the input data is real or fake obtain a score with the second error, (ii) train the modulation network to minimize the first error and maximize the second error, and (iii) transform training data or modulation inputted to the discriminator. At least one transform learning data score or at least one modulated transform learning data score corresponding to the transformed learning data is maximized and the discriminator is trained to minimize the modulated learning data score, the transform learning data or the The modulated transform learning data is data generated by transforming the learning data or the modulated learning data, respectively.

In an embodiment, in the step (a), the learning network comprises a first learning network to an n-th learning network having at least one first learned parameter to at least one n-th learned parameter, wherein n is When it is an integer greater than or equal to 1, the learning device (i) input the modulated training data into each of the first learning network to the n-th learning network to cause each of the first learning network to the n-th learning network ( i-1) to apply a network operation corresponding to each network to the modulated training data using the first learned parameter to the n-th learned parameter of each of the first learning network to the n-th learning network to (i-2) a process of outputting each of 1_1 characteristic information to 1_n characteristic information for the modulated training data, (ii) the training data to each of the first learning network to the n-th learning network input to cause each of the first learning network to the n-th learning network to (ii-1) use the first learned parameter to the n-th learned parameter of each of the first learning network to the n-th learning network (ii-2) a process of outputting each of 2_1 characteristic information to 2_n characteristic information for the training data by applying a network operation corresponding to each network to the training data, (iii) (iii-) 1) (iii-1a) (1_1)_1st error obtained by referring to the first_1 characteristic information and the 2_1 characteristic information to (1_1)th obtained by referring to the 1_n characteristic information and the 2_n characteristic information The first_1 error that is the average of _n errors, (iii-1b) the first task specific output generated using the first_1 characteristic information, and the at least one first GT corresponding thereto (1_2)_1 error to at least generated using the first_n characteristic information Minimizing the first error calculated by referring to at least a portion of the 1_2 errors, which is an average of (1_2)_n-th errors obtained by referring to one n-th task specific output and at least one n-th GT corresponding thereto, (iii-2) a process of learning the modulation network so as to maximize the second error, which is the transform training data score corresponding to the modulated training data input to the discriminator; and (iv) the transform input to the discriminator. The process of learning the discriminator is performed so as to maximize the transform learning data score or the modulated transform learning data score corresponding to the training data or the modulated transform training data, respectively, and to minimize the transform training data score.

According to another aspect of the present invention, there is provided a learning apparatus for learning a modulation network (Obfuscation Network) for concealing original data for personal information protection, comprising: at least one memory for storing instructions; and (I) when the training data is obtained, inputting the training data into a modulation network to cause the modulation network to modulate the training data to generate modulated training data; (II) (i) input the modulated training data into a learning network having at least one learned parameter to cause the learning network to (i-1) use the learned parameter to perform a network operation on the modulated training data operation is applied to (i-2) generate first characteristic information corresponding to the modulated training data, and (ii) input the training data into the learning network to cause the learning network to ( ii-1) a process of applying a network operation to the training data using the learned parameter (ii-2) generating second characteristic information corresponding to the training data; and (III) (i) (i-1) at least one first_1 error obtained with reference to the first characteristic information and the second characteristic information, and (i-2) generated using the first characteristic information At least one first error calculated by referring to at least a portion of the at least one first_2 error obtained by referring to the task specific output and at least one GT (Ground Truth) corresponding thereto is minimized ( ii) a process of learning the modulation network to maximize at least one second error calculated with reference to the training data and the modulated training data; at least one processor;

In one embodiment, the learning network comprises a first learning network to an n-th learning network having at least one first learned parameter to at least one n-th learned parameter, wherein when n is an integer greater than or equal to 1, the ( II) in the process, the processor, (i) input the modulated training data to each of the first learning network to the n-th learning network to cause each of the first learning network to the n-th learning network (i- 1) to apply a network operation corresponding to each network to the modulated training data using the first learned parameter to the n-th learned parameter of each of the first learning network and the n-th learning network ( i-2) to output each of 1_1 characteristic information to 1_n characteristic information for the modulated training data, (ii) input the training data to each of the first learning network to the n-th learning network, respectively (ii-1) to cause the first learning network to the n-th learning network of (ii-1) to perform a network operation corresponding to each network on the training data using each of the first learned parameter to the n-th learned parameter (ii-2) to output the respective 2_1 characteristic information to 2_n characteristic information for the learning data, and in the (III) process, the processor, (i) (i-1) the first The 1_1 error that is the average of the (1_1)_1th error obtained by referring to the 1_1 characteristic information and the 2_1 characteristic information to the (1_1)_nth error obtained by referring to the 1_n characteristic information and the 2_n characteristic information and (i-2) at least one first task specific output generated using the first_1 characteristic information and at least one (1_2)_1th error to 1_nth error obtained by referring to at least one first GT corresponding thereto At least one n-th task specific output generated using characteristic information and a corresponding enemy Minimizes the first error calculated by referring to at least a part of the first_2 errors, which is an average of (1_2)_n errors obtained with reference to one nth GT, and (ii) the learning data and the modulated learning The modulation network is trained to maximize the second error calculated with reference to data.

In one embodiment, the learning network comprises a first learning network to an n-th learning network having at least one first learned parameter to at least one n-th learned parameter, wherein when n is an integer greater than or equal to 1, the ( In process I), the processor inputs the training data into the modulation network to cause the modulation network to modulate the training data to generate first modulated training data, and in the process (II), the The processor is configured to: (i) input the first modulated training data into the first learning network to cause the first learning network to (i-1) use the first learned parameter of the first learning network to 1 By applying a network operation to the modulated training data, (i-2) to output 1_1 characteristic information for the first modulated training data, (ii) input the training data to the first learning network to cause the first learning network to (ii-1) apply a network operation to the training data using the first learned parameter, and (ii-2) output 2_1 characteristic information for the training data, , in the process (III), the processor includes: (i) (i-1) at least one (1_1)_1 error obtained by referring to the first_1 characteristic information and the second_1 characteristic information, and (i- 2) at least one of the at least one first task specific output generated using the first_1 characteristic information and the at least one (1_2)_1 error obtained by referring to the at least one first GT corresponding thereto learning the modulation network so as to minimize the obtained at least one first_1 error and (ii) maximize the at least one second_1 error calculated with reference to the training data and the first modulated training data; Let the network be the first learned modulation network, and the processor increases the integer k from 2 to n, ( i) input the training data into the k-1 th learned modulation network to cause the k-1 th learned modulation network to modulate the training data to generate k th modulated training data, (ii) (ii-1) input the k-th modulated training data to the k-th learning network to cause the k-th learning network to use at least one k-th learned parameter of the k-th learning network and the k-th modulated training data to apply a network operation to output 1_k characteristic information for the k-th modulated training data, and (ii-2) input the training data into the k-th learning network to cause the k-th learning network to (iii) (iii-1) (iii-1a) the first_k characteristic by applying a network operation to the training data using the k-th learned parameter to output 2_k characteristic information for the training data; At least one (1_1)_k error obtained by referring to the information and the second_k characteristic information, (iii-1b) at least one k-th task specific output generated using the 1_k characteristic information, and a corresponding At least one 1_k error calculated by referring to at least a part of at least one (1_2)_k error obtained by further referring to at least one kth GT is minimized, and (iii-2) the training data and the k-th error are minimized. The k-1 learned modulation network is trained to maximize at least one 2_k error calculated with reference to the modulated training data, so that the k-1 learned modulation network becomes the k-th learned modulation network. .

In one embodiment, in the (III) process, at least one modulated training data score corresponding to the modulated training data input to a discriminator for determining whether the input data is real or fake (Obfuscated Training) Data Score) as the second error, the processor, (i) learning the modulation network to minimize the first error and maximize the second error, (ii) the determination At least one transform learning data score or at least one modulated transform learning data score corresponding to the transformed learning data or modulated transform learning data input to the unit is maximized and the discriminator is trained to minimize the modulated learning data score , The transform learning data or the modulated transform learning data is data obtained by converting the learning data or the modulated learning data, respectively.

In one embodiment, the learning network comprises a first learning network to an n-th learning network having at least one first learned parameter to at least one n-th learned parameter, wherein when n is an integer greater than or equal to 1, the ( II) in the process, the processor is configured to: (i) input the modulated training data to each of the first learning network to the n-th learning network to cause each of the first learning network to the n-th learning network to (i) -1) to apply a network operation corresponding to each network to the modulated training data using the first learned parameter to the n-th learned parameter of each of the first learning network to the n-th learning network (i-2) to output each of 1_1 characteristic information to 1_n characteristic information for the modulated training data, (ii) input the training data to each of the first learning network to the n-th learning network, Let each of the first learning network to the n-th learning network perform (ii-1) a network operation corresponding to each network on the training data using each of the first learned parameter to the n-th learned parameter (ii-2) to output each of 2_1 characteristic information to 2_n characteristic information for the training data, and in the (III) process, the processor, (i) (i-1) The 1_1 that is the average of the (1_1)_1th error obtained with reference to the 1_1 characteristic information and the 2_1 characteristic information to the (1_1)_nth error obtained by referring to the 1_n characteristic information and the 2_n characteristic information error, (i-2) at least one first task specific output generated using the first_1 characteristic information, and (1_2)_1th error to the first (1_2)_1st error obtained by referring to at least one first GT corresponding thereto At least one n-th task specific output generated using the 1_n characteristic information and the corresponding The modulation corresponding to the modulated learning data input to the discriminator and minimizing the first error calculated by referring to at least a part of the first_2 errors that are the average of the (1_2)_n errors obtained with reference to n GT The modulation network is trained to maximize the second error, which is the obtained training data score, and (ii) the transformed training data score or the modulation corresponding to the transformed training data input to the discriminator or the modulated transformed training data. The discriminator is trained to maximize the transformed training data score and minimize the modulated training data score.

In one embodiment, the learning network includes a first learning network to an n-th learning network each having a first learned parameter to an n-th learned parameter, and when n is an integer greater than or equal to 1, in the process (I), The processor inputs the training data to the modulation network to cause the modulation network to modulate the training data to generate first modulated training data, wherein in the process (II), the processor comprises: (i) ) input the first modulated training data into the first learning network to cause the first learning network to (i-1) use the first learned parameter of the first learning network to the first modulated training data (i-2) to output the first_1 characteristic information for the first modulated training data by applying a network operation to (ii) input the training data to the first learning network to the first learning (ii-1) causing the network to apply a network operation to the training data using the first learned parameter (ii-2) to output the second_1 characteristic information for the training data, and (III) In the process, the processor includes: (i) (i-1) at least one (1_1)_1 error obtained by referring to the first_1 characteristic information and the second_1 characteristic information, and (i-2) (i- 2a) at least one first task specific output generated using the first_1 characteristic information and (i-2b) at least a portion of the (1_2)_1th error obtained by referring to at least one first GT corresponding thereto At least one first_1 error obtained by reference is minimized and (ii) at least one second_1 error that is at least one first modulated training data score corresponding to the first modulated training data input to the discriminator is maximized. to learn the modulation network so that the modulation network becomes a first learned modulation network, and the processor, (i) to the discriminator Maximizing at least one first transform learning data score or at least one first modulated transform learning data score corresponding to the input transform learning data or the modulated transform learning data, and (ii) the first modulated learning data To minimize the score, the discriminator is trained so that the discriminator becomes the first learned discriminator, and the processor increases k, which is an integer, from 2 to n, (i) assigns the training data to the k-1th Input to the learned modulation network to cause the k-1 learned modulation network to modulate the training data to generate k-th modulated training data, (ii) (ii-1) the k-th modulated learning Input data to the k-th learning network to cause the k-th learning network to apply a network operation to the k-th modulated training data using at least one k-th learned parameter of the k-th learning network, so that the k-th 1_k characteristic information for the modulated training data is output, and (ii-2) the training data is input to the k-th learning network to cause the k-th learning network to use the k-th learned parameter to learn the training data. to output 2_k characteristic information for the training data by applying a network operation to (1_1)_k error, (iii-2) at least one kth task specific output generated using the 1_k characteristic information, and at least one (kth) GT obtained by referring to at least one kth GT corresponding thereto 1_2) At least one k-th modulated learning corresponding to the k-th modulated learning data input to the k-1 learned discriminator while minimizing at least one 1 _ k error calculated by referring to at least a part of 1_2) _ k errors By learning the k-1 th learned modulation network to maximize at least one 2_k error that is the data score, Let the wet modulation network become a k-th learned modulation network, and (iv) at least one k-th transform training data corresponding to the transform training data input to the k-1 learned discriminator or the modulated transform training data. The k-1 th learned discriminator is trained to maximize the score or at least one k th modulated transform training data score and minimize the k th modulated training data score, so that the k-1 th learned discriminator is Let k be a learned discriminator.

In one embodiment, the maximum value of the transform learning data score or the modulated transform learning data score corresponding to the transform learning data or the modulated transform learning data input to the discriminator is the transform learning data or the modulated transform learning data, respectively. 1, which determines that the transformed training data is genuine, and the minimum value of the modulated training data score corresponding to the modulated training data input to the discriminator is 0, which determines that the modulated training data is fake.

In an embodiment, in process (III), the processor is configured to: (i) a difference between the first characteristic information and the second characteristic information; The first error is obtained by referring to at least some of the differences, and the second error is obtained by at least one difference between the training data and the modulated training data.

In an embodiment, the processor obtains the first error with reference to a norm or cosine similarity between the first characteristic information and the second characteristic information.

In one embodiment, in the process (III), the processor measures at least one quality (Quality) with reference to at least a part of entropy and noise degree of the modulated training data, and refers to the measured quality to obtain the first error.

According to another aspect of the present invention, there is provided a test apparatus for learning a modulation network (Obfuscation Network) for concealing original data for personal information protection, comprising: at least one memory for storing instructions; and (I) a process in which a learning device (i) obtains training data and inputs it to a modulation network to cause the modulation network to modulate the training data to generate modulated training data, (ii) the modulated training data input training data into the learning network having at least one learned parameter to cause the learning network to (ii-1) apply a network operation to the modulated training data using the learned parameter (ii- 2) a process of generating first characteristic information corresponding to the modulated training data, (iii) inputting the training data into the learning network to cause the learning network to (iii-1) use the learned parameters (iii-2) a process for generating second characteristic information corresponding to the training data by applying a network operation to the training data; and (iv) (iv-1) (iv-1a) the first characteristic information and at least one first_1 error obtained with reference to the second characteristic information, (iv-1b) at least one task specific output generated using the first characteristic information, and at least one GT (Ground) corresponding thereto Truth) and minimizes at least one first error calculated by referring to at least some of the at least one first_2 error obtained by referring to (iv-2) the at least one calculated by referring to the learning data and the modulated learning data a process of performing a process of acquiring test data while performing the process of learning the modulation network so as to maximize one second error; and (II) inputting the test data into the modulation network to cause the modulation network to modulate the test data using the learned parameters of the modulation network, and output the modulated test data as concealed test data. and at least one processor configured to execute the instructions for performing a process or cause another device to execute.

In an embodiment, in the process (I), the learning network comprises a first learning network to an n-th learning network having at least one first learned parameter to at least one n-th learned parameter, wherein n is When it is an integer greater than or equal to 1, the learning device, (i) input the modulated training data into each of the first learning network to the n-th learning network to cause each of the first learning network to the n-th learning network ( i-1) to apply a network operation corresponding to each network to the modulated training data using the first learned parameter to the n-th learned parameter of each of the first learning network to the n-th learning network to (i-2) a process of outputting each of the first_1 characteristic information to the 1_n first characteristic information for the modulated training data, (ii) the training data to each of the first learning network to the n-th learning network input to cause each of the first learning network to the n-th learning network to (ii-1) use the first learned parameter to the n-th learned parameter of each of the first learning network to the n-th learning network (ii-2) a process of outputting each of 2_1 characteristic information to 2_n characteristic information for the training data by applying a network operation corresponding to each network to the training data; (iii) (iii-) 1) (iii-1a) (1_1)_1th error obtained by referring to the 1_1 characteristic information and the 2_1 characteristic information to (1_1)th obtained by referring to the 1_n characteristic information and the 2_n characteristic information The first_1 error, which is an average of _n errors, (iii-1b) at least one first task specific output generated using the first_1 characteristic information, and at least one first GT corresponding thereto (1_2)_1 error to the enemy generated using the 1_n characteristic information The first error calculated by referring to at least a portion of the first_2 errors, which is an average of (1_2)_n-th errors obtained by referring to one n-th task specific output and at least one n-th GT corresponding thereto, is minimized and (iii-2) a process of learning the modulation network to maximize the second error calculated by referring to the training data and the modulated training data.

In one embodiment, in the (I) process, the learning device (i) at least one modulated learning data corresponding to the modulated learning data input to a discriminator that determines whether the input data is real or fake obtain a score with the second error, (ii) train the modulation network to minimize the first error and maximize the second error, and (iii) transform training data or modulation inputted to the discriminator. At least one transform learning data score or at least one modulated transform learning data score corresponding to each of the transformed learning data is maximized and the discriminator is trained to minimize the modulated learning data score, the transformed learning data or the The modulated transform learning data is data generated by transforming the learning data or the modulated learning data, respectively.

In an embodiment, in the process (I), the learning network comprises a first learning network to an n-th learning network having at least one first learned parameter to at least one n-th learned parameter, wherein n is When it is an integer greater than or equal to 1, the learning device, (i) input the modulated training data into each of the first learning network to the n-th learning network to cause each of the first learning network to the n-th learning network ( i-1) to apply a network operation corresponding to each network to the modulated training data using the first learned parameter to the n-th learned parameter of each of the first learning network to the n-th learning network to (i-2) a process of outputting each of the first_1 characteristic information to the 1_n first characteristic information for the modulated training data, (ii) the training data to each of the first learning network to the n-th learning network input to cause each of the first learning network to the n-th learning network to (ii-1) use the first learned parameter to the n-th learned parameter of each of the first learning network to the n-th learning network (ii-2) a process of outputting each of 2_1 characteristic information to 2_n characteristic information for the training data by applying a network operation corresponding to each network to the training data; (iii) (iii-) 1) (iii-1a) (1_1)_1th error obtained with reference to the 1_1 characteristic information and the 2_1 characteristic information to (1_1)th obtained by referring to the 1_n characteristic information and the 2_n characteristic information The first_1 error, which is an average of _n errors, (iii-1b) at least one first task specific output generated using the first_1 characteristic information, and at least one first GT corresponding thereto (1_2)_1 error to the enemy generated using the 1_n characteristic information The first error calculated by referring to at least a portion of the first_2 errors, which is an average of (1_2)_n-th errors obtained by referring to one n-th task specific output and at least one n-th GT corresponding thereto, is minimized and (iii-2) a process of learning the modulation network so as to maximize the second error that is the transformed learning data score corresponding to the modulated training data input to the discriminator; and (iv) the input to the discriminator. The process of learning the discriminator is performed so as to maximize the transform training data score or the modulated transform training data score corresponding to the transform training data or the modulated transform training data, respectively, and to minimize the transform training data score.

In addition to this, a computer-readable recording medium for recording a computer program for executing the method of the present invention is further provided.

According to the present invention, since the task of finding personal identification information in data is not required, there is an effect that can perform a simple and accurate concealing operation.

In addition, the present invention is irreversibly tampered with from the original data (Obfuscation) to generate concealed data, there is another effect that can protect the security and privacy of the data.

In addition, the present invention has another effect of being able to generate data that looks different to humans, but looks the same or similar to each other to a computer.

In addition, the present invention has another effect that can further activate the big data trading market.

The accompanying drawings for use in the description of the embodiments of the present invention are only a part of the embodiments of the present invention, and those of ordinary skill in the art to which the present invention pertains (hereinafter referred to as "those of ordinary skill in the art") Other drawings may be obtained based on these drawings without any work being done.
1 schematically shows a learning apparatus for learning a modulation network that processes original data by concealing according to an embodiment of the present invention;
2 schematically shows a learning method for learning a modulation network that conceals original data according to an embodiment of the present invention;
3 schematically shows another learning method for learning a modulation network that conceals original data according to an embodiment of the present invention;
4 schematically shows a learning method for learning a modulation network for concealing original data according to another embodiment of the present invention;
5 schematically shows another learning method for learning a modulation network that conceals original data according to another embodiment of the present invention;
6 schematically shows a test apparatus for testing a learned modulation network according to an embodiment of the present invention;
7 schematically illustrates a test method for testing a learned modulation network according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0023] Reference is made to the accompanying drawings, which show by way of illustration specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention are different but need not be mutually exclusive. For example, certain functions, structures, and characteristics described herein may be implemented with changes from one embodiment to another without departing from the spirit and scope of the present invention. In addition, it should be understood that the location or arrangement of individual components within each embodiment may be changed without departing from the spirit and scope of the present invention. Accordingly, the detailed description given below is not intended to be taken in a limiting sense, and the scope of the present invention should be taken to cover the scope of the claims and all equivalents thereto. Like reference numerals in the drawings refer to the same or similar elements throughout the various aspects.

Hereinafter, the term “comprise” and various expressions of the term included in the detailed description and claims of the present invention do not mean excluding other technical features, additions, elements or steps. Other objects, advantages and features of the present invention will become apparent to those skilled in the art, in part from the specification and in part from the practice of the present invention. The following examples and drawings are shown by way of illustration, but the present invention is not limited thereto.

In order to enable those of ordinary skill in the art to easily practice the present invention, various preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 schematically illustrates a learning apparatus for learning a modulation network for concealing (eg, de-identifying) original data according to an embodiment of the present invention.

Referring to FIG. 1 , the learning apparatus 100 according to an embodiment of the present invention learns to input and produce the same or similar result as the result calculated by the learning network 100 inputting the learning data by inputting the modulated learning data It may include a memory 110 storing instructions for learning a modulation network for modulating data, and a processor 120 performing a process of learning a modulation network according to the instructions stored in the memory 110 .

Specifically, the learning device 100 includes at least one computing device (eg, a computer processor, memory, storage, input device, output device, or other component of an existing computing device; an electronic communication device such as a router, switch, etc.; a network attachment). desired system performance using a combination of storage (NAS) and electronic information storage systems such as storage area networks (SANs)) and at least one computer software (i.e., instructions that cause a computing device to function in a particular way). may be achieved with

In addition, the processor of the computing device may include a hardware configuration such as a micro processing unit (MPU) or a central processing unit (CPU), a cache memory, and a data bus. In addition, the computing device may further include an operating system and a software configuration of an application for performing a specific purpose.

However, this does not exclude an integrated device comprising any combination of processor, memory, medium, or other computing element for the computing device to practice the present invention.

On the other hand, when the learning data is acquired, the processor 120 of the learning apparatus 100 according to the instructions stored in the memory 110 inputs the learning data to a modulation network (Obfuscation Network) so that the modulation network modulates the learning data. to generate modulated learning data. Then, the learning apparatus 100 (i) inputs the modulated learning data to a learning network having at least one learned parameter to cause the learning network to (i-1) learn data modulated using the learned parameter. By applying a network operation (i-2) to generate first characteristic information (Characteristic Information) corresponding to the modulated training data, (ii) input the training data to the learning network to cause the learning network to (ii-1) ) to apply a network operation to the training data using the learned parameters (ii-2) to generate the second characteristic information corresponding to the training data. Thereafter, the learning apparatus 100 uses (i) (i-1) at least one first_1 error obtained with reference to the first characteristic information and the second characteristic information, and (i-2) the first characteristic information At least one first error calculated by referring to at least a part of at least one task specific output generated by the above and at least one first_2 error obtained by referring to at least one GT corresponding thereto is minimized, (ii) the modulation network may be trained to maximize the at least one second error calculated by referring to the training data and the modulated training data.

In addition, in a state in which the at least one modulated learning data score corresponding to the modulated learning data input to the discriminator for determining whether the input data is real or fake is obtained with a second error, the learning apparatus 100 (i ) In addition to learning the modulation network to minimize the first error and maximize the second error, (ii) at least one transform training data score corresponding to the transform training data input to the discriminator or the modulated transform training data, or at least The discriminator can be trained to maximize the score of one modulated transform training data and minimize the score of the modulated training data.

2 to 5 show a method of learning a modulation network for concealing (eg, de-identifying) original data for personal information protection using the learning apparatus 100 according to an embodiment of the present invention configured as described above. It is explained with reference to the following.

2 schematically illustrates a method for learning a modulation network for concealing original data according to an embodiment of the present invention.

First, when the learning data (x) is obtained, the learning apparatus 100 inputs the learning data (x) to the modulation network (O) to cause the modulation network (O) to modulate the learning data (x). The training data (x', that is, O(x)) is generated.

In this case, the training data (x) may be original training data, which is original data for use in learning, or transformed training data generated by transforming the original training data, and the transform training data is a random noise generation network in the original training data. It may be generated by adding at least one random noise generated through (not shown). As an example, the random noise generation network may generate random noise having a normal distribution N(0, σ), and may generate transform training data by adding the generated noise to the original training data. In addition, the transformation training data may be generated by blur processing the original training data or changing the resolution of the original training data in addition to random noise, but is not limited thereto, and various methods for transforming the original training data may be applied. .

In addition, the modulated learning data (x') is recognized differently from the learning data (x) by humans, but may be recognized as the same as or similar to the learning data (x) in the learning network.

On the other hand, the modulation network O is, for example, an encoder including at least one convolution layer by applying a convolution operation to an image with input training data x at least once, and an encoder output from the encoder. and a decoder including at least one deconvolution layer that outputs modulated training data (x') by applying a deconvolution operation to at least one feature map at least once, but is not limited thereto. , may include a learning network of various structures that modulates the input learning data.

Next, the learning apparatus 100 (i) inputs the modulated learning data (x') to the learning network (F) having at least one learned parameter to cause the learning network (F) to learn (i-1) (i-2) First characteristic information (F(x')) corresponding to the modulated training data (x') is generated by applying a network operation to the modulated training data (x') using the parameter (ii) input the training data (x) to the learning network (F) to cause the learning network (F) to (ii-1) apply a network operation to the training data (x) using the learned parameters Thus, (ii-2) the second characteristic information (F(x)) corresponding to the learning data (x) can be generated.

In this case, the learning network (F) may include a machine learning network, but is not limited thereto, and applies a network operation to the training data (x') modulated using the learned parameters to apply the first characteristic information (F(x) ')), and may include all learning networks that apply a network operation to the training data (x) to output the second characteristic information (F(x)). In addition, machine learning networks include k-nearest neighbors, linear regression, logistic regression, support vector machine (SVM), and decision trees. ) and Random Forest, Neural Network, Clustering, Visualization and Dimensionality Reduction, Association Rule Learning, Deep Belief Network, It may include at least one of reinforcement learning and deep learning algorithms, but the machine learning network is not limited thereto and may include various learning algorithms. In addition, the target for concealing processing (eg, non-identifying target) may be personal information included in the original data (x), in this case, the personal information is personally identifiable information, personal medical information, biometric information, behavioral information It may include any information related to the individual

And, the first characteristic information (F(x')) and the second characteristic information (F(x)) are the modulated learning data (x') and the feature or logit (Feature) corresponding to the learning data (x), respectively. Logit). In addition, the first characteristic information (F(x')) and the second characteristic information (F(x)) are the modulated training data (x') and the characteristic value associated with a predetermined characteristic in the training data (x), respectively, or It may be a logit including a value related to at least one of a vector, a matrix, and a coordinate related to a predetermined feature. For example, when the training data (x) is face image data, the above result is a class for face recognition, a facial feature, for example, a smiling expression, and a landmark point (Landmark Point, for example, both end points of the eyes). ) may be a coordinate with respect to

Next, the learning apparatus 100 (i) (i-1) at least one first_1 error obtained by referring to the first characteristic information F(x') and the second characteristic information F(x)) and (i-2) at least one task specific output generated using the first characteristic information F(x') and at least one first_2 error obtained by referring to at least one GT corresponding thereto In order to minimize at least one first error calculated with reference to a part and (ii) maximize at least one second error calculated with reference to the training data (x) and the modulated training data (x'), the modulation network ( O) can be learned. That is, in learning the modulation network O, the learning apparatus 100 uses the second error to output the modulated training data (x') with a large difference from the training data (x) to the modulation network (O). And, the learning data (x') modulated by modulating the training data (x) so that the learning data (x') is recognized as the same or similar to the training data (x) by the learning network (F) using the first error (x') ) to output.

In this case, the learning apparatus 100 determines (i) a difference between the first characteristic information F(x') and the second characteristic information F(x)), and (ii) a task specific output and a corresponding GT The first error may be obtained by referring to at least a part of at least one difference of . As an example, the learning apparatus 100 refers to a norm or a cosine similarity between the first specific information F(x') and the second characteristic information F(x) for the first error. may be obtained, but is not limited thereto, and various algorithms capable of calculating a difference between the first characteristic information F(x') and the second characteristic information F(x) may be applied. Also, the learning apparatus 100 may obtain the second error by referring to at least one difference between the learning data x and the modulated learning data x'.

In addition, the learning apparatus 100 measures at least one quality with reference to at least a part of the entropy and the noise level of the modulated learning data x', and further refers to the measured quality to determine the first error. can be obtained That is, the learning apparatus 100 can learn the modulation network O to minimize the quality of the modulated learning data x', for example, to maximize the entropy, noise, etc. of the modulated learning data x'. have.

And, when learning the modulation network O so that the learning device 100 minimizes the first error and maximizes the second error, the learning device 100 fixes the learned parameters of the learning network F It is not updated, and learning can be performed only for the modulation network (O).

On the other hand, the task specific output is an output for the task that the learning network F wants to perform, and the learned task of the learning network F, such as a probability for a class for class classification, and a coordinate value due to regression for position detection, etc. A task according to the task to be performed in the learning network (F) by applying the activation function of the activation unit to the characteristic information output from the learning network (F) It can produce specific output. In this case, the activation function includes a sigmoid function, a linear function, a softmax function, a ReLU(Rlinear) function, a square function, a SQRT function, a Srlinear function, an ABS function, and a TANH function. , a Brlinear function, etc. may be used, but is not limited thereto.

For example, when the learning network (F) performs a task for class classification, the learning apparatus 100 maps the first characteristic information output from the learning network (F) for each class by class for the modulated learning data You can create probabilities.

In this case, the probability for each class may represent a probability that the first characteristic information F(x') output from the learning network F for each set class is the correct answer. For example, when the training data is face image data, the probability that the face is in a smiling state is 0.75, the probability that the face is not in a smiling state is 0.25, and the like may be output. In this case, the softmax algorithm may be used to map the first characteristic information F(x') output from the learning network F for each class, but is not limited thereto, and the first characteristic information F(x') )) for each class, various algorithms may be used.

3 schematically shows another method of learning a modulation network for concealing original data according to an embodiment of the present invention, that is, the learning network F in FIG. It is composed of the learning networks (F1, F2, ..., Fn) of At this time, each of the plurality of learning networks F1, F2, ..., Fn may be in a learned state to perform at least some different tasks. In the following description, a detailed description of parts that can be easily understood from the description of FIG. 2 will be omitted.

First, when the learning data (x) is obtained, the learning apparatus 100 inputs the learning data (x) to the modulation network (O) to cause the modulation network (O) to modulate the learning data (x) to learn modulated learning Let's create data (x', that is, O(x)).

At this time, the learning data (x) may be original learning data, which is original data for use in learning, or transformation learning data (x') generated by transforming the original learning data, and the transformation learning data (x') is for a person Although it is recognized differently from the training data (x), it may be recognized as the same or similar to the training data (x) in the learning network.

Next, the learning device 100 inputs the modulated learning data (x ') to each of the first learning network (F1) to the n-th learning network (Fn) to each of the first learning network (F1) to the n-th learning Let the network Fn (i) the first learning network F1 to the n-th learning network Fn, respectively, with respect to the modulated training data (x') using the first learned parameter to the n-th learned parameter (ii) 1_1 characteristic information (F1(x')) to 1_n characteristic information (Fn(x')) corresponding to the modulated learning data (x') by applying a network operation corresponding to each network to create each. In addition, the learning device 100 inputs the training data (x) to each of the first learning network (F1) to the n-th learning network (Fn) to each of the first learning network (F1) to the n-th learning network (Fn) (i) a first learning network (F1) to a network corresponding to each network for the training data (x) using the first learned parameter to the n-th learned parameter of each of the learning networks (Fn) By applying the operation (ii), each of 2_1 characteristic information F1(x) to 2_n characteristic information Fn(x) corresponding to the learning data x may be generated.

Next, the learning apparatus 100 (i) (i-1) the (1_1)_1th error obtained by referring to the first_1 characteristic information F1(x') and the second_1 characteristic information F1(x)) 1_1 error that is an average of (1_1)_n-th errors obtained by referring to the 1_n-th characteristic information Fn(x') and the 2_n-th characteristic information Fn(x)), and (i-2) the first_1 characteristic At least one first task specific output generated using the information F1(x') and the (1_2)_1th error obtained by referring to at least one first GT corresponding thereto to the 1st_n characteristic information are used A first error calculated by referring to at least a portion of the first_2 errors, which is an average of at least one (1_2)_n-th error obtained by referring to at least one n-th task specific output generated by It is possible to learn the modulation network (O) so as to minimize (ii) the second error calculated by referring to the training data (x) and the modulated training data (x').

That is, the learning apparatus 100 obtains the (1_1)_1th error calculated with reference to the first_1 characteristic information F1(x') and the second_1 characteristic information F1(x), and the first_2 characteristic information ( A (1_1)_2th error calculated by referring to F2(x')) and 2_2 characteristic information (F2(x)) is obtained, and similarly, 1_n characteristic information (Fn(x')) and 2_n characteristic information ( A (1_1)_nth error calculated by referring to Fn(x)) is calculated to obtain a first_1st error that is an average of the obtained (1_1)_1th error to the obtained (1_1)_nth error. In addition, the learning apparatus 100 determines the first task specific output generated using the first_1 characteristic information F1(x') and the (1_2)_1th error to the 1_nth error calculated by referring to the first GT corresponding thereto. The (1_2)_nth error calculated by referring to the nth task specific output generated using the characteristic information Fn(x') and the nth GT corresponding thereto is obtained, and the calculated (1_2)_1th error to The first_2th error, which is the average of the (1_2)_nth errors, is obtained. In addition, the learning apparatus 100 may learn the modulation network O to minimize the calculated first error and maximize the second error by referring to at least some of the first_1 error and the first_2 error. That is, the first error may be any one of a 1_1 error, a 1_2 error, and a sum of the 1_1 error and the 1_2 error, but the present invention is not limited thereto.

In addition, the learning apparatus 100 measures at least one quality with reference to at least a part of the entropy and the noise level of the modulated training data x', and further refers to the measured quality to obtain a first error. have. That is, the learning apparatus 100 can learn the modulation network O to minimize the quality of the modulated learning data x', for example, to maximize the entropy, noise, etc. of the modulated learning data x'. have.

And, the learning apparatus 100 minimizes the first loss and maximizes the second loss. When learning the modulation network O, the learning apparatus 100 fixes the learned parameters of the learning network F It is not updated, and learning can be performed only for the modulation network (O).

Meanwhile, in the above description, the learning apparatus 100 (i) (i-1) the (1_1)_1th obtained by referring to the first_1 characteristic information F1(x') and the second_1 characteristic information F1(x)) 1_1 error, which is an average of (1_1)_n-th errors obtained by referring to the error to 1_n-th characteristic information (Fn(x')) and 2_n-th characteristic information (Fn(x)), and (i-2) 1_1 The (1_2)_1th error to 1_nth characteristic information Fn(x')) obtained by referring to the first task specific output generated using the characteristic information F1(x') and the corresponding first GT Minimize the first error calculated by referring to at least a part of the 1_2 error, which is an average of the (1_2)_n-th error obtained by referring to the n-th task specific output generated using the n-th task specific output and the corresponding n-th GT; (ii) The modulation network O may be trained to maximize the second error calculated by referring to the training data and the modulated training data x'. For example, a first_1st error calculated with reference to at least a portion of a (1_1)_1th error and a (1_2)_1th error, a 1_nth error calculated with reference to at least a portion of a (1_1)_nth error and a (1_2)_nth error It is also possible to learn the modulation network O sequentially to minimize .

That is, the learning device 100 inputs the training data x into the modulation network O to cause the modulation network O to modulate the training data x to generate the first modulated training data x1'. let it do And, the learning device 100 (i) input the first modulated learning data (x1 ') to the first learning network (F1) to cause the first learning network (F1) to (i-1) the first learning network (i-2) 1_1 characteristic corresponding to the first modulated training data (x1') by applying a network operation to the first modulated training data (x1') using the first learned parameter of (F1) a process of outputting information F1(x1'), and (ii) inputting the training data x into the first learning network F1 to cause the first learning network F1 to (ii-1) first A process of applying a network operation to the training data (x) using the learned parameters and (ii-2) outputting the second_1 characteristic information (F1(x)) corresponding to the training data (x) is performed or other Let the device do it. Then, the learning apparatus 100 (i) (i-1) at least one (1_1) first characteristic information F1(x1') and at least one (1_1) obtained by referring to the second_1 characteristic information F1(x)) _1 error, (i-2) at least one first task specific output generated using the first_1 characteristic information F1(x'), and at least one obtained by referring to at least one first GT corresponding thereto At least one first_1 error calculated by referring to at least some of the (1_2)_1 errors of (ii) at least one calculated with reference to the training data (x) and the first modulated training data (x1') To maximize the 2_1 error of , the modulation network O is trained so that the modulation network O becomes the first learned modulation network O1.

Then, the learning apparatus 100 increases the integer k from 2 to n and repeats the above process until the n-th learning network Fn is performed to obtain the n-th modulation network On.

That is, the learning apparatus 100 inputs the training data x into the (k-1)-th learned modulation network O(k-1), and the (k-1)-th learned modulation network O(k). -1)) to modulate the training data (x) to generate the k-th modulated training data (xk'). And, the learning apparatus 100 (i) input the k-th modulated training data (xk ') to the k-th learning network (Fk) to cause the k-th learning network (Fk) to at least the k-th learning network (Fk) 1_k characteristic information (Fk(xk')) corresponding to the k-th modulated training data (xk') by applying a network operation to the k-th modulated training data (xk') using one k-th learned parameter ), and (ii) input the training data (x) to the k-th learning network (Fk) so that the k-th learning network (Fk)) uses the k-th learned parameters to network the training data (x). The operation is applied to output the second_k characteristic information Fk(xk) corresponding to the training data x. Thereafter, the learning apparatus 100 (i) (i-1) at least one (1_1) th (1_1) obtained by referring to the first_k characteristic information (Fk(xk')) and the second_k characteristic information (Fk(x)) _k error and (i-2) at least one (1_2)_k obtained by referring to at least one k-th task specific output generated using the 1_k characteristic information and at least one k-th GT corresponding thereto At least one first_k error obtained by referring to at least some of the errors is minimized, and (ii) at least one second_k error calculated by referring to the training data (x) and the k-th modulated training data (xk′) is maximized. so that the k-1 th learned modulation network O(k-1) is learned so that the k-1 th learned modulation network O(k-1) becomes the k-th learned modulation network Ok. can do.

4 schematically illustrates a method for learning a modulation network for concealing original data according to another embodiment of the present invention. In the following description, detailed descriptions of parts that can be easily understood from the descriptions of FIGS. 2 and 3 will be omitted.

First, when the learning data (x) is obtained, the learning apparatus 100 inputs the learning data (x) to the modulation network (O) to cause the modulation network (O) to modulate the learning data (x). The training data (x', that is, O(x)) is generated.

Next, the learning apparatus 100 (i) inputs the modulated learning data (x') to the learning network (F) having the learned parameters to cause the learning network (F) to (i-1) the learned parameters A process of generating the first characteristic information (F(x')) corresponding to the modulated training data (x') by applying a network operation to the modulated training data (x') using (i-2) , and (ii) input the training data (x) to the learning network (F) to cause the learning network (F) to (ii-1) apply a network operation to the training data (x) using the learned parameters. (ii-2) A process for generating the second characteristic information F(x) corresponding to the learning data x may be performed or another device may be supported to perform.

Next, in a state in which the score of at least one modulated learning data corresponding to the modulated learning data (x') input to the discriminator (D) for determining whether the input data is real or fake is obtained as a second error , the learning apparatus 100 (i) learns the modulation network (O) to minimize the first error and maximize the second error, (ii) transform learning data or modulation input to the discriminator (D) At least one transform learning data score or at least one modulated transform learning data score corresponding to the transformed learning data can be maximized and the discriminator D can be trained to minimize the modulated learning data score. At this time, the transform learning data or the modulated transform learning data is to be generated by adding at least one random noise generated through a random noise generating network (not shown) to the training data (x) or the modulated training data (x'), respectively. can As an example, the random noise generation network generates random noise having a normal distribution N(0, σ), and adds the generated noise to the training data (x) or the modulated training data (x') to transform the training data Alternatively, modulated transform learning data may be generated, respectively. In addition, the transform learning data or the modulated transform learning data is the training data (x) or the modulated training data (x'), respectively, blur processing, or the resolution of the training data (x) or the modulated training data (x') may be generated by changing, but is not limited thereto, and various methods of transforming the learning data (x) or the modulated learning data (x') may be applied.

That is, in learning the modulation network (O), the learning apparatus 100 modulates the training data (x) by using the first error to be recognized as the same or similar to the training data (x). It learns to output the modulated learning data (x'), and can learn to output the modulated learning data (x') that is different from the learning data (x) but difficult to distinguish from the learning data (x) by using the second error. have.

At this time, the maximum value of the transformation learning data score or the modulated transformation learning data score corresponding to the transformation learning data or the modulated transformation learning data input to the discriminator (D) is the transformation learning data or the modulated transformation learning data as real. The minimum value of the modulated learning data score corresponding to the modulated learning data (x') input to the discriminator (D) may be 0 to determine that the modulated learning data (x') is fake. . That is, the discriminator D may be trained to recognize the modulated learning data x' as transform learning data or modulated transform learning data.

5 schematically shows another method of learning a modulation network for concealing original data according to another embodiment of the present invention, and the learning network F in FIG. It is composed of networks F1, F2, ..., Fn. In the following description, detailed descriptions of parts that can be easily understood from the descriptions of FIGS. 2 to 4 will be omitted.

First, when the learning data (x) is obtained, the learning apparatus 100 inputs the learning data (x) to the modulation network (O) to cause the modulation network (O) to modulate the learning data (x). The training data (x', that is, O(x)) is generated.

Next, the learning device 100 inputs the modulated learning data (x ') to each of the first learning network (F1) to the n-th learning network (Fn) to each of the first learning network (F1) to the n-th learning Let the network Fn (i) the first learning network F1 to the n-th learning network Fn, respectively, with respect to the modulated training data (x') using the first learned parameter to the n-th learned parameter (ii) 1_1 characteristic information (F1(x')) to 1_n characteristic information (Fn(x')) corresponding to the modulated learning data (x') by applying a network operation corresponding to each network to create In addition, the learning device 100 inputs the training data (x) to each of the first learning network (F1) to the n-th learning network (Fn) to each of the first learning network (F1) to the n-th learning network (Fn) (i) a first learning network (F1) to a network corresponding to each network for the training data (x) using the first learned parameter to the n-th learned parameter of each of the learning networks (Fn) By applying the operation (ii), each of 2_1 characteristic information F1(x) to 2_n characteristic information Fn(x) corresponding to the learning data x may be generated.

Next, the learning apparatus 100 (i) (i-1) the (1_1)_1th error obtained by referring to the first_1 characteristic information F1(x') and the second_1 characteristic information F1(x)) 1_1 error that is an average of (1_1)_n-th errors obtained by referring to the 1_n-th characteristic information Fn(x') and the 2_n-th characteristic information Fn(x)), and (i-2) the first_1 characteristic At least one first task specific output generated using the information F1(x') and the (1_2)_1th error to 1_nth characteristic information Fn( x'))) and at least one n-th task-specific output generated by using The modulation network O is trained so as to minimize the calculated first error and (ii) maximize the second error, which is the modulated training data score corresponding to the modulated training data x' input to the discriminator (D). can do. And the learning apparatus 100 maximizes the transform learning data score or the modulated transform learning data score corresponding to the transform learning data or the modulated transform learning data input to the discriminator (D) and minimizes the modulated learning data score, The discriminator (D) can be learned.

That is, the learning apparatus 100 obtains the (1_1)_1th error calculated with reference to the first_1 characteristic information F1(x') and the second_1 characteristic information F1(x), and the first_2 characteristic information ( A (1_1)_2th error calculated by referring to F2(x')) and 2_2 characteristic information (F2(x)) is obtained, and similarly, 1_n characteristic information (Fn(x')) and 2_n characteristic information ( By calculating a (1_1)_nth error with reference to Fn(x)), a 1_1st error that is an average of the obtained (1_1)_1th error to the obtained (1_1)_nth error may be obtained. In addition, the learning apparatus 100 provides a first task specific output generated using the first_1 characteristic information F1(x') and a (1_2)_1th error to a 1_nth characteristic calculated with reference to a first GT corresponding thereto. The (1_2)_nth error calculated by referring to the nth task specific output generated using the information Fn(x') and the nth GT corresponding thereto is obtained, and the calculated (1_2)_1th error to the calculated (1_2)_1th error A first_2 error that is an average of (1_2)_n errors may be obtained. In addition, the learning apparatus 100 may learn the modulation network O to minimize the calculated first error and maximize the second error by referring to at least some of the first_1 error and the first_2 error.

Meanwhile, in the above-described learning apparatus 100, (i) (i-1) the (1_1)_1th obtained by referring to the first_1 characteristic information F1(x') and the second_1 characteristic information F1(x)) 1_1 error, which is an average of (1_1)_n-th errors obtained by referring to the error to 1_n-th characteristic information Fn(x') and 2_n-th characteristic information, and (i-2) 1_1 characteristic information F1(x) '))) and the (1_2)_1st error to 1_nth characteristic information (Fn(x')) obtained by referring to the corresponding first GT Minimize the first error calculated by referring to at least a part of the 1_2 error that is the average of the (1_2)_nth error obtained by referring to the task specific output and the nth GT corresponding thereto, and (ii) the modulated modulated input input to the discriminator. The modulation network O was trained to maximize the second error, which is the modulated training data score corresponding to the training data. As another example, the first_1st error calculated with reference to at least a portion of the (1_1)_1th error and the (1_2)_1th error, the (1_1)_nth error calculated with reference to at least a portion of the (1_1)_nth error and the (1_2)_nth error calculated with reference to at least a portion of the (1_2)_nth error It is also possible to learn the modulation network O sequentially to minimize .

That is, the learning apparatus 100 inputs the training data x into the modulation network O so that the modulation network O modulates the training data x to generate the first modulated training data x1'. do. And, the learning apparatus 100 (i) input the first modulated learning data (x1 ') to the first learning network (F1) to cause the first learning network (F1) to (i-1) the first learning network (i-2) 1_1 characteristic corresponding to the first modulated training data (x1') by applying a network operation to the first modulated training data (x1') using the first learned parameter of (F1) a process for outputting information F1(x1'), and (ii) inputting the training data x into the first learning network F1 to cause the first learning network F1 to (ii-1) first A process of applying a network operation to the training data (x) using the learned parameters and (ii-2) outputting the second_1 characteristic information (F1(x)) corresponding to the training data (x) is performed, or It can support the device to perform Then, the learning apparatus 100 (i) (i-1) at least one (1_1) first characteristic information F1(x1') and at least one (1_1) obtained by referring to the second_1 characteristic information F1(x) _1 error, (i-2) at least one first task specific output generated using the first_1 characteristic information F1(x'), and at least one obtained by referring to at least one first GT corresponding thereto (ii) at least one corresponding to the first modulated learning data (x1') input to the discriminator (D) and minimizing at least one first_1 error calculated by referring to at least a part of (1_2)_1 errors The modulation network O may be trained to maximize at least one second_1 error, which is the first modulated training data score of , so that the modulation network O becomes the first learned modulation network O1. And the learning apparatus 100 maximizes the at least one first transform learning data score or at least one first modulated transform learning data score corresponding to the transform learning data or the modulated transform learning data input to the discriminator (D). and learn the discriminator D so as to minimize the score of at least one first modulated learning data, so that the discriminator D becomes the first learned discriminator D1.

Then, the learning apparatus 100 increases the integer k from 2 to n and repeats the above process to obtain the n-th learned modulation network On by performing the above process until the n-th learning network Fn.

That is, the learning apparatus 100 inputs the training data x into the k-1th learned modulation network O(k-1) to cause the k-1th modulation network O(k-1). By modulating the training data (x), the k-th modulated training data (xk') is generated. And, the learning apparatus 100 (i) input the k-th modulated training data (xk ') to the k-th learning network (Fk) to cause the k-th learning network (Fk) to at least the k-th learning network (Fk) 1_k characteristic information (Fk(xk')) corresponding to the k-th modulated training data (xk') by applying a network operation to the k-th modulated training data (xk') using one k-th learned parameter ), and (ii) input the training data (x) to the k-th learning network (Fk) so that the k-th learning network (Fk)) uses the k-th learned parameters to network the training data (x). By applying the operation, the second_k characteristic information Fk(xk) corresponding to the learning data x may be output. Then, the learning apparatus 100 (i) (i-1) at least one (1_1)_k obtained by referring to the first_k characteristic information Fk(xk') and the second_k characteristic information Fk(x) An error, (i-2) at least one kth task specific output generated using the 1_k characteristic information Fk(x'), and at least one kth GT obtained by referring to the at least one kth GT corresponding thereto At least one 1_k error calculated by referring to at least some of the (1_2)_k errors is minimized and (ii) k-th modulated learning input to the k-1 th learned discriminator D(k-1)) By learning the k−1th learned modulation network O(k−1) to maximize at least one 2_k error that is at least one kth modulated training data score corresponding to the data xk′, The k−1 learned modulation network O(k−1) may be the k−th learned modulation network Ok, and the learning apparatus 100 provides the k−1 learned discriminator D(k). At least one k-th transform learning data score or at least one k-th modulated transform learning data score corresponding to the transform learning data or modulated transform learning data input to -1)) is maximized, and at least one k-th modulated transform learning data score is maximized. To minimize the learning data score, the k-1 th learned discriminator (D(k-1)) is trained, and the k-1 th learned discriminator (D(k-1)) is the k-th learned discriminator. (Dk) can be

6 schematically shows a test apparatus for testing a learned modulation network O according to an embodiment of the present invention.

Referring to FIG. 6 , the test apparatus 200 according to an embodiment of the present invention is a learned learning method in which a result calculated using test data modulated in a learning network is the same as or similar to a result calculated using test data It may include a memory 210 storing instructions for testing the modulation network and a processor 220 performing a process of testing the learned modulation network according to the instructions stored in the memory 210 .

Specifically, the test device 200 includes at least one computing device (eg, a computer processor, memory, storage, input device and output device, other components of a conventional computing device; an electronic communication device such as a router, a switch, etc.; a network attachment) Electronic information storage systems such as storage (NAS) and storage area networks (SANs)) and computer software (i.e., instructions that cause a computing device to function in a particular way) are used to generally achieve the desired system performance. it could be

In addition, the processor of the computing device may include a hardware configuration such as a micro processing unit (MPU) or a central processing unit (CPU), a cache memory, and a data bus. In addition, the computing device may further include an operating system and a software configuration of an application for performing a specific purpose.

However, this does not exclude an integrated processor including any combination of processor, memory, medium or other computing element for the computing device to practice the present invention.

On the other hand, the learning device (i) inputs the training data to the modulation network so that the modulation network modulates the training data to generate modulated training data, and (ii) the modulated training data is applied to the learning network having the learned parameters. By inputting, the learning network applies a network operation to the modulated training data using the learned parameters to generate the first characteristic information corresponding to the modulated training data, and input the training data into the learning network to the learning network to generate second characteristic information corresponding to the training data by applying a network operation to the training data using the learned parameters, (iii) (iii-1) the first characteristic information and the second characteristic information The first error calculated by referring to at least a part of the first_1 error obtained by reference, (iii-2) the task specific output generated using the first characteristic information, and the first_2 error obtained by referring to the GT corresponding thereto When the test data is obtained in a state in which the modulation network is learned so as to minimize and maximize the second error obtained by referring to the training data and the modulated learning data, the processor 220 according to the instructions stored in the memory 210 A, input the test data to the modulation network and cause the modulation network to modulate the test data using the learned parameters of the modulation network to output the modulated test data, and to apply the modulated test data to the concealed-processed data of the test data can be obtained with

In addition, the learning device (i) inputs the training data to the modulation network to cause the modulation network to modulate the training data to generate modulated training data, and (ii) the modulated training data to the learning network having the learned parameters. By inputting, the learning network applies a network operation to the modulated training data using the learned parameters to generate the first characteristic information corresponding to the modulated training data, and input the training data into the learning network to the learning network to generate second characteristic information corresponding to the training data by applying a network operation to the training data using the learned parameters, (iii) (iii-1) the first characteristic information and the second characteristic information The first error calculated by referring to at least a part of the first_1 error obtained by reference and (iii-2) the task specific output generated using the first characteristic information and the first_2 error obtained by referring to the GT corresponding thereto In addition to learning the modulation network to minimize and maximize the second error, which is the modulated training data score corresponding to the modulated training data input to the discriminator that determines whether the input data is real or fake, (iv) discrimination When the test data is obtained in the state of learning the discriminator so as to maximize the transform learning data score or the modulated transform learning data score corresponding to the transformed learning data or the modulated transform learning data input to the unit and minimize the modulated learning data score , according to the instructions stored in the memory 210, the processor 220 inputs the test data to the modulation network so that the modulation network modulates the test data using the learned parameters of the modulation network and outputs the modulated test data. , the modulated test data may be acquired as concealed-processed data of the test data.

7 schematically illustrates a method for testing a learned modulation network according to an embodiment of the present invention.

Referring to FIG. 7 , the test device 200 transmits test data (eg For example, by inputting the original image on the left of Figure 7), we cause the modulation network image) to be printed.

For reference, FIG. 7 exemplarily shows 64 image samples selected from the CIFAR-10 data set, which is data collected and organized by the Canadian Institute for Advanced Research (CIFAR) to solve the image classification problem. .

Modulated data obtained by concealing an image sample on the left side of FIG. 7 as original data according to an embodiment of the present invention is shown as the right side of FIG. 7 .

Referring to FIG. 7 , the 64 modulated data on the right side of FIG. 7 that have been concealed according to an embodiment of the present invention are visually different from the 64 pieces of original data on the left side of FIG. As input, the learning network outputs the same or similar results to the original data.

Meanwhile, the learned modulation network O may be in a learned state by the same method as in the description of FIGS. 2 to 5 .

That is, the learning apparatus (i) inputs the training data to the modulation network so that the modulation network modulates the training data to generate modulated training data, and (ii) the learning network having the modulated training data learned parameters. input to the learning network to apply a network operation to the modulated learning data using the learned parameters to generate first characteristic information corresponding to the modulated learning data, and input the learning data into the learning network to learn By causing the network to apply a network operation to the training data using the learned parameters to generate second characteristic information corresponding to the training data, (iii) (iii-1) the first characteristic information and the second characteristic information The first calculated by referring to at least a part of the first_1 error obtained by referring to (iii-2) the task specific output generated using the first characteristic information and the first_2 error obtained by referring to the GT corresponding thereto In order to minimize the error and maximize the second error obtained by referring to the training data and the modulated training data, the modulation network O may be learned.

In addition, in the above, the learning network includes a first learning network to an n-th learning network having a first learned parameter to an n-th learned parameter, respectively, and the learning apparatus includes (i) the modulated learning data in the first learning Input to each of the networks to the n-th learning networks to cause each of the first learning networks to the n-th learning networks to (i-1) the first learned parameters to the n-th learned parameters of each of the first learning networks to the n-th learning networks to apply a network operation corresponding to each network to the modulated training data using (i-2) to output each 1_1 characteristic information to 1_n characteristic information corresponding to the modulated training data, (ii) ) input the training data into each of the first learning network to the n-th learning network to cause each of the first learning network to the n-th learning network to (ii-1) the first learning network to each of the n-th learning networks. By applying the network operation corresponding to each network to the training data using the parameter to the n-th learned parameter (ii-2) to output the respective 2_1 characteristic information to the 2_n characteristic information for the training data, (iii) (iii-1) (iii-1a) (1_1)_1th error obtained by referring to the 1_1 characteristic information and the 2_1 characteristic information to (1_1)_1th error obtained by referring to the 1_n characteristic information and the 2_n characteristic information 1_1) a first error that is an average of _n errors, (iii-1b) at least one first task specific output generated using the first_1 characteristic information, and at least one first GT obtained by referring to the corresponding at least one first GT (1_2)_1 error 1_2th error that is an average of (1_2)_nth errors obtained by referring to at least one nth task specific output generated using the (1_2)_1 error to 1_nth characteristic information and at least one nth GT corresponding thereto (iii-2) the first error calculated with reference to at least a part of the 2 In order to maximize the error, the modulation network may be trained.

And, the learning device, (i) inputs the training data to the modulation network to cause the modulation network to modulate the training data to generate modulated training data, and (ii) the learning network having the modulated training data learned parameters. input to the learning network to apply a network operation to the modulated learning data using the learned parameters to generate first characteristic information corresponding to the modulated learning data, and input the learning data into the learning network to learn The network generates second characteristic information corresponding to the training data by applying a network operation to the training data using the learned parameters, (iii) (iii-1) 1_1 characteristic information and 2_1 characteristic information (1_1)_1st error obtained by referring to the 1_1th error that is an average of the 1_nth characteristic information and the (1_1)_nth error obtained with reference to the 2_nth characteristic information, and (iii-2) 1_1 characteristic information At least one n-th task specification generated using (1_2)_1th error to 1_n-th characteristic information obtained by referring to at least one first task specification output generated using the at least one first GT corresponding thereto Minimizes the first error calculated by referring to at least a part of the 1_2 error that is the average of the (1_2)_n-th error obtained by referring to the output and at least one n-th GT corresponding thereto, and whether the input data is real or fake In addition to learning the modulation network (O) so as to maximize the second error that is the modulated training data score corresponding to the modulated training data input to the discriminator for determining (iv) the transform training data or modulation input to the discriminator To maximize the transformed learning data score or the modulated transformed learning data score corresponding to the transformed learning data, and to minimize the modulated learning data score, the discriminator may be in a learned state.

In addition, in the above, the learning network includes a first learning network to an n-th learning network each having a first learned parameter to an n-th learned parameter, and the learning apparatus includes (i) the modulated learning data in the first learning Input to each of the networks to the n-th learning networks to cause each of the first learning networks to the n-th learning networks to (i-1) the first learned parameters to the n-th learned parameters of each of the first learning networks to the n-th learning networks to apply a network operation corresponding to each network to the modulated training data using (i-2) to output each 1_1 characteristic information to 1_n characteristic information for the modulated training data, (ii) By inputting the training data to each of the first learning network to the n-th learning network, each of the first learning network to the n-th learning network causes (ii-1) the first learned parameter of each of the first learning network to the n-th learning network to apply a network operation corresponding to each network to the training data using the to n-th learned parameters (ii-2) to output each 2_1 characteristic information to 2_n characteristic information for the training data, ( iii) (iii-1) (iii-1a) (1_1)_1th error obtained by referring to the first_1 characteristic information and the 2_1 characteristic information to (1_1)th obtained by referring to the 1_n first characteristic information and the 2_n characteristic information ) _1 error that is an average of _n errors, (iii-1b) at least one first task specific output generated using the 1_1 characteristic information, and at least one first GT obtained by referring to the corresponding at least one first GT 1_2)_1 error, at least one n-th task specific output generated using the 1_n-th characteristic information, and at least one (1_2)_n-th error obtained by referring to at least one n-th GT corresponding thereto The first error calculated by referring to at least a part is minimized (iii-2) and the modulated learning data corresponding to the modulated learning data input to the discriminator is minimized. Learning the modulation network to maximize the second error, which is the wet data score, (iv) maximizing the transform training data score or the modulated transform training data score corresponding to the transform training data input to the discriminator or the modulated transform training data; In order to minimize the modulated learning data score, the discriminator may be in a learned state.

On the other hand, the converted data concealed in the learned modulation network according to the embodiment of the present invention may be provided or sold to the image big data purchaser.

In addition, according to an embodiment of the present invention, while the concealed image data is provided or sold to the above purchaser, the method of testing the learned modulation network is provided in the form of program instructions that can be executed through various computer components. It may be implemented and recorded on a computer-readable recording medium and provided. According to an embodiment of the present invention, the purchaser executes program instructions recorded on the recording medium using a computing device, thereby generating concealed-processed data from original data that he owns separately or obtained from other sources, , the concealed data becomes available in its own learning network. The purchaser may also use in its own learning network at least two or more of the concealed data, the original image data in its possession or obtained from other sources, and the concealed image data provided or sold to the buyer. there will be

On the other hand, according to an embodiment of the present invention, when the method of testing the learned modulation network is implemented in the form of program instructions that can be executed through various computer components, the accuracy is set to high, so that the computing device of the purchaser overcomputs. Since it can incur computational overhead, it is possible to avoid computational overhead by allowing the buyer to reduce the accuracy through settings.

On the other hand, the "average" specified herein means a weighted average (Weighted Average), but is not limited thereto.

The embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and used by those skilled in the computer software field. Examples of the computer-readable recording medium include hard disks, magnetic media such as floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floppy disks. media), and hardware devices specially configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

In the above, the present invention has been described with specific matters such as specific components and limited embodiments and drawings, but these are provided to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. , those of ordinary skill in the art to which the present invention pertains can make various modifications and variations from these descriptions.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and not only the claims described below but also all modifications equivalently or equivalently to the claims described below belong to the scope of the spirit of the present invention. will do it

110: memory,
120: processor;
200: test device;
210: memory,
220: processor

Claims (28)

In the method of learning the modulation network (Obfuscation Network) processing the original data for the protection of personal information by concealing (Concealing),
(a) when the training data is obtained, the learning device inputs the training data to a modulation network to cause the modulation network to modulate the training data to generate modulated training data;
(b) the learning device, (i) inputs the modulated training data to a learning network having at least one learned parameter to cause the learning network to (i-1) use the learned parameter By applying a network operation to the training data (i-2) to generate first characteristic information (Characteristic Information) corresponding to the modulated training data, (ii) to input the training data into the learning network (ii-1) causing a learning network to apply a network operation to the training data using the learned parameters (ii-2) to generate second characteristic information corresponding to the training data; and
(c) the learning apparatus, (i) (i-1) at least one first_1 error obtained by referring to the first characteristic information and the second characteristic information, and (i-2) the first characteristic information At least one first error calculated by referring to at least a part of at least one first_2 error obtained by referring to a task specific output generated using and (ii) learning the modulation network to maximize at least one second error calculated with reference to the training data and the modulated training data;
How to include. The method of claim 1,
wherein the learning network comprises a first learning network to an n-th learning network having at least one first learned parameter to at least one n-th learned parameter, wherein when n is an integer greater than or equal to 1,
In step (b),
The learning device, (i) by inputting the modulated training data to each of the first learning network to the n-th learning network, to cause each of the first learning network to the n-th learning network to (i-1) the To apply a network operation corresponding to each network to the modulated training data using the first learned parameter to the n-th learned parameter of each of the first learning network to the n-th learning network (i-2) to output each of 1_1 characteristic information to 1_n characteristic information for the modulated training data, and (ii) input the training data into each of the first learning network to the n-th learning network to each of the first (ii-1) causing the learning network to the n-th learning network to apply a network operation corresponding to each network to the training data using each of the first learned parameter to the n-th learned parameter ( ii-2) to output each of 2_1 characteristic information to 2_n characteristic information for the learning data,
In step (c),
The learning apparatus, (i) (i-1) a (1_1)_1 error obtained by referring to the first_1 characteristic information and the 2_1 characteristic information, to the 1_n characteristic information and the 2_n characteristic information The first_1 error that is the average of the obtained (1_1)_nth errors, (i-2) (i-2a) at least one first task specific output generated using the first_1 characteristic information, and (i-2b) ) (1_2)_1th error obtained with reference to at least one first GT corresponding thereto to (i-2a) at least one nth task specific output generated using the 1_n first characteristic information and (i-2b) ) minimizes the first error calculated by referring to at least a part of the first_2 errors, which is an average of (1_2)_n-th errors obtained with reference to at least one n-th GT corresponding thereto, and (ii) the training data and A method of learning the modulation network to maximize the second error calculated with reference to the modulated training data. The method of claim 1,
wherein the learning network comprises a first learning network to an n-th learning network having at least one first learned parameter to at least one n-th learned parameter, wherein when n is an integer greater than or equal to 1,
In step (a),
The learning device inputs the learning data to the modulation network to cause the modulation network to modulate the learning data to generate first modulated learning data,
In step (b),
The learning device, (i) input the first modulated training data to the first learning network to cause the first learning network to (i-1) use the first learned parameter of the first learning network By applying a network operation to the first modulated training data (i-2) to output the first_1 characteristic information for the first modulated training data, (ii) the training data to the first learning network to (ii-1) the first learning network to apply a network operation to the training data using the first learned parameter, and (ii-2) output the second_1 characteristic information for the training data. to do,
In step (c),
The learning apparatus includes (i) (i-1) at least one (1_1)_1 error obtained by referring to the first_1 characteristic information and the second_1 characteristic information, and (i-2) the first_1 characteristic information At least one first_1 obtained by referring to at least a part of at least one first task specific output generated using , and at least one (1_2)_1th error obtained by referring to at least one first GT corresponding thereto To minimize the error and (ii) maximize the at least one second_1 error calculated with reference to the training data and the first modulated training data, the modulation network is trained so that the modulation network is a first learned modulation network. to become,
The learning apparatus increases k, which is an integer, from 2 to n, (i) inputs the training data to the k-1 th learned modulation network, and modulates the training data by the k-1 th learned modulation network. to generate the k-th modulated training data, (ii) (ii-1) input the k-th modulated training data to the k-th learning network to cause the k-th learning network to perform at least the k-th learning network A network operation is applied to the k-th modulated training data using one k-th learned parameter to output 1_k characteristic information for the k-th modulated training data, (ii-2) the training data is input to the k-th learning network to cause the k-th learning network to apply a network operation to the training data using the k-th learned parameter to output 2_k characteristic information for the training data, ( iii) (iii-1) (iii-1a) at least one (1_1)_k error obtained by referring to the first_k characteristic information and the second_k characteristic information, and (iii-1b) the first_k characteristic information At least one 1_k calculated by referring to at least a part of at least one (1_2)_k th error obtained by further referring to at least one k th task specific output generated by using and at least one k th GT corresponding thereto By learning the k-1 th learned modulation network so as to minimize the error and (iii-2) maximize the at least one 2_k error calculated with reference to the training data and the k-th modulated training data, -1 How to make the learned modulation network become the kth learned modulation network. The method of claim 1,
In step (c),
A state in which at least one obfuscated training data score corresponding to the modulated training data input to a discriminator for determining whether the input data is real or fake is obtained as the second error In, the learning device, (i) learning the modulation network to minimize the first error and maximize the second error, (ii) transform learning data input to the discriminator or modulated transform learning learning the discriminator so as to maximize at least one transform training data score or at least one modulated transform training data score corresponding to data and minimize the modulated training data score, the transform training data or the modulated transform learning data The data is the data obtained by converting the learning data or the modulated learning data, respectively. 5. The method of claim 4,
wherein the learning network comprises a first learning network to an n-th learning network having at least one first learned parameter to at least one n-th learned parameter, wherein when n is an integer greater than or equal to 1,
In step (b),
The learning apparatus, (i) by inputting the modulated learning data to each of the first learning network to the n-th learning network, to cause each of the first learning network to the n-th learning network to (i-1) the To apply a network operation corresponding to each network to the modulated training data using the first learned parameter to the n-th learned parameter of each of the first learning network to the n-th learning network (i-2 ) to output each of 1_1 characteristic information to 1_n characteristic information for the modulated training data, (ii) input the training data into each of the first learning network to the n-th learning network, (ii-1) causing the first learning network to the n-th learning network to apply a network operation corresponding to each network to the training data using each of the first learned parameter to the n-th learned parameter (ii-2) to output each of 2_1 characteristic information to 2_n characteristic information for the learning data,
In step (c),
The learning apparatus is configured to: (i) (i-1) (1_1)_1th error obtained by referring to the 1_1 characteristic information and the 2_1 characteristic information to the 1_n characteristic information and the 2_n characteristic information by referring to the The first_1 error that is the average of the obtained (1_1)_nth errors, (i-2) (i-2a) at least one first task specific output generated using the first_1 characteristic information, and (i-2b) ) (1_2)_1th error obtained with reference to at least one first GT corresponding thereto to (i-2a) at least one nth task specific output generated using the 1_n first characteristic information and (i-2b) ) The modulated learning data input to the discriminator by minimizing the first error calculated by referring to at least a part of the first_2 errors, which is an average of the (1_2)_nth errors obtained with reference to the corresponding nth GT The modulation network is trained to maximize the second error, which is the modulated training data score corresponding to , (ii) the transformation learning data input to the discriminator or the transformation learning corresponding to the modulated transformation training data. A method of learning the discriminator to maximize a data score or the modulated transformed training data score and minimize the modulated training data score. 5. The method of claim 4,
The learning network includes a first learning network to an n-th learning network each having a first learned parameter to an n-th learned parameter, and when n is an integer greater than or equal to 1,
In step (a),
The learning device inputs the learning data to the modulation network to cause the modulation network to modulate the learning data to generate first modulated learning data,
In step (b),
The learning device, (i) input the first modulated training data to the first learning network to cause the first learning network to (i-1) use the first learned parameter of the first learning network By applying a network operation to the first modulated training data (i-2) to output the first_1 characteristic information for the first modulated training data, (ii) the training data to the first learning network to (ii-1) the first learning network to apply a network operation to the training data using the first learned parameter, and (ii-2) output the second_1 characteristic information for the training data. to do,
In step (c),
The learning apparatus includes (i) (i-1) at least one (1_1)_1 error obtained by referring to the first_1 characteristic information and the second_1 characteristic information, and (i-2) (i-2a) At least one first task specific output generated using the first_1 characteristic information and (i-2b) at least a part of the (1_2)_1th error obtained by referring to the at least one first GT corresponding thereto to minimize the obtained at least one first_1 error and (ii) maximize the at least one second_1 error that is the at least one first modulated training data score corresponding to the first modulated training data input to the discriminator, The modulation network is learned so that the modulation network becomes a first learned modulation network, and the learning apparatus includes (i) at least one corresponding to the transformation learning data input to the discriminator or the modulated transformation learning data. learning the discriminator to maximize the first transformed training data score or at least one first modulated transformed training data score of (ii) minimizing the first modulated training data score so that the discriminator is a first learned discriminator make it giddy,
The learning apparatus increases k, which is an integer, from 2 to n, (i) inputs the training data to the k-1 th learned modulation network, and modulates the training data by the k-1 th learned modulation network. to generate the k-th modulated training data, (ii) (ii-1) input the k-th modulated training data to the k-th learning network to cause the k-th learning network to perform at least the k-th learning network A network operation is applied to the k-th modulated training data using one k-th learned parameter to output 1_k characteristic information for the k-th modulated training data, (ii-2) the training data is input to the k-th learning network to cause the k-th learning network to apply a network operation to the training data using the k-th learned parameter to output 2_k characteristic information for the training data, ( iii) (iii-1) at least one (1_1)_k error obtained by referring to the first_k characteristic information and the second_k characteristic information, and (iii-2) at least one generated using the first_k characteristic information At least one 1_k error calculated by referring to at least a part of at least one (1_2)_k error obtained by referring to one k-th task specific output and at least one k-th GT corresponding thereto is minimized, and -1 By learning the k-1th learned modulation network to maximize the at least one 2_k error that is the score of at least one k-th modulated training data corresponding to the k-th modulated training data input to the learned discriminator , such that the k-1 th learned modulation network becomes a k-th learned modulation network, and (iv) at least corresponding to the transformation learning data input to the k-1 th learned discriminator or the modulated transformation learning data. Maximizes one kth transform training data score or at least one kth modulated transform training data score and minimizes the kth modulated training data score A method of learning the k-1 th learned discriminator so that the k-1 th learned discriminator becomes a k th learned discriminator. 5. The method of claim 4,
The maximum value of the transformation learning data score or the modulated transformation learning data score corresponding to the transformation learning data or the modulated transformation learning data input to the discriminator is the transformation learning data or the modulated transformation learning data, respectively. 1, which is determined to be 1, and the minimum value of the modulated training data score corresponding to the modulated training data input to the discriminator is 0 to determine that the modulated training data is fake. The method of claim 1,
In step (c),
The learning apparatus is configured to refer to at least a portion of (i) a difference between the first characteristic information and the second characteristic information, and (ii) at least one difference between the task specific output and the GT, to provide the first error and obtaining the second error by at least one difference between the training data and the modulated training data. 9. The method of claim 8,
The learning apparatus obtains the first error with reference to a norm or cosine similarity between the first characteristic information and the second characteristic information. The method of claim 1,
In step (c),
The learning apparatus measures at least one quality with reference to at least a part of entropy and noise degree of the modulated learning data, and obtains the first error with reference to the measured quality. In a test method for learning a tampering network (Obfuscation Network) that conceals original data for personal information protection,
(a) a process by which a learning device (i) acquires training data and inputs it to a modulation network to cause the modulation network to modulate the training data to generate modulated training data, (ii) the modulated learning Input data to a learning network having at least one learned parameter to cause the learning network to (ii-1) apply a network operation to the modulated training data using the learned parameter (ii-2) a process of generating first characteristic information corresponding to the modulated training data, (iii) inputting the training data into the learning network to cause the learning network to (iii-1) learn the learning using the learned parameters (iii-2) a process for generating second characteristic information corresponding to the learning data by applying a network operation to data; and (iv) (iv-1) (iv-1a) the first characteristic information and the At least one first_1 error obtained with reference to the second characteristic information, (iv-1b) at least one task specific output generated using the first characteristic information, and at least one GT (Ground Truth) corresponding thereto (iv-2) at least one first error calculated by referring to at least a part of the at least one first_2 error obtained by referring to (iv-2) at least one calculated by referring to the learning data and the modulated learning data performing, by the test apparatus, a process of acquiring test data while performing the process of learning the modulation network so as to maximize a second error; and
(b) the test device inputs the test data to the modulation network and causes the modulation network to modulate the test data using the learned parameters of the modulation network to conceal the modulated test data. How to output data. 12. The method of claim 11,
In step (a),
wherein the learning network comprises a first learning network to an n-th learning network having at least one first learned parameter to at least one n-th learned parameter, wherein when n is an integer greater than or equal to 1,
The learning device, (i) by inputting the modulated training data to each of the first learning network to the n-th learning network, to cause each of the first learning network to the n-th learning network to (i-1) the To apply a network operation corresponding to each network to the modulated training data using the first learned parameter to the n-th learned parameter of each of the first learning network to the n-th learning network (i-2 ) a process of outputting each of the first_1 characteristic information to the 1_n first characteristic information for the modulated training data, (ii) inputting the training data into each of the first learning network to the n-th learning network to each of the Let the first learning network to the n-th learning network (ii-1) use the first learned parameter to the n-th learned parameter of each of the first learning network to the n-th learning network for the training data (ii-2) a process of outputting each of 2_1 characteristic information to 2_n characteristic information for the learning data by applying a network operation corresponding to each network; (iii) (iii-1) (iii-) 1a) the average of (1_1)_1th error obtained with reference to the 1_1 characteristic information and the 2_1 characteristic information to (1_1)_nth error obtained with reference to the 1_n characteristic information and the 2_n characteristic information The first_1 error and (iii-1b) the (1_2)_1th error obtained by referring to at least one first task specific output generated using the first_1 characteristic information and at least one first GT corresponding thereto to at least a portion of the first_2 errors that are the average of the (1_2)_nth errors obtained by referring to at least one nth task specific output generated using the 1_nth characteristic information and at least one nth GT corresponding thereto Minimize the first error calculated by referring to (iii-2) the training data and A method of performing a process of learning the modulation network to maximize the second error calculated with reference to the modulated training data. 12. The method of claim 11,
In step (a),
The learning apparatus (i) obtains, as the second error, at least one modulated training data score corresponding to the modulated training data input to a discriminator for determining whether the input data is real or fake, (ii) ) while learning the modulation network to minimize the first error and maximize the second error, (iii) at least one transform corresponding to the transform training data input to the discriminator or the modulated transform training data, respectively learning the discriminator so as to maximize a training data score or at least one modulated transformed training data score and minimize the modulated training data score, wherein the transformed training data or the modulated transformed training data are respectively the training data or the A method that is data generated by transforming the modulated training data. 14. The method of claim 13,
In step (a),
wherein the learning network comprises a first learning network to an n-th learning network having at least one first learned parameter to at least one n-th learned parameter, wherein when n is an integer greater than or equal to 1,
The learning device, (i) by inputting the modulated training data to each of the first learning network to the n-th learning network, to cause each of the first learning network to the n-th learning network to (i-1) the To apply a network operation corresponding to each network to the modulated training data using the first learned parameter to the n-th learned parameter of each of the first learning network to the n-th learning network (i-2 ) a process of outputting each of the first_1 characteristic information to the 1_n first characteristic information for the modulated training data, (ii) inputting the training data into each of the first learning network to the n-th learning network to each of the Let the first learning network to the n-th learning network (ii-1) use the first learned parameter to the n-th learned parameter of each of the first learning network to the n-th learning network for the training data (ii-2) a process of outputting each of 2_1 characteristic information to 2_n characteristic information for the learning data by applying a network operation corresponding to each network; (iii) (iii-1) (iii-) 1a) an average of (1_1)_1th error obtained with reference to the 1_1 characteristic information and the 2_1 characteristic information to (1_1)_nth error obtained with reference to the 1_n characteristic information and the 2_n characteristic information The first_1 error and (iii-1b) the (1_2)_1th error obtained by referring to at least one first task specific output generated using the first_1 characteristic information and at least one first GT corresponding thereto to at least a portion of the first_2 errors that are the average of the (1_2)_nth errors obtained by referring to at least one nth task specific output generated using the 1_nth characteristic information and at least one nth GT corresponding thereto (iii-2) to minimize the first error calculated by referring to a process of learning the modulation network so as to maximize the second error, which is the score of the transformed training data corresponding to the modulated training data, and (iv) the transformed training data or the modulated transform training data input to the discriminator. A method of performing a process of learning the discriminator so as to maximize the transformed learning data score or the modulated transformed learning data score and minimize the transformed learning data score respectively corresponding to . In the learning device for learning a modulation network (Obfuscation Network) for concealing original data for personal information protection,
at least one memory for storing instructions; and
(I) a process of inputting the training data into a modulation network when training data is obtained, causing the modulation network to modulate the training data to generate modulated training data; (II) (i) input the modulated training data into a learning network having at least one learned parameter to cause the learning network to (i-1) use the learned parameter to perform a network operation on the modulated training data operation is applied to (i-2) generate first characteristic information corresponding to the modulated training data, and (ii) input the training data into the learning network to cause the learning network to ( ii-1) a process of applying a network operation to the training data using the learned parameter (ii-2) generating second characteristic information corresponding to the training data; and (III) (i) (i-1) at least one first_1 error obtained with reference to the first characteristic information and the second characteristic information, and (i-2) generated using the first characteristic information At least one first error calculated by referring to at least a portion of the at least one first_2 error obtained by referring to the task specific output and at least one GT (Ground Truth) corresponding thereto is minimized ( ii) a process of learning the modulation network to maximize at least one second error calculated with reference to the training data and the modulated training data; at least one processor;
A learning device comprising a. 16. The method of claim 15,
wherein the learning network comprises a first learning network to an n-th learning network having at least one first learned parameter to at least one n-th learned parameter, wherein when n is an integer greater than or equal to 1,
In the process (II) above,
The processor, (i) by inputting the modulated training data to each of the first learning network to the n-th learning network to cause each of the first learning network to the n-th learning network to (i-1) the first (i-2) to apply a network operation corresponding to each network to the modulated training data using the first learned parameter to the n-th learned parameter of each of the learning network and the n-th learning network (i-2) Each of the 1_1 characteristic information to 1_n characteristic information for the modulated learning data is output, and (ii) the training data is input to each of the first learning network to the n-th learning network to each of the first learning (ii-1) causing the network to the n-th learning network to apply a network operation corresponding to each network to the training data using each of the first learned parameter to the n-th learned parameter (ii) -2) to output each of 2_1 characteristic information to 2_n characteristic information for the learning data,
In the process (III) above,
The processor is configured to: (i) (i-1) a (1_1)_1th error obtained by referring to the first_1 characteristic information and the 2_1 characteristic information to the 1_n characteristic information and the 2_n characteristic information obtained by referring to the second_n characteristic information The first_1 error that is the average of the (1_1)_nth errors, (i-2) at least one first task specific output generated using the first_1 characteristic information, and at least one first GT corresponding thereto (1_2)_1th error obtained with reference to at least one nth task specific output generated using the 1_nth characteristic information and (1_2)_nth error obtained by referring to at least one nth GT corresponding thereto The modulation network to minimize the first error calculated with reference to at least a portion of the first_2 errors that are the average of , and (ii) maximize the second error calculated with reference to the training data and the modulated training data. Learning device to learn. 16. The method of claim 15,
wherein the learning network comprises a first learning network to an n-th learning network having at least one first learned parameter to at least one n-th learned parameter, wherein when n is an integer greater than or equal to 1,
In the process (I) above,
The processor inputs the training data to the modulation network to cause the modulation network to modulate the training data to generate first modulated training data,
In the process (II) above,
The processor is configured to: (i) input the first modulated training data to the first learning network to cause the first learning network to (i-1) use the first learned parameter of the first learning network. By applying a network operation to the first modulated training data (i-2) to output 1_1 characteristic information for the first modulated training data, (ii) the training data to the first learning network input to cause the first learning network to (ii-1) apply a network operation to the training data using the first learned parameter, and (ii-2) output 2_1 characteristic information for the training data. and
In the process (III) above,
The processor is configured to: (i) (i-1) at least one (1_1)_1 error obtained by referring to the first_1 characteristic information and the second_1 characteristic information, and (i-2) the first_1 characteristic information At least one first_1 error obtained by referring to at least a part of at least one first task specific output generated by using the at least one (1_2)_1 error obtained by referring to at least one first GT corresponding thereto and (ii) learning the modulation network to maximize at least one second_1 error calculated with reference to the training data and the first modulated training data, so that the modulation network is a first learned modulation network to become,
The processor increases k, which is an integer, from 2 to n, (i) inputting the training data to the k-1 th learned modulation network so that the k-1 th learned modulation network modulates the training data. to generate the k-th modulated training data, (ii) (ii-1) input the k-th modulated training data to the k-th learning network to cause the k-th learning network to be at least one of the k-th learning networks A network operation is applied to the k-th modulated training data using the k-th learned parameter of to output 1_k characteristic information for the k-th modulated training data, (ii-2) the training data input to the k-th learning network to cause the k-th learning network to apply a network operation to the training data using the k-th learned parameter to output 2_k characteristic information for the training data, (iii) ) (iii-1) (iii-1a) at least one (1_1)_k error obtained by referring to the first_k characteristic information and the second_k characteristic information, and (iii-1b) using the first_k characteristic information at least one (1_2)_k error calculated by referring to at least a part of at least one (1_2)_k error obtained by further referring to at least one kth task specific output generated by (iii-2) by learning the k-1 th learned modulation network to maximize at least one 2_k error calculated with reference to the training data and the k-th modulated training data, 1 A learning apparatus for making the learned modulation network a k-th learned modulation network. 16. The method of claim 15,
In the process (III) above,
A state in which at least one obfuscated training data score corresponding to the modulated training data input to a discriminator for determining whether the input data is real or fake is obtained as the second error In , the processor (i) learns the modulation network to minimize the first error and maximize the second error, and (ii) transform learning data or modulated transform learning data input to the discriminator. Learning the discriminator so as to maximize at least one transform learning data score or at least one modulated transform training data score corresponding to and minimize the modulated training data score, the transform training data or the modulated transform training data is a learning device, each of which is data obtained by converting the learning data or the modulated learning data. 19. The method of claim 18,
wherein the learning network comprises a first learning network to an n-th learning network having at least one first learned parameter to at least one n-th learned parameter, wherein when n is an integer greater than or equal to 1,
In the process (II) above,
The processor, (i) by inputting the modulated training data to each of the first learning network to the n-th learning network to cause each of the first learning network to the n-th learning network to (i-1) the To apply a network operation corresponding to each network to the modulated training data using the first learned parameter to the n-th learned parameter of each of the first learning network to the n-th learning network (i-2) to output each of 1_1 characteristic information to 1_n characteristic information for the modulated training data, and (ii) input the training data into each of the first learning network to the n-th learning network to each of the first (ii-1) causing the learning network to the n-th learning network to apply a network operation corresponding to each network to the training data using each of the first learned parameter to the n-th learned parameter ( ii-2) to output each of 2_1 characteristic information to 2_n characteristic information for the learning data,
In the process (III) above,
The processor is configured to: (i) (i-1) (1_1)_1th error obtained by referring to the first_1 characteristic information and the second_1 characteristic information to the 1_n characteristic information and the 2_n characteristic information obtained by referring to the second_n characteristic information The first_1 error, which is an average of one (1_1)_nth error, (i-2) at least one first task specific output generated using the first_1 characteristic information, and at least one first GT corresponding thereto The average of the (1_2)_1th error obtained with reference to at least one nth task specific output generated using the 1_n characteristic information and the (1_2)_nth error obtained by referring to the nth GT corresponding thereto The modulation network to minimize the first error calculated by referring to at least some of the first_2 errors and to maximize the second error that is the modulated training data score corresponding to the modulated training data input to the discriminator. (ii) maximize the transformed learning data score or the modulated transform learning data score corresponding to the transformed learning data or the modulated transform learning data input to the discriminator, and minimize the modulated learning data score A learning apparatus for learning the discriminator so as to 19. The method of claim 18,
The learning network includes a first learning network to an n-th learning network each having a first learned parameter to an n-th learned parameter, and when n is an integer greater than or equal to 1,
In the process (I) above,
The processor inputs the training data to the modulation network to cause the modulation network to modulate the training data to generate first modulated training data,
In the process (II) above,
The processor is configured to: (i) input the first modulated training data to the first learning network to cause the first learning network to (i-1) use the first learned parameter of the first learning network. By applying a network operation to the first modulated training data (i-2) to output 1_1 characteristic information for the first modulated training data, (ii) the training data to the first learning network input to cause the first learning network to (ii-1) apply a network operation to the training data using the first learned parameter, and (ii-2) output 2_1 characteristic information for the training data. and
In the process (III) above,
The processor is configured to: (i) (i-1) at least one (1_1)_1 error obtained by referring to the first_1 characteristic information and the second_1 characteristic information, and (i-2) (i-2a) the Obtained by referring to at least a part of at least one first task specific output generated using the first_1 characteristic information and (i-2b) the (1_2)_1th error obtained by referring to at least one first GT corresponding thereto to minimize at least one first_1 error and (ii) maximize at least one second_1 error that is at least one first modulated training data score corresponding to the first modulated training data input to the discriminator, learning a modulation network so that the modulation network becomes a first learned modulation network, and the processor, (i) at least one second corresponding to the transformation learning data input to the discriminator or the modulated transformation learning data maximizing one transformed training data score or at least one first modulated transformed training data score and (ii) minimizing the first modulated training data score, learning the discriminator so that the discriminator is a first learned discriminator and,
The processor increases k, which is an integer, from 2 to n, (i) inputting the training data to the k-1 th learned modulation network so that the k-1 th learned modulation network modulates the training data. to generate the k-th modulated training data, (ii) (ii-1) input the k-th modulated training data to the k-th learning network to cause the k-th learning network to be at least one of the k-th learning networks A network operation is applied to the k-th modulated training data using the k-th learned parameter of to output 1_k characteristic information for the k-th modulated training data, (ii-2) the training data input to the k-th learning network to cause the k-th learning network to apply a network operation to the training data using the k-th learned parameter to output 2_k characteristic information for the training data, (iii) ) (iii-1) at least one (1_1)_k error obtained by referring to the first_k characteristic information and the second_k characteristic information, and (iii-2) at least one generated using the first_k characteristic information At least one 1_k error calculated by referring to at least a part of at least one (1_2)_k error obtained by referring to the k-th task specific output of , and at least one k-th GT corresponding thereto, is minimized, -1 By learning the k-1 th learned modulation network to maximize at least one 2_k error that is at least one k th modulated training data score corresponding to the k th modulated training data input to the learned discriminator , such that the k-1 th learned modulation network becomes a k-th learned modulation network, and (iv) at least corresponding to the transformation learning data input to the k-1 th learned discriminator or the modulated transformation learning data. Maximizing one kth transform training data score or at least one kth modulated transform training data score, and maximizing the kth modulated training data score A learning apparatus for learning the k-1 th learned discriminator so as to digest the discriminator so that the k-1 th learned discriminator becomes a k th discriminator. 19. The method of claim 18,
The maximum value of the transformation learning data score or the modulated transformation learning data score corresponding to the transformation learning data or the modulated transformation learning data input to the discriminator is the transformation learning data or the modulated transformation learning data, respectively. 1, which is determined to be , and the minimum value of the modulated training data score corresponding to the modulated training data input to the discriminator is 0, which determines that the modulated training data is fake. 16. The method of claim 15,
In the process (III) above,
The processor is configured to: calculate the first error with reference to at least a portion of (i) a difference between the first characteristic information and the second characteristic information, and (ii) at least one difference between the task specific output and the GT. A learning apparatus for obtaining the second error by at least one difference between the learning data and the modulated learning data. 23. The method of claim 22,
The processor is configured to obtain the first error with reference to a norm or a cosine similarity between the first characteristic information and the second characteristic information. 16. The method of claim 15,
In the process (III) above,
The processor measures at least one quality (Quality) with reference to at least a part of entropy and noise degree of the modulated learning data, and obtains the first error with reference to the measured quality. In a test device for learning a tampering network (Obfuscation Network) that conceals original data for personal information protection,
at least one memory for storing instructions; and
(I) a process in which a learning device (i) acquires training data and inputs it to a modulation network to cause the modulation network to modulate the training data to generate modulated training data, (ii) the modulated learning Input data to a learning network having at least one learned parameter to cause the learning network to (ii-1) apply a network operation to the modulated training data using the learned parameter (ii-2) a process of generating first characteristic information corresponding to the modulated training data, (iii) inputting the training data into the learning network to cause the learning network to (iii-1) learn the learning using the learned parameters (iii-2) a process for generating second characteristic information corresponding to the learning data by applying a network operation to data; and (iv) (iv-1) (iv-1a) the first characteristic information and the At least one first_1 error obtained with reference to the second characteristic information, (iv-1b) at least one task specific output generated using the first characteristic information, and at least one GT (Ground Truth) corresponding thereto (iv-2) at least one first error calculated by referring to at least a part of the at least one first_2 error obtained by referring to (iv-2) at least one calculated by referring to the learning data and the modulated learning data a process of performing a process of acquiring test data while performing a process of learning the modulation network so as to maximize a second error; and (II) inputting the test data into the modulation network to cause the modulation network to modulate the test data using the learned parameters of the modulation network, and output the modulated test data as concealed test data. at least one processor configured to execute the instructions for performing a process or cause another device to execute;
test device comprising 26. The method of claim 25,
In the process (I) above,
wherein the learning network comprises a first learning network to an n-th learning network having at least one first learned parameter to at least one n-th learned parameter, wherein when n is an integer greater than or equal to 1,
The learning device, (i) by inputting the modulated training data to each of the first learning network to the n-th learning network, to cause each of the first learning network to the n-th learning network to (i-1) the To apply a network operation corresponding to each network to the modulated training data using the first learned parameter to the n-th learned parameter of each of the first learning network to the n-th learning network (i-2 ) a process of outputting each of the first_1 characteristic information to the 1_n first characteristic information for the modulated training data, (ii) inputting the training data into each of the first learning network to the n-th learning network to each of the Let the first learning network to the n-th learning network (ii-1) use the first learned parameter to the n-th learned parameter of each of the first learning network to the n-th learning network for the training data (ii-2) a process of outputting each of 2_1 characteristic information to 2_n characteristic information for the learning data by applying a network operation corresponding to each network; (iii) (iii-1) (iii-) 1a) the average of (1_1)_1th error obtained with reference to the 1_1 characteristic information and the 2_1 characteristic information to (1_1)_nth error obtained with reference to the 1_n characteristic information and the 2_n characteristic information The first_1 error and (iii-1b) the (1_2)_1th error obtained by referring to at least one first task specific output generated using the first_1 characteristic information and at least one first GT corresponding thereto to at least a portion of the first_2 errors that are the average of the (1_2)_nth errors obtained by referring to at least one nth task specific output generated using the 1_nth characteristic information and at least one nth GT corresponding thereto Minimize the first error calculated by referring to (iii-2) the training data and A test apparatus for performing a process of learning the modulation network so as to maximize the second error calculated with reference to the modulated training data. 26. The method of claim 25,
In the process (I) above,
The learning apparatus (i) obtains, as the second error, at least one modulated training data score corresponding to the modulated training data input to a discriminator for determining whether the input data is real or fake, (ii) ) while learning the modulation network to minimize the first error and maximize the second error, (iii) at least one transform corresponding to the transform training data input to the discriminator or the modulated transform training data, respectively learning the discriminator so as to maximize a training data score or at least one modulated transformed training data score and minimize the modulated training data score, wherein the transformed training data or the modulated transformed training data are respectively the training data or the A test device that is data generated by transforming the modulated learning data. 28. The method of claim 27,
In the process (I) above,
wherein the learning network comprises a first learning network to an n-th learning network having at least one first learned parameter to at least one n-th learned parameter, wherein when n is an integer greater than or equal to 1,
The learning device, (i) by inputting the modulated training data to each of the first learning network to the n-th learning network, to cause each of the first learning network to the n-th learning network to (i-1) the To apply a network operation corresponding to each network to the modulated training data using the first learned parameter to the n-th learned parameter of each of the first learning network to the n-th learning network (i-2 ) a process of outputting each of the first_1 characteristic information to the 1_n first characteristic information for the modulated training data, (ii) inputting the training data into each of the first learning network to the n-th learning network to each of the Let the first learning network to the n-th learning network (ii-1) use the first learned parameter to the n-th learned parameter of each of the first learning network to the n-th learning network for the training data (ii-2) a process of outputting each of 2_1 characteristic information to 2_n characteristic information for the learning data by applying a network operation corresponding to each network; (iii) (iii-1) (iii-) 1a) an average of (1_1)_1th error obtained with reference to the 1_1 characteristic information and the 2_1 characteristic information to (1_1)_nth error obtained with reference to the 1_n characteristic information and the 2_n characteristic information The first_1 error and (iii-1b) the (1_2)_1th error obtained by referring to at least one first task specific output generated using the first_1 characteristic information and at least one first GT corresponding thereto to at least a portion of the first_2 errors that are the average of the (1_2)_nth errors obtained by referring to at least one nth task specific output generated using the 1_nth characteristic information and at least one nth GT corresponding thereto (iii-2) to minimize the first error calculated by referring to a process of learning the modulation network so as to maximize the second error, which is the score of the transformed training data corresponding to the modulated training data, and (iv) the transformed training data or the modulated transform training data input to the discriminator. A test apparatus for performing a process of learning the discriminator so as to maximize the transform learning data score or the modulated transform learning data score corresponding to each and minimize the transform learning data score.

Images