KR102531368B1 - Method for training and testing obfuscation network capable of performing distinct concealing processes for distinct parts of original medical image, and training device and testing device using the same - Google Patents

Abstract

The present invention creates a modulated medical image by concealing the original medical image by mixing characteristic concealing processes corresponding to each region of the original medical image, so that a human recognizes the modulated medical image as different from the original medical image and learns It is to enable the network to recognize the modulated medical image as the original medical image.

Description

A method for learning and testing a modulation network that conceals an original medical image by allowing different concealing processes to be performed for each area of the original medical image, and a learning device and test device using the same {METHOD FOR TRAINING AND TESTING OBFUSCATION NETWORK CAPABLE OF PERFORMING DISTINCT CONCEALING PROCESSES FOR DISTINCT PARTS OF ORIGINAL MEDICAL IMAGE, AND TRAINING DEVICE AND TESTING DEVICE USING THE SAME}

The present invention relates to an obfuscation network for concealing an original medical image, and more particularly, to an obfuscation network for concealing an original medical image in an area including medical information used to determine a medical condition and other areas. A method for learning and testing a modulation network that conceals an original medical image by applying a sealing process, and a learning device and a test device using the same.

Big data refers to standardized data used in existing corporate environments or public institutions, as well as e-commerce data, meta data, web log data, radio frequency identification (RFID) data, sensor network data, Data that includes all unstructured or semi-structured data that has not been previously utilized, such as social network data, Internet text and document data, and Internet search indexing data. It is called big data in the sense that it has an amount of data that is difficult to deal with.

Such big data may not be meaningful by itself, but it can be usefully used to create new data or make judgments or predictions in various fields through machine learning on patterns.

As a representative example, medical big data, which converts medical information on patients generated by each medical institution into big data, is analyzed and refined, and then companies such as medical institutions, research institutes, and pharmaceutical companies conduct research, or companies such as the government and insurance companies use statistics. It is expected to contribute greatly to the creation and analysis of

However, although many fields are making rapid progress through open innovation based on big data analysis, the data of the medical industry contains a lot of sensitive data, so there have been many difficulties in external cooperation using the data. In recent years, with the passage of the 3 data laws, expectations for the use of big data are rising, but it can be said that the demand for anonymization has grown proportionately.

A high-level anonymization method that goes beyond simply deleting patients' personal information is expected to further enhance the utilization value of medical big data. Medical images that can be treated as personal information: patient observation videos, affected area photos, before and after plastic surgery photos, X-Ray, ultrasound, CT scan photos, PET (positron emission tomography: Positron Emission Tomography), f-MRI (functional magnetism) Resonance Imaging (Functional Magnetic Resonance Imaging), etc., can be personal image information that can identify an individual in a specific situation because many complex situations occur. In fact, overseas, there have been cases of identifying individuals using leaked X-rays. Therefore, there is a need for a method to safely use medical images in a form in which personal information and copyright are protected.

In particular, recently, various types of concealing processing methods have been proposed to prevent leakage of identification information capable of personal identification by irreversibly converting information included in original medical data.

As an example, a method of erasing or hiding all objects included in the original medical data by masking or blurring has the advantage of relatively simple concealing, but all objects included in the original medical data In case of concealing, there is a disadvantage in that the useful value as medical big data is reduced because all the meaningful data necessary to determine the name of the disease are also concealed.

As another example, a method of determining a region including identification information among information included in the original medical data and concealing the corresponding region is advantageous for data utilization by leaving the other parts as they are except for the region including the identification information, but including the identification information There is a limitation that it is vulnerable to leakage of personal information if omission occurs in the process of determining the area.

As another method, as disclosed in Patent Publication No. 10-2020-0014396, there is a method of irreversibly processing the other areas while leaving only important areas necessary for data analysis, such as disease determination, on the original medical data. Specifically, the medical image concealing system described in the corresponding published patent separates the facial region from the medical image including the patient's facial region, and performs concealing on the remaining facial region except for the region required for analysis.

Such concealing technology may be more suitable for preventing leakage of personal information, but since areas that are not concealed can be recognized by the human eye, if the area contains important identification information for personal identification, There is still a limitation that it cannot be completely safe against leakage of personal information.

Therefore, an improvement plan for solving the above problems is required.

The object of the present invention is to solve all of the above problems.

In addition, an object of the present invention is to minimize the risk of personal information exposure while being relatively simple by applying different concealing processes for each area of an original medical image instead of a separate task of finding personal identification information included in an original medical image.

Another object of the present invention is to prevent problems related to security or privacy that may occur by restoring an original medical image by generating data obtained by irreversibly concealing the original medical image.

In addition, the present invention applies a sufficient concealing processor to the original medical image while preserving important features that the machine learning network can interpret, so that it is impossible for humans to infer the original medical image from the modulated medical image, but the machine learning network Another object is to make it possible to generate a recognition result identical or similar to an original medical image from a modulated medical image.

In addition, another object of the present invention is to further activate the big data trading market.

In order to achieve the object of the present invention as described above and realize the characteristic effects of the present invention described later, the characteristic configuration of the present invention is as follows.

According to one aspect of the present invention, in a method for learning a modulation network for concealing an original medical image by performing different concealing processes for each region of the original medical image, (a) the learning device comprises a medical image for learning is input to a modulation network to cause the modulation network to perform a first concealing process and a second concealing process respectively corresponding to the first learning area and the second learning area of the learning medical image, respectively, to the first learning area and the learning area. generating a modulated medical image for learning corresponding to the medical image for learning by performing the operation in a second area; (b) the learning device comprises: (i) a first discriminator for determining the first reference medical image on which the first concealing process is performed over the entire area of the training medical image as genuine, and the entire area of the training medical image The modulated medical image for learning is input to each of the second discriminators that determine the second reference medical image that has undergone the second concealing process as genuine, and causes the first discriminator and the second discriminator to determine the modulated medical image for learning. A process of outputting a first modulated medical image score to a second modulated medical image score that determines whether an image is real or fake, and (ii) inputting the modulated medical image for learning into an image identification network, so that the image identification network , performing a process of performing a running operation on the modulated medical image for learning and outputting medical feature information for learning corresponding to the modulated medical image for learning; and (c) the learning device maximizes a sum loss obtained by adding a weighted sum of a 1_1 discrimination loss and a 2_1 discrimination loss generated by referring to the first modulated medical image score and the second modulated medical image score, and Training the modulation network to minimize an accuracy loss generated by referring to the medical characteristic information for learning of the identification network or a task-specific output generated using the medical characteristic information for learning and a ground truth corresponding to the medical image for training ; A method including is disclosed.

As an example, (d) the learning device determines the 1_1 discrimination loss generated by the first discriminator and the second discriminator by referring to the first modulated medical image score and the second modulated medical image score, respectively; 2_1 discrimination loss is minimized, and a first reference medical image score and a second reference medical image corresponding to the first reference medical image and the second reference medical image respectively input to the first discriminator and the second discriminator, respectively. training each of the first discriminator and the second discriminator to maximize a 1_2 discriminant loss and a 2_2 discriminant loss generated by referring to an image score; A method further comprising is disclosed.

As an example, (e) training, by the learning device, the image identification network to minimize the accuracy loss; A method further comprising is disclosed.

As an example, a method is disclosed wherein the learning device is in a state in which the image recognition network is trained to recognize the medical image for learning.

As an example, when the learning device proceeds with learning of the modulation network using the summation loss, (i) for the first area for learning, the 1_1 discrimination loss corresponding to the first area for learning is applied to the learning area. Learning is performed by applying a weight higher than the 2_1 discrimination loss corresponding to the second area, and (ii) for the second area for learning, the second learning loss corresponds to the 2_1 discriminant loss corresponding to the second area A method characterized in that learning is performed by applying a weight higher than the 1_1 determination loss corresponding to region 1 is disclosed.

As an example, the learning device causes the modulation network to conceal the first area for learning to have a relatively low noise density according to the degree of concealment through the first concealing process, and Disclosed is a method characterized by subjecting the second area for learning to a concealing process to have a relatively high noise density according to the degree of concealing through a concealing process.

As an example, the learning device sets a region including medical information used to determine a medical condition among regions of the medical image for learning as the first region for learning, and sets the first region for learning in the medical image for learning. A method characterized by setting the excluded area as the second area for learning is disclosed.

As an example, the learning device obtains information of the first region for learning and information of the second region for learning of the medical image for learning from labeling information for learning in which a first region for learning and a second region for learning are set in the medical image for learning. and input to the modulation network.

As an example, the learning device inputs the medical image for learning into the modulation network, causes the modulation network to perform a convolution operation on the medical image for learning at least once through an encoder, and generates a feature map for learning through a decoder. Disclosed is a method characterized by outputting the modulated medical image for learning by performing a deconvolution operation on the feature map for learning at least once.

According to another aspect of the present invention, in the method for testing a modulation network for concealing an original medical image by performing different concealing processes for each region of the original medical image, (a) by a learning device, (i ) A medical image for learning is input to a modulation network, and the modulation network performs a first concealing process for learning and a second concealing process for learning corresponding to the first area for learning and the second area for learning, respectively, of the medical image for learning. To generate a modulated medical image for learning corresponding to the medical image for learning by performing the first area and the second area for learning, (ii) (ii-1) the first control for learning in the entire area of the medical image for learning A first discriminator for determining the first reference medical image subjected to the sealing process as genuine, and a second discriminator for determining the second reference medical image subjected to the second concealing process for learning to be genuine over the entire area of the learning medical image. The modulated medical image for learning is input to each of the learning groups, and the first discriminator and the second discriminator determine whether the modulated medical image for learning is real or fake, and the first modulated medical image score to the second modulated medical image score are determined. and (ii-2) inputting the modulated medical image for learning to an image identification network, which causes the image identification network to perform a running operation on the modulated medical image for learning, and (ii-2) a medical training corresponding to the modulated medical image for training. A process of outputting feature information is performed, and (iii) a sum loss obtained by adding a weighted sum of a 1_1 discrimination loss and a 2_1 discrimination loss generated by referring to the first modulated medical image score and the second modulated medical image score Modulation to maximize and minimize accuracy loss generated by referring to the medical feature information for learning or the task specific output generated using the medical feature information for learning of the image identification network and the ground truth corresponding to the medical image for learning obtaining, by a test device, a medical image for a test in a state in which the network has been trained; and (b) the test device inputs the medical image for test to the modulation network, and causes the modulation network to generate first areas for testing corresponding to the first area for testing and the second area for testing, respectively, of the medical image for testing. generating a modulated medical image for test corresponding to the medical image for test by performing a concealing process and a second concealing process for test on the first area for testing and the second area for testing, respectively; A method including is disclosed.

As an example, in step (a),

The first discriminator and the second discriminator respectively minimize the 1_1 discriminant loss and the 2_1 discriminator loss by the learning device, respectively, and the first discriminator input to the first discriminator and the second discriminator, respectively. The first and second reference medical image scores corresponding to the reference medical image and the second reference medical image are respectively learned to maximize the first_2 discriminant loss and the second_2 discriminant loss generated by referring to the score. A method of characterization is disclosed.

As an example, the method disclosed herein is characterized in that the image identification network is in a state where it is learned by the learning device to minimize the accuracy loss or to recognize the medical image for training.

As an example, the test device causes the modulation network to conceal the first area for testing to have a relatively low noise density according to the degree of concealment through the first concealing process for testing, Disclosed is a method characterized by subjecting the second area for test to a concealing process to have a relatively high noise density according to the degree of concealment through the second concealing process for test.

As an example, the test device sets an area including medical information used to determine a medical state among areas of the medical image for test as the first area for testing, and in the medical image for test, the area including medical information used for determining the medical condition. A method characterized by setting an area other than the first area as the second area for the test is disclosed.

As an example, the test device may obtain information on the first area for testing of the medical image for testing and the test from labeling information for testing in which a first area for testing and a second area for testing are set in the medical image for testing. Disclosed is a method characterized in that the second area information for use is obtained and inputted to the modulation network.

According to another aspect of the present invention, a learning apparatus for learning a modulation network for concealing an original medical image by performing different concealing processes for each region of the original medical image, comprising: at least one memory for storing instructions; and at least one processor configured to execute the instructions, wherein the processor (I) inputs the medical image for training to a modulation network to cause the modulation network to generate a first area for learning and a first area for learning of the medical image for training. A process of generating a modulated medical image for learning corresponding to the medical image for training by performing a first concealing process and a second concealing process corresponding to each of the two areas on the first area for learning and the second area for learning, respectively , (II) (i) a first discriminator for determining the first reference medical image on which the first concealing process is performed over the entire area of the training medical image as genuine, and the second concealment over the entire area of the training medical image The modulated medical image for learning is input to each of the second discriminators for determining whether the second reference medical image that has undergone the sealing process is genuine, and the first discriminator and the second discriminator determine whether the modulated medical image for learning is genuine. A sub-process for outputting a first modulated medical image score to a second modulated medical image score determined to be fake, and (ii) inputting the modulated medical image for learning into an image identification network, which causes the image identification network to output the score for learning a process of performing a sub-process of running a learning operation on a modulated medical image and outputting medical feature information for learning corresponding to the modulated medical image for learning; and (III) maximizing a weighted summation loss of a 1_1 discrimination loss and a 2_1 discrimination loss generated by referring to the first modulated medical image score and the second modulated medical image score, for the training of the image identification network. a process of training the modulation network to minimize an accuracy loss generated by referring to medical characteristic information or a task-specific output generated using the medical characteristic information for learning and a ground truth corresponding to the medical image for training; A learning device that performs is disclosed.

As an example, (IV) the processor determines the 1_1 discrimination loss generated by the first discriminator and the second discriminator by referring to the first modulated medical image score and the second modulated medical image score, respectively, and the first modulated medical image score, and the second discriminator. A first reference medical image score and a second reference medical image corresponding to the first reference medical image and the second reference medical image respectively input to the first discriminator and the second discriminator, respectively, while minimizing discrimination loss of 2_1 a process of learning each of the first discriminator and the second discriminator to maximize a 1_2 discriminant loss and a 2_2 discriminator loss, respectively, generated by referring to scores; A learning device further comprising a is disclosed.

As an example, (V) the process of causing the processor to train the image identification network to minimize the accuracy loss; A learning device further comprising a is disclosed.

As an example, the method is disclosed in which the processor is in a state in which the image identification network is trained to recognize the medical image for training.

As an example, when the processor learns the modulation network using the summation loss, (i) for the first area for learning, the 1_1 discrimination loss corresponding to the first area for learning is applied to the first area for learning. Learning is performed by applying a weight higher than the 2_1 discrimination loss corresponding to area 2, and (ii) for the second area for learning, the 2_1 discrimination loss corresponding to the second area for learning is applied to the first area for learning. A learning apparatus characterized in that learning is performed by applying a weight higher than the 1_1 determination loss corresponding to a region.

As an example, the processor causes the modulation network to conceal the first area for learning to have a relatively low noise density according to the degree of concealment through the first concealing process, A learning device characterized by subjecting the second area for learning to a concealing process to have a relatively high noise density according to a degree of concealment through a sealing process is disclosed.

As an example, the processor sets a region including medical information used to determine a medical state among regions of the medical image for learning as the first region for learning, and excludes the first region for learning from the medical image for learning. A learning device characterized by setting an area as the second area for learning is disclosed.

As an example, the processor obtains information of the first region for learning and information of the second region for learning of the medical image for learning from labeling information for learning in which a first region for learning and a second region for learning are set in the medical image for learning, A learning device characterized in that input to the modulation network is disclosed.

As an example, the processor inputs the medical image for learning to the modulation network, causes the modulation network to perform a convolution operation on the medical image for learning at least once through an encoder, and generates a feature map for learning through a decoder. A learning device characterized in that the learning feature map is deconvolved at least once to output the modulated medical image for learning.

According to another aspect of the present invention, a test device for testing a modulation network that conceals an original medical image by performing different concealing processes for each region of the original medical image, comprising: at least one memory for storing instructions; and at least one processor configured to execute the instructions, wherein the processor, by (I) the learning device, (i) inputs the medical image for training to a modulation network to cause the modulation network to perform the medical image for training. Corresponding to the medical image for learning by performing a first concealing process for learning and a second concealing process for learning corresponding to the first area for learning and the second area for learning, respectively, on the first area for learning and the second area for learning, respectively. (ii) (ii-1) a first discriminator for determining that the first reference medical image obtained by performing the first concealing process for learning over the entire area of the medical image for training is genuine; The modulated medical image for learning is input to each of the second discriminators for determining that the second reference medical image, which has undergone the second concealing process for learning in the entire area of the medical image for learning, is genuine, and the first discriminator and the first discriminator 2 a process of causing a discriminator to output a first modulated medical image score to a second modulated medical image score for determining whether the modulated medical image for learning is real or fake, and (ii-2) image identification of the modulated medical image for training network, and performs a process of causing the image identification network to perform a learning operation on the modulated medical image for training and output medical feature information for learning corresponding to the modulated medical image for learning, (iii) the first modulated medical image; Maximizes a weighted summation loss of the 1_1 discrimination loss and the 2_1 discrimination loss generated by referring to the image score and the second modulated medical image score, and the medical feature information for learning or the medical feature information for learning of the image identification network a process of acquiring a medical image for test in a state in which the modulation network is trained to minimize an accuracy loss generated by referring to a task-specific output generated using , and a ground truth corresponding to the medical image for training; and (II) inputting the medical image for test to the modulation network to cause the modulation network to conduct a first concealing process and a test corresponding to the first area for testing and the second area for testing, respectively, of the medical image for testing. a process of generating a modulated medical image for test corresponding to the medical image for test by performing a second concealing process on the first area for test and the second area for test, respectively; A test device that performs is disclosed.

As an example, in the process (I), the first discriminator and the second discriminator respectively minimize the 1_1 determination loss and the 2_1 determination loss by the learning device, respectively, and 1st_2 discrimination loss and 2_2nd discrimination loss generated by referring to the first reference medical image score and the second reference medical image score corresponding to the first reference medical image and the second reference medical image respectively input to the second discriminator. A test device characterized in that it is in a state learned to maximize each loss is disclosed.

As an example, the test device is disclosed, wherein the image identification network is in a state in which the learning device is trained to minimize the accuracy loss or to recognize the medical image for learning.

As an example, the processor causes the modulation network to conceal the first area for testing to have a relatively low noise density according to the degree of concealment through the first concealing process for testing, wherein the Disclosed is a test device characterized by subjecting the second area for test to a concealing process to have a relatively high noise density according to the degree of concealment through a second concealing process for test.

As an example, the processor sets a region including medical information used to determine a medical condition among regions of the medical image for test as the first region for test, and in the medical image for test, the first region for test. Disclosed is a test device characterized in that an area other than area 1 is set as the second area for the test.

As an example, the processor may, from labeling information for testing in which a first area for testing and a second area for testing are set in the medical image for testing, obtain information on the first area for testing and information on the first area for testing in the medical image for testing. A test apparatus is disclosed which acquires second area information and inputs it to the modulation network.

In addition to this, a computer readable recording medium for recording a computer program for executing the method of the present invention is further provided.

The present invention is relatively simple and has an effect of reducing the risk of personal information exposure by applying different concealing processes for each area of the original medical image instead of a separate task of finding personal identification information included in the original medical image.

In addition, the present invention has an effect of preventing problems related to security or privacy that may occur by restoring the original medical image by generating data obtained by irreversibly concealing the original medical image.

In addition, the present invention applies a sufficient concealing processor to the original medical image while preserving important features that the machine learning network can interpret, so that it is impossible for humans to infer the original medical image from the modulated medical image, but the machine learning network It is possible to generate a recognition result identical or similar to the original medical image from the modulated medical image.

In addition, the present invention has the effect of further activating the big data trading market.

The accompanying drawings for use in describing the embodiments of the present invention are only some of the embodiments of the present invention, and to those of ordinary skill in the art (hereinafter referred to as "ordinary technicians"), the invention Other drawings can be obtained based on these drawings without redundant work being done.
1 schematically shows a learning device for learning an obfuscation network for concealing an original medical image by allowing different concealing processes to be performed for each region of the original medical image according to an embodiment of the present invention. did,
2 schematically illustrates a method of learning a modulation network for concealing an original medical image by performing different concealing processes for each region of the original medical image according to an embodiment of the present invention;
3 is a medical image for learning and labeling information for learning used in a method for learning a modulation network that conceals an original medical image by performing different concealing processes for each region of the original medical image according to an embodiment of the present invention. , Illustratively showing the concealing process reference value,
4 shows a modulated medical image for training to which concealing is applied in a method for learning a modulation network that conceals an original medical image by performing different concealing processes for each region of an original medical image according to an embodiment of the present invention. It is illustrated by way of example,
5 is a method of learning a discriminator in a method of learning a modulation network that conceals an original medical image by allowing different concealing processes to be performed for each region of the original medical image according to an embodiment of the present invention. is schematically shown,
6 illustrates an image recognition network in a method of learning a modulation network for concealing an original medical image by performing different concealing processes for each region of the original medical image according to an embodiment of the present invention. It schematically shows how to learn,
7 schematically illustrates a test apparatus for testing a modulation network that conceals an original medical image by performing different concealing processes for each region of the original medical image according to an embodiment of the present invention;
8 schematically illustrates a method of testing a modulation network that conceals an original medical image by performing different concealing processes for each region of the original medical image according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following detailed description of the present invention refers to the accompanying drawings, which illustrate specific embodiments in which the present invention may be practiced in order to make the objects, technical solutions and advantages of the present invention clear. These embodiments are described in sufficient detail to enable a person skilled in the art to practice the present invention.

Also, throughout the description and claims of the present invention, the word "comprise" and variations thereof are not intended to exclude other technical features, additions, components or steps. Other objects, advantages and characteristics of the present invention will appear to those skilled in the art, in part from this description and in part from practice of the invention. The examples and drawings below are provided as examples and are not intended to limit the invention.

Moreover, the present invention covers all possible combinations of the embodiments shown herein. It should be understood that the various embodiments of the present invention are different from each other but are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in one embodiment in another embodiment without departing from the spirit and scope of the invention. Additionally, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the detailed description set forth below is not to be taken in a limiting sense, and the scope of the present invention, if properly described, is limited only by the appended claims, along with all equivalents as claimed by those claims. Like reference numbers in the drawings indicate the same or similar function throughout the various aspects.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily practice the present invention.

1 shows a learning device 1000 for learning an obfuscation network for concealing an original medical image by allowing different concealing processes to be performed for each region of the original medical image according to an embodiment of the present invention. is shown schematically.

Referring to FIG. 1 , the learning apparatus 1000 according to an embodiment of the present invention provides instructions for learning a modulation network that conceals an original medical image by performing different concealing processes for each region of the original medical image. According to the memory 1001 that stores the , and the instructions stored in the memory 1001, different concealing processes are performed for each area of the original medical image to learn a modulation network that conceals the original medical image. processor 1002.

Specifically, learning device 1000 is typically a computing device (e.g., a device that may include computer processors, memory, storage, input and output devices, and other components of conventional computing devices; electronic devices such as routers, switches, etc.) A desired system using a combination of communication devices; electronic information storage systems such as network attached storage (NAS) and storage area networks (SAN)) and computer software (ie, instructions that allow a computing device to function in a particular way) It may be to achieve performance.

In addition, the processor of the computing device may include hardware components such as a micro processing unit (MPU) or a central processing unit (CPU), a cache memory, and a data bus. In addition, the computing device may further include an operating system and a software component of an application performing a specific purpose.

However, it is not excluded that the computing device includes an integrated processor in which a medium for implementing the present invention, a processor, and a memory are integrated.

On the other hand, the learning device 1000 may be a device or a server for performing a concealing process on an original medical image, and may be mounted on a medical device producing an original medical image and a computer device linked therewith, so that the acquired original medical image A concealing process may be performed, but the present invention is not limited thereto.

Using the learning apparatus 1000 according to an embodiment of the present invention configured as described above, a method of learning a modulation network that conceals an original medical image by performing different concealing processes for each region of the original medical image 2 to 6 will be described as follows.

First, FIG. 2 schematically illustrates a method of learning a modulation network that conceals an original medical image by performing different concealing processes for each region of the original medical image according to an embodiment of the present invention.

Referring to FIG. 2 , when a medical image for learning (x) is obtained, the learning device 1000 inputs the acquired medical image for learning (x) to the modulation network 100 to cause the modulation network 100 to be medical for learning. Medical image (x) for learning by performing the first concealing process and the second concealing process corresponding to the first area for learning and the second area for learning, respectively, in the first area for learning and the second area for learning, respectively, of the image (x) A modulated medical image (x') for learning corresponding to may be generated.

At this time, the modulation network 100 may conceal the medical image for learning (x) with the modulated medical image for learning (x') by allowing different concealing processes to be performed for each area of the medical image for learning. For example, modulation The network 100 may generate a modulated medical image (x′) for learning in which a degree of concealment is differently performed for each of the first area for learning and the second area for learning of the medical image (x) for learning. Then, the modulation network 100 simultaneously performs the first concealing process and the second concealing process so that the first area for learning is concealed by the first concealing process and the second area for learning is covered by the second concealing process. It can be concealed by That is, the modulation network 100 mixes the first concealing process and the second concealing process to conceal the medical image (x) for learning, so that the first area for learning in the medical image (x) for learning is covered by the first concealing process. The second area for learning may be concealed by the second concealing process.

To this end, the learning apparatus 1000 sets an area including medical information used to determine a medical condition among areas of the medical image (x) for learning as a first area for learning, and in the medical image for learning, the first area for learning. The area except for is set as the second area for learning, and the modulation network 100 conceals the first area for learning to have a relatively low noise density according to the degree of concealment through the first concealing process, , the second area for learning may be subjected to concealing processing so as to have a relatively high noise density according to the degree of concealing through the second concealing process.

Here, the learning apparatus 1000 provides information on the first area for learning and the second area for learning of the medical image (x) for learning from labeling information for learning in which the first area for learning and the second area for learning are set in the medical image (x) for learning. information can be obtained.

At this time, the labeling information for learning on the first area for learning and the second area for learning in the medical image (x) for learning is generated by the human labeler setting the first area for learning and the second area for learning in the medical image (x) for learning. , It can be generated by segmenting the medical image (x) for learning into a first area for learning and a second area for learning through a machine learning-based segmentation network.

Accordingly, when the learning device 1000 acquires the first area information for learning and the second area information for learning of the medical image (x) for learning, the first area information for learning and the second area information for learning are transmitted to the modulation network 100. input to make the modulation network 100 perform a first concealing process on the first area for learning and a second concealing process on the second area for learning by referring to information on the first area for learning and the information on the second area for learning. can do.

The modulated medical image (x') for learning generated through the above process is recognized differently from the medical image (x) for learning by humans, but may be recognized as the same as or similar to the medical image (x) for training in a learning network.

As an example, FIG. 3 is for training used in a method of training a modulation network 100 that conceals an original medical image by allowing different concealing processes to be performed for each region of the original medical image according to an embodiment of the present invention. Assuming that the medical image, labeling information for learning, and the concealing process reference value are shown as an example, a modulated medical image (x′) for learning to which concealment is applied may be generated as shown in FIG. 4 .

Here, the area indicated by A1 of the modulated medical image (x') for learning in FIG. 4 is determined to contain medical information used to determine the medical condition and is concealed through a first concealing process to have a relatively low noise density. It may be the processed first area for learning, and the area marked A2 may be a second area for learning that is concealed through a second concealing process to have a relatively high noise density with respect to areas other than the first area for learning. .

On the other hand, the modulation network 100, for example, uses an encoder including a plurality of convolution layers for performing a convolution operation on an input medical image (x) for learning and deconvolution of a feature map output from the encoder for learning. It may include, but is not limited to, a decoder including a plurality of deconvolution layers that outputs a modulated medical image (x'), and the input medical image (x) for training is converted into a modulated medical image (x' for training). ) may include various structures of learning networks that modulate.

Next, referring to FIG. 2 again, the learning apparatus 1000 determines the first reference medical image (x + noise_1) obtained by performing the first concealing process on the entire area of the training medical image (x) as real. In each of the first discriminator 200_1 and the second discriminator 200_2 that determines the second reference medical image (x + noise_2) obtained by performing the second concealing process on the entire area of the training medical image (x) as real. The first modulated medical image (x') for learning is input, and each of the first discriminator 200_1 and the second discriminator 200_2 determines whether the modulated medical image (x') for learning is real or fake. A process of outputting the image score and the second modulated medical image score may be performed.

At this time, the maximum value of the image score generated by the discriminator for the input image may be 1, which determines the input image as genuine, and the minimum value of the image score may be 0, which determines the input image as fake.

In addition, the learning device 1000 inputs the modulated medical image (x') for learning into the image identification network 300, and causes the image identification network 300 to perform a learning operation on the modulated medical image (x') for learning. A process of outputting medical feature information for learning corresponding to the modulated medical image (x') may be performed.

At this time, the image identification network 300 may include a machine learning network, and the machine learning network may include k-nearest neighbors, linear regression, logistic regression, and support vector Support Vector Machine (SVM), Decision Tree and Random Forest, Neural Network, Clustering, Visualization and Dimensionality Reduction, Association Rule Learning ( It may include at least one of Association Rule Learning, Deep Belief Network, Reinforcement Learning, and Deep Learning algorithms, but the machine learning network is not limited thereto and may include various learning algorithms. .

Further, the medical feature information for learning may be a feature or logits of the modulated medical image (x') for learning. In addition, the medical feature information for learning includes a feature value related to a predetermined feature in the modulated medical image for training (x'), or a value related to at least one of a vector, a matrix, and coordinates related to the predetermined feature. It can be a logit of For example, it may be a class for determining cancer, a feature related to cancer, for example, a metastasis state of cancer, and coordinates related to a location of cancer.

Subsequently, the learning device 1000 uses the first discriminator 200_1, the second discriminator 200_2, and the respective losses generated from the image identification network 300 to generate the modulation network 100 and the first discriminator. (200_1), the second discriminator 200_2, and the image identification network 300 may each be trained.

First, the learning apparatus 1000 may generate a 1_1 discrimination loss by referring to the first modulated medical image score from the first discriminator 200_1, and may generate a second modulated medical image score from the second discriminator 200_2. The 2_1 determination loss may be generated with reference to , and the sum loss may be generated by weighted summing the 1_1 determination loss and the 2_1 determination loss.

Here, the learning apparatus 1000 may weight-sum the 1_1 determination loss and the 2_1 determination loss corresponding to the weights referring to the first area information for learning and the second area information for learning to obtain the sum loss. Here, the weights referring to the first area information for learning and the second area information for learning may be additionally adjusted according to the performance of the learned modulation network 100 .

In addition, the learning apparatus 1000 includes a task specific output generated by using the medical characteristic information for learning or the medical characteristic information for learning of the image identification network 300 from the image identification network 300 and the ground truth corresponding to the medical image for training. (ground truth) to generate the generated accuracy loss. Here, the task-specific output may be a probability distribution for the modulated medical image (x′) for learning in which medical feature information for learning is mapped by class, and classification information corresponding to the first area for learning on the medical image for learning using the task-specific output. and location information.

Accordingly, the learning device 1000 may train the modulation network 100 to maximize the sum loss and minimize the accuracy loss. Here, when learning the modulation network 100 is performed, overfitting is prevented by learning the modulation network 100 by further including a regularization loss for parameters of the modulation network 100, It can improve generalization performance.

At this time, when the learning device 1000 proceeds with learning of the modulation network 100 using the sum loss, for the first area for learning, the second area for learning corresponds to the 1_1 discrimination loss corresponding to the first area for learning. Learning is performed by applying a weight higher than the 2_1 discrimination loss corresponding to the second area for learning, and for the second area for learning, a weight higher than the 1_1 discrimination loss corresponding to the first area for learning is applied to the 2_1 discriminant loss corresponding to the second area for learning. learning can be applied. That is, the parameters of the modulation network 100 corresponding to the first concealing process for concealing the first area for learning can be updated by backpropagation using a sum loss obtained by applying a high weight to the 1_1 discrimination loss. , and parameters of the modulation network 100 corresponding to the second concealing process for concealing the second region for learning may be updated by backpropagation using a sum loss obtained by applying a high weight to the 2_1 determination loss. Accordingly, the modulated medical image x' for training can be recognized as real by the first discriminator 200_1 and the second discriminator 200_2.

While the modulation network 100 is trained as described above, referring to FIG. 5 , the learning device 1000 provides the first modulated medical image score and the second discriminator 200_1 and the second discriminator 200_2 respectively. The first reference medical image (x) is input to the first discriminator 200_1 and the second discriminator 200_2, respectively, while minimizing the 1_1 discriminant loss and the 2_1 discriminant loss generated by referring to the second modulated medical image score. + noise_1) and the second reference medical image (x + noise_2) to maximize the 1st and 2nd discriminant losses generated by referring to the first reference medical image score and the second reference medical image score, respectively. Each of the discriminator 200_1 and the second discriminator 200_2 may be trained. That is, the learning device 100 trains the first discriminator 200_1 to minimize the 1_1 discriminant loss and maximize the 1_2 discriminant loss, and minimize the 2_1 discriminant loss and maximize the 2_2 discriminant loss. The group 200_2 can be learned. Accordingly, the first reference medical image (x + noise_1) and the second reference medical image (x + noise_2) can be determined to be genuine, and the modulated medical image (x') for training can be determined to be fake.

Also, referring to FIG. 6 , the learning device 1000 may train the image identification network 300 to minimize accuracy loss. Meanwhile, the image identification network 300 may be previously trained using the medical image (x) for learning, unlike learning using accuracy loss.

By this learning method, for each batch in each epoch, the modulation network 100, the first discriminator 200_1, the second discriminator 200_2 and the image identification network 300 can be updated individually.

And, in the state in which the modulation network 100 has been learned as described above, the test device inputs the medical image for testing to the modulation network 100 where learning has been completed, and modulates the test medical image so that the human It is prevented from being recognized as a medical image for test, and the image identification network 300 can generate a modulated medical image for test that outputs the same or similar recognition result as the medical image for test.

Accordingly, FIG. 7 is a test device 2000 for testing the modulation network 100 that conceals the original medical image by performing different concealing processes for each region of the original medical image according to an embodiment of the present invention. ) is schematically shown.

Referring to FIG. 7 , the test device 2000 according to an embodiment of the present invention performs different concealing processes for each area of the original medical image to test the modulation network 100 that conceals the original medical image. A memory 1001 that stores instructions to perform a different concealing process for each region of the original medical image according to the instructions stored in the memory 1001, and the modulation network 100 that conceals the original medical image. It may include a processor 1002 that performs an operation to test.

Specifically, the test device 2000 is typically a computing device (e.g., a device that may include computer processors, memory, storage, input and output devices, and other components of conventional computing devices; electronic devices such as routers, switches, and the like). A desired system using a combination of communication devices; electronic information storage systems such as network attached storage (NAS) and storage area networks (SAN)) and computer software (ie, instructions that allow a computing device to function in a particular way) It may be to achieve performance.

In addition, the processor of the computing device may include hardware components such as a micro processing unit (MPU) or a central processing unit (CPU), a cache memory, and a data bus. In addition, the computing device may further include an operating system and a software component of an application performing a specific purpose.

However, it is not excluded that the computing device includes an integrated processor in which a medium for implementing the present invention, a processor, and a memory are integrated.

On the other hand, the test device 2000 may be a device or a server for performing a concealing process on an original medical image, and may be mounted on a medical device producing an original medical image and a computer device linked therewith, so that the obtained original medical image A concealing process may be performed, but the present invention is not limited thereto.

A method of testing a modulation network that conceals an original medical image by performing different concealing processes for each region of the original medical image using the test apparatus 2000 according to an embodiment of the present invention configured as described above. Referring to 8, the explanation is as follows. Hereinafter, detailed descriptions of parts that can be easily understood from the description with reference to FIGS. 2 to 6 will be omitted.

In a state in which the modulation network 100 has been learned by the learning method with reference to FIGS. 2 to 6 , the test apparatus 2000, when a test medical image (x) is acquired, displays the acquired test medical image (x). By inputting the input to the modulation network 100, the modulation network 100 performs a first concealing process and a second concealing process corresponding to the first area for test and the second area for test, respectively, of the medical image (x) for test. Each of the processes may be performed on the first area for testing and the second area for testing to generate a modulated medical image for testing (x') corresponding to the medical image for testing (x).

At this time, the modulation network 100 may conceal the test medical image x with the test modulated medical image x' by performing different concealing processes for each area of the test medical image. For example, the modulation network 100 may generate a modulated medical image (x′) for test performed with different concealing degrees for the first and second areas of the test medical image (x). Further, the modulation network 100 simultaneously performs the first concealing process and the second concealing process so that the first area for testing is concealed by the first concealing process and the second area for testing is covered by the second concealing process. It can be made to be concealed by the sealing process. That is, the modulation network 100 mixes the first concealing process and the second concealing process to conceal the test medical image (x) so that the first area for test in the test medical image (x) is It can be concealed by one concealing process, and the second area for testing can be concealed by a second concealing process.

To this end, the test apparatus 2000 sets a region including medical information used for determining a medical state among regions of the medical image (x) for test as a first region for test, and sets the test in the medical image for test. An area other than the first area for testing is set as a second area for testing, and the modulation network 100 causes the first area for testing to have a relatively low noise density according to the degree of concealment through the first concealing process. , and the second area for testing may be subjected to concealing processing to have a relatively high noise density according to the degree of concealment through the second concealing process.

Here, the test device 2000 determines the first area for testing of the medical image (x) for testing from labeling information for testing in which the first area for testing and the second area for testing are set in the medical image (x) for testing. Information and second area information for testing may be obtained and input to the modulation network 100 .

In this case, the labeling information for testing of the first area for testing and the second area for testing in the medical image (x) for testing is determined by the human labeler in the medical image (x) for testing. It can be generated by setting two regions or by segmenting the medical image x for testing into a first region for testing and a second region for testing through a machine learning-based segmentation network.

Accordingly, the test device 2000 modulates the first area information for testing and the second area information for testing when acquiring the first area information for testing and the second area information for testing of the medical image x for testing. It is input to the network 100 to cause the modulation network 100 to perform a first concealing process on the first area for testing by referring to the information on the first area for testing and the information on the second area for testing. , a second concealing process may be performed.

The modulated medical image (x') for testing generated through the above process is recognized differently from the medical image (x) for testing by humans, but may be recognized as the same or similar to the medical image (x) for testing in the learning network. .

Meanwhile, modulated medical data concealed in the modulated network learned according to an embodiment of the present invention may be provided (or sold) to a buyer who wants to purchase medical image big data. In addition, the modulated medical data is transmitted to a third party or other institution, so that the third party or other institution can use the modulated medical data to diagnose the user's health condition without exposing the user's personal information. can

In addition, according to one embodiment of the present invention, while the concealed image data is provided (or sold) to the above buyer, a method of testing the learned modulation network is provided in the form of program instructions that can be executed through various computer components. It may be implemented in a form and recorded on a computer readable recording medium and provided. According to an embodiment of the present invention, the purchaser creates concealed data from original data that he or she has separately or obtained from other sources or routes by executing program instructions recorded on a recording medium using a computing device. You can use the concealed data in your own learning network. In addition, the buyer uses at least two or more of the concealed data, the original image data owned by the buyer or obtained from other sources or routes, and the concealed image data provided (or sold) above to his learning network. can also be used together.

On the other hand, according to an embodiment of the present invention, when the method of testing the learned modulation network is implemented in the form of program instructions that can be executed through various computer components, the accuracy is set high, and the purchaser's computing device overcomputes. Head (computational overhead) can be generated, so it is possible to prevent computational overhead by allowing the purchaser to reduce the accuracy through setting.

Embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and usable to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. media), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes such as those produced by a compiler. The hardware device may be configured to act as one or more software modules to perform processing according to the present invention and vice versa.

In the above, the present invention has been described by specific details such as specific components and limited embodiments and drawings, but these are provided to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. , Those skilled in the art to which the present invention pertains may seek various modifications and variations from these descriptions.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and not only the claims described later, but also all modifications equivalent or equivalent to these claims belong to the scope of the spirit of the present invention. will do it

1000: learning device
1001: memory of learning device
1002: processor of learning device
2000: test device
2001: Memory of test devices
2002: Processors in Test Devices
100: modulation network
200_1: first discriminator
200_2: second discriminator
300: image identification network
x: medical image for training or medical image for testing
x': modulated medical image for training or modulated medical image for testing
x + noise_1: first reference medical image
x + noise_2: second reference medical image

Claims (30)

A method for learning a modulation network for concealing an original medical image by performing different concealing processes for each region of the original medical image, the method comprising:
(a) A learning device inputs a medical image for learning to a modulation network and causes the modulation network to perform a first concealing process and a second concealing process corresponding to the first area for learning and the second area for learning, respectively, of the medical image for learning. generating a modulated medical image for learning corresponding to the medical image for learning by performing each of the first area for learning and the second area for learning;
(b) the learning device comprises: (i) a first discriminator for determining the first reference medical image on which the first concealing process is performed over the entire area of the training medical image as genuine, and the entire area of the training medical image The modulated medical image for learning is input to each of the second discriminators that determine the second reference medical image that has undergone the second concealing process as genuine, and causes the first discriminator and the second discriminator to determine the modulated medical image for learning. A process of outputting a first modulated medical image score to a second modulated medical image score that determines whether an image is real or fake, and (ii) inputting the modulated medical image for learning into an image identification network, so that the image identification network , performing a process of performing a running operation on the modulated medical image for learning and outputting medical feature information for learning corresponding to the modulated medical image for learning; and
(c) the learning device maximizes a sum loss obtained by adding a weighted sum of a 1_1 discrimination loss and a 2_1 discrimination loss generated by referring to the first modulated medical image score and the second modulated medical image score, and identifying the image; training the modulation network to minimize an accuracy loss generated by referring to the medical characteristic information for learning of the network or a task-specific output generated using the medical characteristic information for learning and a ground truth corresponding to the medical image for training;
How to include. According to claim 1,
(d) the 1_1 discrimination loss and the 2_1 discrimination generated by the learning device by referring to the first modulated medical image score and the second modulated medical image score by the first discriminator and the second discriminator, respectively. loss is minimized, and a first reference medical image score and a second reference medical image score corresponding to the first reference medical image and the second reference medical image respectively input to the first discriminator and the second discriminator are obtained. learning each of the first discriminator and the second discriminator to maximize a 1_2 th discriminating loss and a 2_2 nd discriminating loss generated with reference to each other;
A method characterized in that it further comprises. According to claim 2,
(e) training, by the learning device, the image identification network to minimize the accuracy loss;
A method characterized in that it further comprises. According to claim 1,
The method characterized in that the learning device is in a state in which the image identification network is trained to recognize the medical image for learning. According to claim 1,
When the learning device proceeds with learning of the modulation network using the summation loss, (i) for the first area for learning, the second area for learning corresponds to the 1_1 discrimination loss corresponding to the first area for learning. Learning is performed by applying a weight higher than the 2_1 discrimination loss corresponding to (ii) for the second area for learning, the 2_1 discrimination loss corresponding to the second area for learning is applied to the first area for learning. Characterized in that the learning proceeds by applying a weight higher than the corresponding 1_1 discrimination loss. According to claim 1,
The learning device causes the modulation network to conceal the first area for learning to have a relatively low noise density according to the degree of concealment through the first concealing process, and the second concealing process The method characterized in that the second area for learning is subjected to concealing processing so as to have a relatively high noise density according to the degree of concealment through According to claim 6,
The learning device sets an area including medical information used to determine a medical condition among areas of the medical image for learning as the first area for learning, and sets an area other than the first area for learning in the medical image for learning. characterized in that it is set as the second area for learning. According to claim 1,
The learning device obtains information on the first region for learning and information on the second region for learning of the medical image for learning from labeling information for learning in which a first region for learning and a second region for learning are set in the medical image for learning, and the modulation is performed. A method characterized by inputting into a network. According to claim 1,
The learning device inputs the training medical image to the modulation network, causes the modulation network to perform a convolution operation on the training medical image at least once through an encoder, and generates a feature map for learning through a decoder. A method characterized by outputting the modulated medical image for learning by performing deconvolution on a map at least once. A method for testing a modulation network that conceals an original medical image by performing different concealing processes for each area of the original medical image, the method comprising:
(a) by a learning device, (i) inputting a medical image for learning to a modulation network and causing the modulation network to perform a first concealing process for learning corresponding to each of the first area for learning and the second area for learning of the medical image for learning; and A second concealing process for learning is performed on the first area for learning and the second area for learning, respectively, to generate a modulated medical image for learning corresponding to the medical image for learning, (ii) (ii-1) the medical for learning A first discriminator for determining the first reference medical image subjected to the first concealing process for learning to the entire area of the image as genuine, and a second reference for performing the second concealing process for learning on the entire area of the medical image for training. The first modulated medical image for training is input to each of the second discriminators to determine whether the modulated medical image for learning is real or fake, and the first discriminator and the second discriminator determine whether the modulated medical image for learning is real or fake. A process of outputting an image score or a second modulated medical image score, and (ii-2) inputting the modulated medical image for training into an image identification network, which causes the image identification network to perform a learning operation on the modulated medical image for training A process of outputting medical feature information for learning corresponding to the modulated medical image for training is performed, and (iii) a 1_1 discrimination loss generated by referring to the first modulated medical image score and the second modulated medical image score and a second modulated medical image score are generated. Maximize the weighted summation loss of 2_1 discrimination losses, and refer to the medical feature information for learning or the medical feature information for learning of the image identification network and the ground truth corresponding to the medical image for training and the task specific output generated. obtaining, by a test device, a medical image for test in a state in which the modulation network is trained to minimize an accuracy loss generated by the above; and
(b) The test device inputs a medical image for test to the modulation network, and causes the modulation network to first control for testing corresponding to a first area for testing and a second area for testing, respectively, of the medical image for testing. generating a modulated medical image for test corresponding to the medical image for test by performing a sealing process and a second concealing process for test on the first area for test and the second area for test, respectively;
How to include. According to claim 10,
In step (a),
The first discriminator and the second discriminator respectively minimize the 1_1 discriminant loss and the 2_1 discriminator loss by the learning device, respectively, and the first discriminator input to the first discriminator and the second discriminator, respectively. The first and second reference medical image scores corresponding to the reference medical image and the second reference medical image are respectively learned to maximize the first_2 discriminant loss and the second_2 discriminant loss generated by referring to the score. How to characterize. According to claim 11,
The method of claim 1 , wherein the image identification network is in a state where it is learned by the learning device to minimize the accuracy loss or to recognize the medical image for learning. According to claim 10,
The test device causes the modulation network to conceal the first area for testing to have a relatively low noise density according to the degree of concealment through the first concealing process for testing, The method characterized in that the second area for the test is subjected to a concealing process to have a relatively high noise density according to the degree of concealing through a second concealing process. According to claim 13,
The test device sets an area including medical information used to determine a medical condition among areas of the medical image for test as the first area for testing, and in the medical image for test, the first area for testing. A method characterized by setting an area except for the second area for the test. According to claim 10,
The test device may obtain information about the first area for testing and the second area for testing of the medical image for testing from labeling information for testing in which a first area for testing and a second area for testing are set in the medical image for testing. A method characterized by obtaining area information and inputting it to the modulation network. A learning device for learning a modulation network that conceals an original medical image by performing different concealing processes for each region of the original medical image,
at least one memory for storing instructions; and
At least one processor configured to execute the instructions, wherein the processor (I) inputs a medical image for learning to a modulation network to cause the modulation network to generate a first area for learning and a second area for learning of the medical image for learning. A process of generating a modulated medical image for learning corresponding to the medical image for learning by performing a first concealing process and a second concealing process corresponding to each area on the first area for learning and the second area for learning, respectively; (II) (i) A first discriminator for determining that the first reference medical image on which the first concealing process is performed on the entire area of the training medical image is genuine and the second concealing process on the entire area of the training medical image The modulated medical image for training is input to each of the second discriminators for determining the processed second reference medical image as real, and the first discriminator and the second discriminator determine whether the modulated medical image for training is real or fake. a sub-process for outputting a first modulated medical image score to a second modulated medical image score, and (ii) inputting the modulated medical image for learning into an image identification network, which causes the image identification network to output the modulated medical image score for learning; a process of performing a sub-process of performing a learning operation on a medical image and outputting medical feature information for learning corresponding to the modulated medical image for learning; and (III) maximizing a weighted summation loss of a 1_1 discrimination loss and a 2_1 discrimination loss generated by referring to the first modulated medical image score and the second modulated medical image score, for the training of the image identification network. a process of training the modulation network to minimize an accuracy loss generated by referring to medical characteristic information or a task-specific output generated using the medical characteristic information for learning and a ground truth corresponding to the medical image for training; A learning device that performs According to claim 16,
(IV) the 1_1 discrimination loss and the 2_1 discrimination loss generated by the processor by referring to the first modulated medical image score and the second modulated medical image score by the first discriminator and the second discriminator, respectively. and refer to first reference medical image scores and second reference medical image scores corresponding to the first reference medical image and the second reference medical image respectively input to the first discriminator and the second discriminator, respectively. a process of learning each of the first discriminator and the second discriminator to maximize a 1_2 discriminating loss and a 2_2 discriminating loss generated by the process;
Learning device further comprising a. According to claim 17,
(V) a process by which the processor trains the image identification network to minimize the accuracy loss;
Learning device further comprising a. According to claim 16,
The learning device, characterized in that the processor is in a state in which the image identification network has been trained to recognize the medical image for learning. According to claim 16,
When the processor learns the modulation network using the summation loss, (i) for the first region for learning, the second region for learning is determined by the 1_1 discrimination loss corresponding to the first region for learning. Learning is performed by applying a weight higher than the corresponding 2_1 discrimination loss, and (ii) for the second learning area, the 2_1 discrimination loss corresponding to the second learning area corresponds to the first learning area. Learning apparatus characterized in that learning proceeds by applying a weight higher than the 1_1 discrimination loss. According to claim 16,
The processor causes the modulation network to conceal the first area for learning to have a relatively low noise density according to the degree of concealment through the first concealing process, and to perform the second concealing process. and subjecting the second area for learning to a concealing process so as to have a relatively high noise density according to the degree of concealment. According to claim 21,
The processor sets a region including medical information used to determine a medical state among regions of the medical image for learning as the first region for learning, and sets an area excluding the first region for learning in the medical image for learning to the learning region. A learning device characterized in that it is set as a second area for learning. According to claim 16,
The processor obtains the information of the first region for learning and the information of the second region for learning of the medical image for learning from labeling information for learning in which the first region for learning and the second region for learning are set in the medical image for learning, and the modulation network A learning device characterized in that input as. According to claim 16,
The processor inputs the training medical image to the modulation network, causes the modulation network to perform a convolution operation on the training medical image at least once through an encoder, and generates a training feature map through a decoder. The learning device characterized in that for outputting the modulated medical image for learning by performing deconvolution at least once. A test device for testing a modulation network that conceals an original medical image by allowing different concealing processes to be performed for each area of the original medical image,
at least one memory for storing instructions; and
At least one processor configured to execute the instructions, wherein the processor, (I) by a learning device, (i) inputs the medical image for training to a modulation network to cause the modulation network to generate the medical image for training. A first concealing process for learning and a second concealing process for learning corresponding to the first area for learning and the second area for learning are performed on the first area for learning and the second area for learning, respectively, corresponding to the medical image for learning. (ii) (ii-1) a first discriminator configured to generate a modulated medical image for learning, and (ii) (ii-1) determine a first reference medical image obtained by performing the first concealing process for learning on the entire area of the medical image for learning as genuine; The modulated medical image for learning is input to each of the second discriminators for determining that the second reference medical image, which has undergone the second concealing process for learning in the entire area of the medical image for learning, is genuine, and the first discriminator and the second A process of causing a discriminator to output a first modulated medical image score to a second modulated medical image score for determining whether the modulated medical image for learning is real or fake, and (ii-2) outputting the modulated medical image for training to an image identification network. to perform a process of causing the image identification network to perform a running operation on the modulated medical image for training and output medical feature information for training corresponding to the modulated medical image for training, and (iii) the first modulated medical image Maximizes a weighted sum loss of the 1_1 discrimination loss and the 2_1 discrimination loss generated by referring to the score and the second modulated medical image score, and the medical characteristic information for learning or the medical characteristic information for learning of the image identification network a process of acquiring a medical image for a test in a state where the modulation network is trained to minimize an accuracy loss generated by referring to a task-specific output generated using a task-specific output and a ground truth corresponding to the medical image for training; and (II) inputting the medical image for test to the modulation network to cause the modulation network to conduct a first concealing process and a test corresponding to the first area for testing and the second area for testing, respectively, of the medical image for testing. a process of generating a modulated medical image for test corresponding to the medical image for test by performing a second concealing process on the first area for test and the second area for test, respectively; A test device that performs According to claim 25,
In the above (I) process,
The first discriminator and the second discriminator respectively minimize the 1_1 discriminant loss and the 2_1 discriminator loss by the learning device, respectively, and the first discriminator input to the first discriminator and the second discriminator, respectively. The first and second reference medical image scores corresponding to the reference medical image and the second reference medical image are respectively learned to maximize the first_2 discriminant loss and the second_2 discriminant loss generated by referring to the score. characterized test device. The method of claim 26,
The test apparatus, characterized in that the image identification network is in a state where it is learned to minimize the accuracy loss or to recognize the medical image for training by the learning device. According to claim 25,
The processor causes the modulation network to conceal the first area for testing to have a relatively low noise density according to the degree of concealment through the first concealing process for testing, and the first area for testing is concealed. 2 The test device characterized in that the second area for the test is concealed through a concealing process to have a relatively high noise density according to the degree of concealment. According to claim 28,
The processor sets an area including medical information used to determine a medical condition among areas of the medical image for test as the first area for testing, and selects the first area for testing in the medical image for test. The test device characterized in that the excluded area is set as the second area for the test. According to claim 25,
The processor determines the information on the first area for testing and the second area for testing of the medical image for testing from labeling information for testing in which the first area for testing and the second area for testing are set in the medical image for testing. A test device characterized in that for obtaining information and inputting it to the modulation network.

Images