KR20230171160A - Method and apparatus for adjusting quality of medical image - Google Patents

Abstract

To adjust the quality of medical images, the adjustment method includes obtaining an input image as an input to an artificial intelligence model, extracting feature maps from the input image, and squeezing the feature maps according to the channel ( generating compressed and re-calibrated feature maps by performing a squeeze operation and a spatial excitation operation, and based on the compressed and re-calibrated feature maps, a quality adjusted output. It may include the step of generating an image.

Description

Method and apparatus for adjusting the quality of medical images {METHOD AND APPARATUS FOR ADJUSTING QUALITY OF MEDICAL IMAGE}

The present invention relates to medical images, and particularly to a method and device for controlling the quality of medical images.

Disease refers to a condition that causes disorders in the human mind and body, impeding normal functioning. Depending on the disease, a person may suffer and may even be unable to sustain his or her life. Accordingly, various social systems and technologies for diagnosing, treating, and even preventing diseases have developed along with human history. In the diagnosis and treatment of diseases, various tools and methods have been developed in accordance with the remarkable advancement of technology, but the reality is that they are still ultimately dependent on the judgment of doctors.

Meanwhile, artificial intelligence (AI) technology has recently developed significantly and is attracting attention in various fields. In particular, due to the environment of vast amounts of accumulated medical data and image-oriented diagnostic data, various attempts and research are underway to apply artificial intelligence algorithms to the medical field. Specifically, various studies are being conducted to use artificial intelligence algorithms to solve tasks that have traditionally been limited to clinical judgment, such as diagnosing and predicting diseases. In addition, various studies are being conducted to solve the task of processing and analyzing medical data as an intermediate process for diagnosis, etc. using artificial intelligence algorithms.

US published patent US 2021/0312242

US published patent US 2018/0293712

The present invention is intended to provide a method and device for effectively controlling the quality of medical images using an artificial intelligence (AI) algorithm.

The present invention is intended to provide a method and device for removing noise from medical images.

The present invention is intended to provide a method and device for generating a standard-dose or high-dose CT image based on a low-dose CT (computer tomography) image.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below. You will be able to.

According to an embodiment of the present invention, a method for adjusting the quality of a medical image includes obtaining an input image as an input to an artificial intelligence model, extracting feature maps from the input image, and regarding the feature maps, generating compressed and re-calibrated feature maps by performing a channel-dependent squeeze operation and a spatial excitation operation, and based on the compressed and re-calibrated feature maps. Thus, the step of generating a quality-adjusted output image may be included.

According to one embodiment of the present invention, the method may further include applying at least one dropout pattern to the artificial intelligence model.

According to one embodiment of the present invention, the at least one dropout pattern may be selected independently from dropout patterns applied differently for each epoch during training of the artificial intelligence model.

According to one embodiment of the present invention, the performance is determined by radiomics, peak signal-to-noise (PSNR), and structural similarity index method for the output image generated during the training. SSIM) may be evaluated based on at least one of the following:

According to an embodiment of the present invention, the artificial intelligence model includes a generator network that generates the output image, and a discrimination function to distinguish between a fake image and a real image generated by the generator network. May include a discriminator network.

According to one embodiment of the present invention, the generator network and the discriminator network are trained to minimize the discriminator loss and the generator loss, and the discriminator loss is the Wasistein distance. (Wasserstein distance), and the generator loss may be defined based on adversarial loss and similarity loss.

According to an embodiment of the present invention, the adversarial loss is determined based on patches extracted from the fake image generated by the generator network, and the similarity loss is determined based on the patches extracted from the fake image generated by the generator network. It can be determined based on the entire video.

According to one embodiment of the present invention, the input image includes a low-dose CT (computer tomography) image, and the output image includes a virtual standard-dose or high-dose CT (computer tomography) image. dose) may include CT images.

According to an embodiment of the present invention, a device for controlling the quality of medical images includes a storage unit that stores a set of instructions for operating the device, and at least one processor connected to the storage unit, wherein the at least One processor acquires an input image as an input to an artificial intelligence model, extracts feature maps from the input image, and performs a squeeze operation and a spatial excitation operation according to the channel for the feature maps. By performing , it is possible to generate compressed and re-calibrated feature maps, and control to generate a quality-adjusted output image based on the compressed and re-calibrated feature maps.

According to an embodiment of the present invention, a program stored in a computer-readable medium can execute the above-described method when operated by a processor.

The features briefly summarized above with respect to the present invention are merely exemplary aspects of the detailed description of the present invention that follows, and do not limit the scope of the present invention.

According to the present invention, medical images for training artificial intelligence models can be generated more effectively.

The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. will be.

1 shows a system according to one embodiment of the present invention.
Figure 2 shows the structure of a device for predicting the possibility of disease occurrence according to an embodiment of the present invention.
Figure 3 shows an example of a perceptron constituting an artificial intelligence model applicable to the present invention.
Figure 4 shows an example of an artificial neural network constituting an artificial intelligence model applicable to the present invention.
Figure 5 shows an example of a generative adversarial network (GAN) applicable to the present invention.
Figure 6 shows an example of the structure of an artificial intelligence model according to an embodiment of the present invention.
Figure 7 shows an example of a procedure for adjusting the quality of medical images according to an embodiment of the present invention.
Figure 8 shows an example of a procedure for training and operating an artificial intelligence model according to an embodiment of the present invention.
Figure 9 shows an example of loss values for training an artificial intelligence model according to an embodiment of the present invention.

Hereinafter, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein.

In describing embodiments of the present invention, if it is determined that a detailed description of a known configuration or function may obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, in the drawings, parts that are not related to the description of the present invention are omitted, and similar parts are given similar reference numerals.

The present invention is intended to control the quality of medical images. Specifically, the present invention relates to a technology for controlling the quality of a given medical image using a GAN (generative adversarial network)-based artificial intelligence model, and describes various embodiments of the structure and operation of the artificial intelligence model.

1 shows a system according to one embodiment of the present invention.

Referring to FIG. 1, the system includes a service server 110, a data server 120, and at least one client device 130.

The service server 110 provides services based on artificial intelligence models. That is, the service server 110 performs learning and prediction operations using an artificial intelligence model. The service server 110 may communicate with the data server 120 or at least one client device 130 through a network. For example, the service server 110 may receive learning data for training an artificial intelligence model from the data server 120 and perform training. The service server 110 may receive data required for learning and prediction operations from at least one client device 130. Additionally, the service server 110 may transmit information about the prediction result to at least one client device 130.

The data server 120 provides learning data for training the artificial intelligence model stored in the service server 110. According to various embodiments, the data server 120 may provide public data that anyone can access or provide data that requires permission. If necessary, the learning data may be preprocessed by the data server 120 or the service server 120. According to another embodiment, the data server 120 may be omitted. In this case, the service server 110 may use an externally trained artificial intelligence model, or learning data may be provided to the service server 110 offline.

At least one client device 130 transmits and receives data related to an artificial intelligence model operated by the service server 110 with the service server 110. At least one client device 130 is equipment used by the user, and transmits information input by the user to the service server 110, and stores the information received from the service server 110 or provides it to the user (e.g. : mark) is possible. In some cases, a prediction operation may be performed based on data transmitted from one client, and information related to the result of the prediction may be provided to another client. At least one client device 130 may be various types of computing devices, such as desktop computers, laptop computers, smartphones, tablets, and wearable devices.

Although not shown in FIG. 1, the system may further include a management device for managing the service server 110. The management device is a device used by the entity that manages the service, and monitors the status of the service server 110 or controls settings of the service server 110. The management device may be connected to the service server 110 through a network or directly through a cable connection. According to the control of the management device, the service server 110 can set parameters for operation.

As described with reference to FIG. 1, the service server 110, the data server 120, at least one client device 130, a management device, etc. may be connected and interact through a network. Here, the network may include at least one of a wired network and a wireless network, and may be comprised of any one or a combination of two or more of a cellular network, a local area network, and a wide area network. For example, the network is based on at least one of LAN (local area network), WLAN (wireless LAN), Bluetooth, long term evolution (LTE), LTE-advanced (LTE-A), and 5th generation (5G). This can be implemented.

Figure 2 shows the structure of a device for predicting the possibility of disease occurrence according to an embodiment of the present invention. The structure illustrated in FIG. 2 may be understood as the structure of the service server 110, data server 120, and at least one client device 130 of FIG. 1.

Referring to FIG. 2, the device includes a communication unit 210, a storage unit 220, and a control unit 230.

The communication unit 210 performs functions to connect to the network and communicate with other devices. The communication unit 210 may support at least one of wired communication and wireless communication. For communication, the communication unit 210 may include at least one of a radio frequency (RF) processing circuit and a digital data processing circuit. In some cases, the communication unit 210 may be understood as a component including a terminal for connecting a cable. Since the communication unit 210 is a component for transmitting and receiving data and signals, it may be referred to as a 'transceiver'.

The storage unit 220 stores data, programs, microcode, instruction sets, applications, etc. necessary for the operation of the device. The storage unit 220 may be implemented as a temporary or non-transitory storage medium. Additionally, the storage unit 220 may be fixed to the device or may be implemented in a detachable form. For example, the storage unit 220 may include compact flash (CF) cards, secure digital (SD) cards, memory sticks, solid-state drives (SSD), and micro It may be implemented as at least one of magnetic computer storage devices such as NAND flash memory such as an SD card and a hard disk drive (HDD).

The control unit 230 controls the overall operation of the device. To this end, the control unit 230 may include at least one processor, at least one microprocessor, etc. The control unit 230 can execute a program stored in the storage unit 220 and access the network through the communication unit 210. In particular, the control unit 230 may perform algorithms according to various embodiments described later and control the device to operate according to embodiments described later.

Based on the structure described with reference to FIGS. 1 and 2, services based on artificial intelligence algorithms can be provided according to various embodiments of the present invention. Here, an artificial intelligence model consisting of an artificial neural network can be used to implement the artificial intelligence algorithm. The concepts of perceptron, which is a structural unit of artificial neural network, and artificial neural network are as follows.

The perceptron is modeled after a biological nerve cell and has a structure that takes multiple signals as input and outputs a single signal. Figure 3 shows an example of a perceptron constituting an artificial intelligence model applicable to the present invention. Referring to Figure 3, the perceptron sets weights (302-1 to 302-n) (e.g., w _1j , w _2j , _w ) for each of the input values (e.g., x ₁ , x ₂ , x ₃ , ..., x n). After multiplying by _3j , ..., w _nj ), the weighted input values are summed using a transfer function 304. During the summation process, a bias value (e.g., b _k ) may be added. The perceptron generates an output value (e.g., o _{j) by applying an activation function (406) to the net input value (e.g., net j )} , which is the output of the transformation function (304). In some cases, activation function 406 may operate based on a threshold (eg, θ _j ). Activation functions can be defined in various ways. The present invention is not limited to this, but for example, a step function, sigmoid, Relu, Tanh, etc. may be used as the activation function.

An artificial neural network can be designed by arranging perceptrons as shown in Figure 3 and forming layers. Figure 4 shows an example of an artificial neural network constituting an artificial intelligence model applicable to the present invention. In FIG. 4, each node represented by a circle can be understood as the perceptron of FIG. 3. Referring to FIG. 4, the artificial neural network includes an input layer 402, a plurality of hidden layers 404a and 404b, and an output layer 406.

When performing prediction, when input data is provided to each node of the input layer 402, the input data is weighted and transformed by the perceptrons that make up the input layer 402 and the hidden layers 404a and 404b. and forward propagation to the output layer 406 through activation function calculation, etc. Conversely, when training is performed, the error is calculated through backward propagation from the output layer 406 to the input layer 402, and the weight values defined in each perceptron can be updated according to the calculated error. there is.

GAN (generative adversarial network) is an artificial intelligence network designed to perform adversarial learning, and is trained in a way that the generator and discriminator learn competitively and both develop. In other words, the generator strives to lower the discriminator's classification success probability, and the discriminator strives to increase the classification success probability, developing competitively with each other. The structure of GAN is shown in Figure 5 below.

Figure 5 shows an example of a GAN applicable to the present invention. Figure 5 illustrates the structure of a GAN that classifies samples (e.g., images). Referring to Figure 5, GAN includes a generator 510 and a discriminator 520. The generator 510 and the discriminator 520 each include an artificial neural network, the generator 510 generates fake samples, and the discriminator 520 distinguishes the fake samples from real samples. Specifically, the generator 510 generates a synthetic sample based on random input. For example, random input may include latent vectors. The discriminator 520 is trained in advance using real samples and is further trained to distinguish between real samples and synthetic samples.

To this end, the discriminator 520 uses real samples and synthetic samples as learning data and outputs at least one loss value, for example, a discriminator loss and a generator loss. The discriminator loss includes information about misclassifying a real sample as a synthetic sample or misclassifying a synthetic sample as a real sample, and acts as a penalty to the discriminator 520. When backpropagation is performed using the discriminator loss, the weights of the artificial neural network in the discriminator 520 are updated. Generator 510 is trained based on feedback (e.g., generator loss) from discriminator 520. Through training, generator 510 can evolve to generate synthetic samples that are more similar to real samples. The generator loss includes information about cases where the discriminator 520 classified without error, and acts as a penalty to the generator 510. When backpropagation is performed using the generator loss, the weights of the artificial neural network in the generator 510 are updated.

The structure of the GAN described above can be applied to an artificial intelligence model for medical image control according to various embodiments. Hereinafter, as an example of a medical image, embodiments of quality control for CT (computer tomography) images will be described. However, various embodiments of the present invention can be used in various forms of medical imaging, such as CT images, X-rays, ultrasound, magnetic resonance imaging (MRI), positron emission tomography (PET), and single photon emission computed tomography (SPET). It can also be applied to fields.

CT is one of the reliable imaging tests for diagnosing diseases. The quality of CT images depends on scan factors such as voltage and current. X-ray dose is directly related to image quality. High-dose results in high-definition images, but can cause DNA damage and cell malformations, causing disease or tumors. On the other hand, low doses can reduce the dose irradiated to the patient, but can cause deterioration of image quality due to noise or artifacts. At this time, the reconstruction algorithm can improve the quality of the CT image obtained by low-dose parameters. Recently, deep learning has shown promising performance in the fields of computer vision and medical image processing. GAN (generative adversarial networks), an unsupervised learning method, shows excellent performance in CT noise removal research. For example, WGAN (Wasserstein GAN) is an improved GAN framework based on the Wasserstein distance and shows higher training stability than existing GANs. However, clinical adoption of GAN-based post-processing algorithms still has limitations.

To solve the above-mentioned problem, the present invention proposes a new CT image denoising model including uncertainty estimation. Artificial intelligence models according to various embodiments improved performance through residual dense blocks and skip connections. At the same time, artificial intelligence models according to various embodiments use an attention module to deliver inherently important data rather than simply conveying information. Additionally, in order to create more stable noise removal performance, artificial intelligence models according to various embodiments measure model uncertainty using Bayesian approximation.

The relationship between low-dose CT (low-does CT, LDCT) and the corresponding standard-dose CT (SDCT) can be expressed as [Equation 1] below.

In [Equation 1], y means standard dose CT, x means low dose CT, and T() means the transformation matrix (translation function).

In general, the noise distribution of CT images can be modeled by a complex synthesis of Poisson quantum noise and Gaussian electronic noise. Because the relationship T between noises cannot be accurately defined, existing conventional noise removal techniques cannot achieve good performance in CT noise removal. The network is trained to generate images by solving an inverse problem. The inverse-transformation function can be expressed as [Equation 2] below.

는 역-변환 함수에 의해 생성된 저선량 CT, x는 저선량 CT를 의미한다. 네트워크는 비선형 함수를 통해 고차원 기능을 효율적으로 학습한다. 결과적으로 x에 가까운 잡음 제거 영상이 생성될 수 있다.In [Equation 2], T ^-1 () is the inverse function of the conversion function, y is the standard dose CT,

means low-dose CT generated by the inverse-transform function, and x means low-dose CT. The network efficiently learns high-dimensional features through nonlinear functions. As a result, a noise-removed image close to x can be generated.

If a GAN-based artificial intelligence model is designed with a fully convolutional structure, kernels within the artificial intelligence model can be trained using images of arbitrary sizes. In this case, the convolution kernels in the generator network can be trained using randomly sampled local patches. However, the convolution kernels of the artificial intelligence model according to embodiments of the present invention can be trained using images of the original size (e.g., 512 × 512) so as to generate an output image of the original size. . This allows the perceptual difference between standard-dose CT and fake standard-dose CT to be measured in the original dimension, rather than in the locally sampled dimension. Meanwhile, the discriminator network can be trained using randomly sampled local patches to distinguish the local texture of standard dose CT and fake standard dose CT. Specific examples of the structure of artificial intelligence models according to various embodiments are shown in FIG. 6 below.

Figure 6 shows an example of the structure of an artificial intelligence model according to an embodiment of the present invention. The description below with reference to FIG. 6 specifically illustrates the sizes of blocks/modules/layers constituting the artificial intelligence model, but the sizes illustrated below may be modified depending on specific embodiments.

The generator network 610 takes a low-dose CT of the original size (e.g., 512×512) as input and produces a dummy standard-dose CT of the same size. The convolution layer 612 encodes the input low-dose CT images into 32 feature maps through 3×3 convolution kernels with leaky ReLU activation. Encoded feature maps are provided as DRBD (dense residual block with dropout) blocks. The DRBD block includes three consecutive convolutional layers 613-1 to 613-3, and are connected by a residual operation. Additionally, the DRBD block includes an additional convolution layer 613-4 and a squeeze and spatial excitation (sSE) module 614 that processes the operation results of the three convolution layers 613-1 to 613-3. .

Specifically, the output of the convolution layer 613-1 and the output of the convolution layer 612 are summed by the summer 617a and then input to the convolution layer 613-2. The output of the convolution layer 613-1, the output of the convolution layer 613-2, and the output of the convolution layer 612 are summed by the summer 617b, and then added to the convolution layer 613-3. is entered into The output of the convolution layer 613-1, the output of the convolution layer 613-2, the output of the convolution layer 613-3, and the output of the convolution layer 612 are summed by a summer 617c. After that, it is input to the convolution layer (613-4). Each of the convolutional layers 613-1 to 613-4 may have leaky ReLU activation. For example, the Leaky ReLU activation function may have a negative slope coefficient of 0.3.

The squeeze and spatial excitation (sSE) module 614 can enhance the use of feature maps by compressing and re-calibrating the feature maps. Specifically, the sSE module 614 performs a squeeze operation that summarizes the entire information about the feature maps and an excitation operation that scales the importance of each feature map. At this time, the sSE module 614 effectively uses pixel-specific spatial information by performing a squeeze operation along a channel and a stimulation operation spatially. Specifically, the sSE module 614 extracts one feature map encompassing all channels and applies the extracted feature map to each channel. That is, the sSE module 614 multiplies the extracted feature map by the feature map of each channel. Here, the feature maps input to the sSE module 614 have a three-dimensional structure of H × W × C, and the extracted feature maps have a two-dimensional structure of H × W × 1, and the feature maps output from the sSE module 614 The result can again have a three-dimensional structure of H×W×C. Here, C means the number of channels. Additionally, the sSE module 614 processes the input in a 1×1 convolutional layer with sigmoid activation to obtain the squeezed information, and multiplies the squeezed information and the original input. A three-dimensional, that is, spatially recalibrated feature map can be obtained.

In Figure 6, the output of the sSE module 614 and the output of the convolution layer 613-4 are expressed as being multiplied by the multiplier 618, but the operation of the sSE module 614 described above is performed by the multiplier 618. Includes a multiplication operation by . Accordingly, the multiplier 618 of FIG. 6 may be understood as being included in the sSE module 614, or the multiplication operation among the above-described operations may be understood as being excluded from the operation of the sSE module 614.

The dropout layer 615 performs a dropout operation at a set rate. The input to the dropout layer 615 is the result of summing the output of the sSE module 614 (e.g., the output of the multiplier 618) and the output of the convolution module 612 by the summer 617d. The dropout layer 615 can induce balanced training among nodes of artificial intelligence models and prevent overfitting by excluding certain neurons, that is, nodes, from training with a certain probability. In other words, the dropout layer 615 may exclude at least one node during training and perform forward/backward propagation using the remaining nodes. At least one specific node may be excluded from training by dropout, and the pattern excluded by the dropout operation (hereinafter referred to as 'dropout pattern') may be adjusted differently for each epoch. Here, excluding a node can be expressed as turning the node off. According to one embodiment, the exclusion probability applied to each node may be defined as a fixed value or may be adaptively adjusted depending on the training situation. For example, the exclusion probability may be set to 0.5. Determination of the exclusion probability, control of the dropout pattern, etc. may be performed by a separate control module (e.g., the control unit 230 of FIG. 2).

Convolutional layers except the last convolutional layer 616, i.e., convolutional layers 612, 613-1 to 613-4, include 3×3 convolution kernels. The weights and biases of the convolution kernels can be initialized by He normal initialization. The convolution kernel included in the last convolution layer 616 may have a size of 9×9 to enable fast learning. The final convolution layer 616 processes the feature maps output from the dropout layer 615 into a form that matches the desired channel. For example, the form of the feature maps output from the DRBD block is [batch size, 512, 512, 16], and the feature maps are converted into the final output form by the last convolution layer 616, [batch size, 512, 512, 1] can be processed into sizes. In other words, the final output is an image with one channel, and the last convolution layer 616 generates the final output by rearranging and compressing the previous feature maps. To preserve the dimensionality of the feature map, zero padding may be applied to all convolutional layers 612, 613-1 to 613-4, 616.

Discriminator network 620 is trained to distinguish local patches from standard-dose CT and sham standard-dose CT. For example, local patches of a given size (e.g., 64 there is. The discriminator network 620 includes four convolutional blocks 622-1 to 622-4 and one convolutional layer 623. In each of the convolution blocks 622-1 to 622-4, the first layer includes 3×3 convolution kernels using 1-stride, and the second layer includes 3×3 convolution kernels using 2-strides. Contains kernels. Here, stride refers to how many pixels are skipped while processing the image. The two convolutional layers, the first layer and the second layer, do not include zero-padding and are activated using ReLU. The number of convolutional feature maps is doubled when passing through the next convolutional block. The last convolution layer 623 includes a 4×4 convolution kernel with one stride. The output of convolutional layer 623 may include a single probability value.

Figure 7 shows an example of a procedure for adjusting the quality of medical images according to an embodiment of the present invention. FIG. 7 illustrates a method of operating a device with computing capabilities (eg, the service server 110 of FIG. 1).

Referring to FIG. 7, in step S701, the device acquires an input image. The input image is an image with relatively low quality. For example, the input image may be a low-dose CT image. The input image may be provided in real time along with imaging from medical imaging equipment or may be provided from a database storing medical images.

In step S703, the device extracts feature maps from the input image. To extract feature maps, the device may use at least one convolutional layer included in the artificial intelligence model. The device can extract feature maps corresponding to a plurality of channels, and each feature map can have a two-dimensional structure.

In step S705, the device performs a squeeze operation and a spatial stimulation operation according to the channel. That is, the device extracts key information from the extracted feature maps and spatially readjusts the extracted information. That is, the device summarizes the information of the extracted feature maps and scales the summarized information. Specifically, the device may create a filter for summary corresponding to all feature maps and multiply the filter by the feature maps. Here, the filter can be understood as another feature map. Accordingly, the device can generate compressed and re-calibrated feature maps.

In step S707, the device generates an output image with adjusted quality. The device may generate an image with adjusted quality based on summarized and scaled feature maps, that is, compressed and re-calibrated feature maps. Adjusted quality may include enhanced or improved quality. Here, improved quality means having higher quality compared to the image acquired in step S701. For example, an image with improved quality may be clearer, contain more feature information, or have greater resolution than the input image.

Through the procedure described with reference to FIG. 7, the quality of the input image can be improved. Although not shown in FIG. 7, according to one embodiment, the device may apply a dropout pattern. Here, when performing dropout, weights can be updated for nodes removed in each epoch according to a preset probability without depending on other nodes. By dropout, a prediction operation may be performed with at least one node included in the artificial intelligence model excluded. At this time, the dropout pattern can be randomly defined in various ways based on probability. The procedure for applying the dropout pattern is shown in FIG. 8 below.

Figure 8 shows an example of a procedure for training and operating an artificial intelligence model according to an embodiment of the present invention. FIG. 8 illustrates a method of operating a device with computing capabilities (eg, the service server 110 of FIG. 1).

Referring to FIG. 8, in step S801, the device performs training by changing the dropout pattern for each epoch. That is, during the training process, the device excludes at least one node by randomly determining and applying dropout patterns and performs forward/backward propagation using the remaining nodes. At this time, the device can perform backward propagation by calculating the generator loss and discriminator loss based on the results of forward propagation, and updating the weights of the generator network and discriminator network to minimize the generator loss and discriminator loss. In this case, only the weights related to the remaining nodes except at least one node can be updated. Here, the dropout pattern changes on an epoch basis. As dropout is performed, weights can be updated for nodes removed in each epoch according to a preset probability without depending on other nodes.

In step S803, the device performs performance evaluation for each epoch. The device can evaluate the performance of the artificial intelligence model for each epoch using the learning data selected for performance evaluation. Performance evaluation can be performed in a variety of ways. According to one embodiment, performance evaluation may be performed based on radiomics. Specifically, the device extracts patches from each of the imitation images generated by the generator and the image corresponding to the label, and generates radiomics features for each of the extracted patches. For example, radiomics features include gray-level co-occurrence matrix (GLCM), gray-level run length matrix (GLRLM), gray-level size zone matrix (GLSZM), gray-level dependence matrix (GLDM), and NGTDM ( neighboring gray tone difference matrix) or modified information thereof. The device may determine a comparison index (e.g., concordance correlation coefficient (CCC)) based on the generated radiomics features and evaluate performance based on the determined comparison index. For example, if the comparison index is greater than the threshold, it may be evaluated as having sufficient performance. In other words, comparative indicators can be interpreted as performance.

In step S805, the device operates an artificial intelligence model by applying a plurality of dropout patterns. That is, the device uses an artificial intelligence model that includes nodes with independently updated weights in multiple epochs, generates multiple prediction results by applying multiple dropout patterns determined based on probability, and then predicts By combining the results, a single prediction result can be generated. According to one embodiment, a device may combine prediction results by averaging a plurality of prediction results. According to another embodiment, a device may combine prediction results by weight averaging a plurality of prediction results.

With reference to FIG. 8, an embodiment related to the dropout pattern has been described. Here, the dropout pattern may be a combination of multiple dropout results. For example, by a dropout operation with probability 0.5, a subset A of nodes may be selected by excluding some of the total nodes in the artificial intelligence model. Additionally, when the dropout operation is performed again, subset B of nodes can be selected by excluding other parts of all nodes. The subset of nodes selected during each dropout operation may be different, and one dropout pattern can be defined by grouping a plurality of subsets. In this case, applying a dropout pattern can be understood as averaging images resulting from a plurality of dropout operations that derive subsets of different nodes. As a specific example, one dropout pattern may include subsets of nodes that have been dropped out 20 times.

In the training process of the aforementioned artificial intelligence model, objective functions can be defined as minimizing discriminator loss and generator loss. Here, the discriminator loss and generator loss can be defined as follows.

First, the discriminator loss can be defined as [Equation 3] below.

는 입력 영상 및 생성자의 출력 간 직선(straight line) 상에서 랜덤하게 샘플링된 2D 가우시안 점(gaussian point), 는

의 경사도, 는 판별자에 의해 판별된

에 대한 결과 값을 의미한다.In [Equation 3], L _D is the discriminator loss, E[] is the expected value operator, D(x) is the output of the discriminator, G(x) is the output of the generator,

is a 2D Gaussian point randomly sampled on a straight line between the input image and the output of the generator, Is

The slope of is determined by the discriminator

It means the result value for .

Here, the first two terms are discriminator losses using the Wasistein distance, and the last term means the gradient penalty.

At this time, the adversarial loss for the generator can be defined as [Equation 4] below.

In [Equation 4], L _adv means the adversarial loss, E[] means the expected value operator, D(x) means the output of the discriminator, and G(x) means the output of the generator.

According to one embodiment, multi-scale structural similarity (MS-SSIM) may be used as the similarity loss for analysis flexibility. MS-SSIM can be defined as follows [Equation 5].

In [Equation 5], SSIM(x,y) is the structural similarity for image x and image y, C ₁ and C ₂ are constants, μ _x is the standard deviation for image x, and μ _y is the image standard deviation for y, σ _x is the cross-covariance for image x, σ _y is the cross-covariance for image x, σ _xy is the cross-covariance for image x and image y, _and is the local image content at the jth level for image y, y _j is the local image content at the jth level for image y, and M is the scale level.

Based on the above MS-SSIM, the similarity loss can be defined as [Equation 6] below.

In [Equation 6], L _similarity means similarity loss, and MS_SSIM(x,y) means MS-SSIM for image x and image y.

Generator loss according to one embodiment may be defined based on the above-described adversarial loss and similarity loss. For example, as shown in [Equation 7] below, the generator loss can be defined as the sum of the adversarial loss and the similarity loss.

In [Equation 7], L _G refers to the generator loss, L _adv refers to the adversarial loss, L _similarity refers to the similarity loss, and α refers to the weight for the similarity loss. As shown in [Equation 7], the adversarial loss and similarity loss can be combined with the same weight or different weights.

Figure 9 shows an example of loss values for training an artificial intelligence model according to an embodiment of the present invention. Figure 9 shows the relationship between similarity loss and adversarial loss. Referring to FIG. 9, a low-quality image 902 is input to the generator 910, and the generator 910 generates a fake high-quality image 904. Then, an actual high-quality image 906 corresponding to the label is provided. Here, the similarity loss (L _similarity ) may be determined based on the fake high-quality image 904 and the actual high-quality image 906. A plurality of patches 954 and 956 are randomly extracted from each of the fake high-quality image 904 and the actual high-quality image 906. The extracted plurality of patches 954 and 956 are input to the discriminator 920, and the adversary stone loss (L _adv ) can be determined from the discrimination result by the discriminator 920. That is, the adversarial loss (L _adv ) is determined based on the patches 954 extracted from the generated fake high-quality image 904 and the patches 956 of the actual high-quality image 906, and the similarity loss (L _similarity ) may be determined based on the totality of the simulated high quality image 904 and the actual high quality image 906.

Exemplary methods of the present invention are expressed as a series of operations for clarity of explanation, but this is not intended to limit the order in which the steps are performed, and each step may be performed simultaneously or in a different order, if necessary. In order to implement the method according to the present invention, other steps may be included in addition to the exemplified steps, some steps may be excluded and the remaining steps may be included, or some steps may be excluded and additional other steps may be included.

The various embodiments of the present invention do not list all possible combinations, but are intended to explain representative aspects of the present invention, and matters described in the various embodiments may be applied independently or in combination of two or more.

Additionally, various embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof. For hardware implementation, one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general purpose It can be implemented by a processor (general processor), controller, microcontroller, microprocessor, etc.

The scope of the present invention includes software or machine-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that enable operations according to the methods of various embodiments to be executed on a device or computer, and such software or It includes non-transitory computer-readable medium in which instructions, etc. are stored and can be executed on a device or computer.

Claims (10)

In a method for controlling the quality of medical images,
Obtaining an input image as input to an artificial intelligence model;
extracting feature maps from the input image;
generating compressed and re-calibrated feature maps by performing a channel-dependent squeeze operation and a spatial excitation operation on the feature maps; and
A method comprising generating a quality-adjusted output image based on the compressed and re-calibrated feature maps.
In claim 1,
The method further includes applying at least one dropout pattern to the artificial intelligence model.
In claim 2,
A method in which the at least one dropout pattern is selected independently from dropout patterns applied differently for each epoch during training of the artificial intelligence model.
In claim 3,
The performance is evaluated based on at least one of radiomics, peak signal-to-noise ratio (PSNR), and structural similarity index method (SSIM) for the output image generated during the training. How to become.
In claim 1,
The artificial intelligence model includes a generator network that generates the output image, and a discriminator network that distinguishes between a fake image and a real image generated by the generator network. .
In claim 5,
The generator network and the discriminator network are trained to minimize discriminator loss and generator loss,
The discriminator loss is defined based on the Wasserstein distance,
A method in which the generator loss is defined based on adversarial loss and similarity loss.
In claim 6,
The adversarial loss is determined based on patches extracted from the fake image generated by the generator network,
The method wherein the similarity loss is determined based on the totality of fake images generated by the generator network.
In claim 1,
The input image includes a low-dose CT (computer tomography) image,
The output image includes a virtual standard-dose or high-dose CT image.
In a device for controlling the quality of medical images,
a storage unit that stores a set of instructions for operating the device; and
It includes at least one processor connected to the storage unit,
The at least one processor,
Obtain the input image as input to the artificial intelligence model,
Extract feature maps from the input image,
Generate compressed and re-calibrated feature maps by performing a channel-dependent squeeze operation and a spatial excitation operation on the feature maps,
A device that controls to generate a quality-adjusted output image based on the compressed and re-calibrated feature maps.
A program stored in a computer-readable medium to execute the method according to any one of claims 1 to 8 when operated by a processor.

Images