KR102186632B1 - Device for training analysis model of medical image and training method thereof - Google Patents

Abstract

It relates to a learning device for efficiently learning an analysis model of a medical image and a learning method thereof. The learning device includes a memory and a processor, and the memory stores an analysis model for analyzing a medical image. Accordingly, it is possible to shorten the time required for labeling lesions of a plurality of 3D medical images.

Description

TECHNICAL FIELD [Device for training analysis model of medical image and training method thereof]

The present invention relates to a learning device for learning an analysis model of a medical image and a learning method thereof, and more particularly, to a learning device and a learning method for learning an analysis model based on a lesion directly checked by a user.

With the development of electronic technology, various electronic devices are also used in the medical field. In particular, medical image analysis devices capable of analyzing a medical image captured through a camera to detect a patient's diseased part or disease name are increasingly being used.

In the case of a conventional medical image analysis device, it was generally implemented in a labeling method. The labeling method refers to a method in which a user directly checks a lesion to extract a lesion from a brain magnetic resonance (MR) image.

The conventional medical image analysis apparatus uses an analysis model in order to further improve analysis performance and efficiency. The analysis model refers to a software model for labeling the input medical image based on previously labeled data when a medical image is input.

Such an analysis model should be trained frequently or periodically to improve performance. In this case, it is common to label data arbitrarily selected by the user for a long time and then perform learning using them. Therefore, in the conventional analysis model, if randomly selected data is not good, there may be a problem that learning is inefficiently performed. In addition, there are many cases of learning meaningless data, which can lead to a problem that learning proceeds inefficiently.

The present invention was created to solve such a problem, and an object of the present invention is a learning device capable of learning an analytic model using an image useful for a learning device for learning an analytic model of a medical image, and a learning method thereof In providing.

A learning apparatus for learning an analysis model of a medical image according to an embodiment of the present invention for achieving the above object includes: a memory storing the analysis model; And a processor for analyzing a medical image using the analysis model, wherein the processor learns the analysis model using the first medical image for which labeling has been completed, and the learned on the second medical image before labeling. By applying an analysis model, the uncertainty of each frame of the second medical image is calculated, and at least one frame in which the uncertainty exceeds a preset condition is selected from among the second medical images, and labeling of the selected frame is performed. It is a learning device that learns the analysis model by using the third medical image after generating a third medical image by adding a frame to which the input is received and the labeling is inputted to the first medical image.

In addition, the processor generates a probability map of the second medical image by applying the learned analysis model to the second medical image before labeling, and calculates an uncertainty of each frame using the probability map, and the The uncertainty may be characterized by a sum of entropy of each voxel of the probability map.

In addition, it may further include a display; in this case, the processor is a UI (User Interface) for requesting a labeling input for at least one frame selected as the uncertainty exceeds a preset condition among the second medical images. ) Is displayed on the display, and when a lesion portion of each of the at least one frame is checked on the UI, data on the checked portion is added to the data of each frame, so that the at least one frame can be labeled. .

Meanwhile, a method of learning an analysis model for a medical image according to an embodiment of the present invention includes: learning an analysis model using a first medical image for which labeling has been completed; Calculating an uncertainty of each frame of the second medical image by applying the learned analysis model to a second medical image before labeling; Selecting at least one frame from among the second medical images in which the uncertainty exceeds a preset condition; When labeling for the selected frame is input, generating a third medical image by adding the label inputted frame to the first medical image; And learning the analysis model using the third medical image.

In addition, calculating the uncertainty may include generating a probability map of the second medical image by applying the learned analysis model to the second medical image before labeling; And calculating an uncertainty of the second medical image using the probability map, wherein the uncertainty may be a sum of entropy of each voxel of the probability map.

In addition, the generating of the third medical image may include displaying a UI (User Interface) requesting a labeling input for at least one frame selected as the uncertainty of the second medical image exceeding a preset condition. step; When a lesion portion of each of the at least one frame is checked on the UI, labeling the at least one frame by adding data on the checked portion to the data of each frame; And generating the third medical image by adding the labeled frame to the first medical image.

Meanwhile, a recording medium in which a program for executing a learning method for learning an analysis model for medical images according to an embodiment of the present invention may be stored may be disclosed. Here, the learning method includes: learning an analysis model using the first medical image for which labeling has been completed; Calculating an uncertainty of each frame of the second medical image by applying the learned analysis model to a second medical image before labeling; Selecting at least one frame from among the second medical images in which the uncertainty exceeds a preset condition; When labeling for the selected frame is input, generating a third medical image by adding the label inputted frame to the first medical image; And learning the analysis model using the third medical image.

According to various embodiments of the present invention as described above, the analysis device automatically selects an image useful for updating a learning model among image data without labeling, thereby reducing the labeling time of the user.

1 is a block diagram showing the configuration of a learning device according to an embodiment of the present invention.
2 is a diagram showing a first medical image, a second medical image, and a third medical image according to an embodiment of the present invention.
3 is a block diagram showing the configuration of a medical image analysis device further including a display in the learning device.
4 is a diagram illustrating a division of one 3D frame into a 2D image according to an embodiment of the present invention.
5(a) is a diagram illustrating an example of a UI configuration displayed according to an embodiment of the present invention.
FIG. 5(b) is a diagram illustrating labeling a 2D image displayed by a user according to an embodiment of the present invention.
FIG. 5(c) is a diagram illustrating that a user transfers a displayed 2D image to a next 2D image according to an embodiment of the present invention.
6 is a flowchart illustrating a method of learning an analysis model for a medical image according to an embodiment of the present invention.
7 is a table comparing the domainization performance between a learning model made from randomly selected data and data selected by a proposed technique.
8 is a table comparing the domainization performance of a random selection technique and a proposed selection technique (active learning).

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. Transmission or transmission referred to in this specification may mean transmission of data, information, or signals, and encryption/decryption may be applied as necessary.

In addition, in this specification, expressions in the form of "transmitted from A to B (transmitted)" or "received from A by B" include those transmitted (transmitted) or received with another medium in the middle, and must be It does not express only what is directly transmitted (delivered) or received up to B. In addition, each of the devices illustrated and referred to in the present specification may be implemented as devices that are independent of each other, but are not necessarily limited thereto and may be implemented with several components included in one device.

In the description of the present invention, the order of each step should be understood without limitation, unless the preceding step must be performed logically and temporally prior to the subsequent step. That is, except for the above exceptional cases, even if a process described as a subsequent step is performed prior to a process described as a preceding step, the essence of the invention is not affected, and the scope of rights should be defined regardless of the order of the steps.

And in the present specification, "A or B" is defined to mean not only selectively indicating any one of A and B, but also including both A and B. In addition, in the present specification, the term "comprising" has a meaning to include further components other than the listed elements.

In the present specification, components necessary for the description of each embodiment of the present invention have been described, and the present disclosure is not limited thereto. Accordingly, some components may be changed or omitted, and other components may be added. It can also be distributed and arranged in different independent devices.

The mathematical operation and calculation of each step of the present invention to be described later may be implemented as a computer operation by a known coding method for performing the corresponding operation or calculation and/or coding designed suitable for the present invention.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing the configuration of a learning device according to an embodiment of the present invention.

The learning device 100 refers to a learning device that learns an analysis model of a medical image. The learning device 100 may be implemented as an analysis device that analyzes a medical image, but may be implemented as a separate device. For example, the learning device 100 may be implemented as a smart phone, a tablet PC, a desktop PC, a laptop PC, a netbook computer, a server, a PDA, a PMP, a kiosk, or the like. When the learning device 100 is implemented as a device separate from the analysis device, the learning device 100 may be connected to the analysis device through a wired or wireless interface to provide an analysis model.

As shown in FIG. 1, the learning apparatus 100 may include a memory 120 and a processor 110.

The memory 120 may store various programs and data necessary for the operation of the learning device 100. In particular, the memory 120 may store an analysis model for analyzing a medical image.

The analytic model refers to an artificial intelligence model learned by an artificial intelligence algorithm. The analysis model may label a lesion area in a medical image frame or generate a probability map for the medical image frame. For example, when a medical image frame is input to an artificial intelligence model, a medical image labeled with a lesion area may be obtained as output data.

The analysis model may be a model based on a neural network. Specifically, there may be a deep neural network (DNN), a recurrent neural network (RNN), a bidirectional recurrent depp neural network (BRDNN), and the like, but is not limited thereto.

The memory 120 may include, for example, an internal memory or an external memory. The built-in memory may include, for example, at least one of a volatile memory and a nonvolatile memory. The external memory may include a flash drive, for example, CF, SD, Micro-SD, Mini-SD, xD, MMC or memory stick.

The memory 120 is accessed by the processor 110, and data read/write/edit/delete/update by the processor 110 may be performed.

The processor 110 is a component for controlling the overall operation of the medical image analysis apparatus. For example, the processor 110 may control a plurality of hardware or software components connected to the processor 110 by driving an operating system and an application, and may perform various data processing and operations. The processor 110 may be a CPU or a GPU or both. The processor 110 may be implemented with at least one general-purpose processor, a digital signal processor, an ASIC, an SoC, or MICOM.

The processor 110 may train an analysis model stored in the memory 120. Specifically, the processor 110 performs initial learning of learning an analysis model using the first medical image for which labeling has been completed.

Further, the processor 110 calculates the uncertainty of each frame of the second medical image by applying the learned analysis model to the second medical image before labeling. Here, the first medical image refers to an image in which labeling has been performed, and the second medical image refers to an image in a state in which labeling is not performed unlike the first medical image. Here, the frame means one medical image belonging to the second medical image set. That is, a frame according to an embodiment of the present invention may be one brain MR image.

The processor 110 may select at least one frame exceeding a preset condition according to the uncertainty of each frame of the second medical image. Here, the condition may be a threshold of uncertainty and the number of frames to be selected. For example, if the threshold of uncertainty is α and the number of frames is k, the processor 110 may select the upper k frames among frames having uncertainties exceeding the α value. Conditions for selection may be variously set according to the use environment.

The processor 110 requests the user to label the selected frame. The labeling request method can be implemented in various ways. For example, when the learning device 100 has a display on its own, the processor 110 displays a UI for labeling input on the display. This will be described in detail in a part to be described later. Meanwhile, if the learning device 100 does not have its own display and is communicatively connected with an external display device, the processor 110 transmits UI data for labeling input to the external display device and receives labeling through the device. May be.

When labeling is input, the processor 110 generates a third medical image by adding the frame to which the labeling is input to the first medical image. That is, the third medical image refers to an image obtained by adding a newly labeled frame to the existing first medical image.

The processor 110 learns an analysis model using the third medical image.

The operation of the processor 110 may be repeatedly performed as many times as desired by the user. If the learning model is repeatedly trained, the accuracy of labeling can be increased.

As described above, when the learning device 100 itself is implemented as an analysis device, the processor 110 may analyze the medical image using the learned analysis model.

In FIG. 1, although one processor 110 is provided, a plurality of processors may be used to process the learning of an analysis model and analysis using the same. For example, an existing general-purpose processor (eg, CPU) may be used for artificial intelligence processing, or a single purpose processor (eg, GPU, FPGA, ASIC, etc.) may be used.

Meanwhile, when the learning device 100 is implemented as an independent device separate from the analysis device, an image analysis operation based on the analysis model may be performed by an external analysis device.

2 is a diagram showing a first medical image (a), a second medical image (b), and a third medical image (c) according to an embodiment of the present invention. The medical image according to an embodiment of the present invention may be a brain MR image, but is not limited thereto. In the case of the brain MR image, since it is a three-dimensional image, a three-dimensional frame is expressed in the shape of a cube in FIG. 2.

The first medical image (a) refers to a set of frames for which labeling has been completed. Here, labeling means checking a lesion in a medical image. Therefore, the first medical image (a) can be viewed as a set of frames in which the lesion portion is checked. In FIG. 2, the lesion portion was checked by marking a dot on the brain MR image. The processor 110 learns the analysis model based on the first medical image (a).

The second medical image (b) refers to a set of frames before labeling. That is, it means a set of frames in which the lesion portion is not checked. The processor 110 calculates the uncertainty of each frame of the second medical image (b) by applying the second medical image (b) to the analysis model learned based on the first medical image (a). The processor 110 selects at least one frame whose uncertainty exceeds a preset condition, and requests input of labeling for the selected medical medical image.

A third medical image is generated by adding a frame to which labeling is input to the first medical image. In FIG. 2, the last frame of the third medical image c is added. Accordingly, the third medical image (c) is a set obtained by adding a frame received labeling among the second medical image (b) to the existing first medical image (a).

In order to learn a segmentation model according to an embodiment of the present invention, a U-NET, which has recently shown high performance in various medical image segmentation problems, may be used. However, it is not limited thereto, and other models may also be used. In case of using U-NET, the output of the learning model becomes a prediction map that predicts the presence or absence of a lesion. In a regionalization problem, this probability map value is usually binarized to a lesion if it is greater than 0.5 and as a background if it is less than 0.5. However, the method of measuring uncertainty according to an embodiment of the present invention measures the uncertainty by calculating entropy based on a probability map without performing the binarization operation as described above. There are several ways to calculate the uncertainty. A method of calculating the sum of the entropy of each voxel of the probability map as the uncertainty of the image may be used.

Voxel is a volume element and represents a value of a regular grid unit in a three-dimensional space. The term voxel is a hybrid word that combines volume and pixels, and is a metaphor for displaying two-dimensional image data as pixels.

When the uncertainty of a frame is defined as the sum of the entropy of each voxel in the probability map as described above, when the number of voxels in the frame is N, each voxel is n, and the probability value that the voxel is the region of interest is defined as p_n, the uncertainty H is the following formula: It can be defined as

In other words, the higher the entropy, the more ambiguous whether the lesion is in the frame, and the lower the entropy, the more obvious it is.

As described above, the uncertainty of each frame of the second medical image is calculated, and at least one frame of the second medical image in which the uncertainty exceeds a preset condition is selected. In this case, since the reference value of uncertainty is changed for each data and every update, the preset condition may be changed by the user to a preset condition such that k frames having the highest uncertainty are selected. Also, k can be set by the user, and k can be increased when there is a lot of time or resources. In this case, if the number of k is increased and then updated, it will be advantageous to improve the performance of the model, but the time to label the lesion area for multiple data will increase in proportion to the number of data.

Among the second medical images, at least one frame in which the uncertainty exceeds a preset condition may be selected, and labeling of a lesion region for the selected frame may be input. In this case, the labeling input of the lesion site can be made on the display as described in FIG. 3.

3 is a block diagram illustrating a configuration of a learning device 100 further including a display 330 in the learning device 100 according to an embodiment of the present invention.

The display 330 is a component for displaying a medical image frame, and may be implemented as, for example, a liquid crystal display (LCD), and in some cases, a cathode-ray tube (CRT), a plasma display panel (PDP), an organic light emitting diodes), TOLED (transparent OLED), etc. The display 330 may be implemented in the form of a touch screen capable of detecting a user's touch manipulation.

A UI (User Interface) requesting a labeling input for at least one frame selected as the uncertainty exceeding a preset condition among the second medical images may be displayed on the display.

4 is a diagram illustrating a 3D medical image 400 divided into a 2D image for labeling a lesion. Depending on the size of the image, it can be divided into tens or hundreds of 2D images. The processor 110 divides the medical image into divided 2D image units and displays it on the display 330. In this case, the processor 110 may display a message requesting to select a lesion site from each 2D image and a method of selecting the lesion site together. In this case, the processor 110 may use software such as ITK-SNAP that allows labeling while passing over a 2D image for labeling a lesion, but is not limited thereto.

5 is a diagram illustrating an example of a UI configuration displayed according to an embodiment of the present invention.

The processor 110 may display a guide message requesting a lesion check together with a 2D image on the display 330 (520). In Fig. 5(a), a 2D first image 410 is displayed on the display. When the user completes the lesion labeling for the first two-dimensional image 410 displayed on the display as shown in Fig. 5(b), it moves to the next chapter (click on the 530 icon) as shown in Fig. When the lesion labeling is performed on the image 420 and labeling of each frame is completed, each 2D image is merged into one medical image frame to complete labeling of the medical image frame. After generating a third medical image by adding the labeled frame to the first medical image, the learning model may be retrained using the third medical image.

6 is a flowchart illustrating a method of learning an analysis model for medical images according to an embodiment of the present invention.

6, the learning device first learns an analysis model using a first medical image (S601). In this case, the learning device can learn an initial analysis model using a small number of first medical image data for which labeling has been completed. have.

Existing supervised learning techniques that detect lesions from brain MR images usually learn a model with dozens to hundreds of frames and labeling. Therefore, since it takes a long time to create hundreds of labels, in order to efficiently proceed with the operation, in the case of the present invention, the learning device may learn an initial analysis model using a small number of first medical image frames. After that, the learning device calculates the uncertainty of each frame of the second medical image by applying the initial analysis model to the second medical image before labeling (S602). And the learning device calculates the uncertainty of each frame of the second medical image (S602). At least one frame is selected. (S603) The learning device receives labeling for at least one selected frame. (S604) In this case, the labeling may be an activity in which the user checks the lesion area in the frame. The learning apparatus generates a third medical image by adding the labeled frame to the first medical image (S605). The processing ends by the learning apparatus re-learning the analysis model using the third medical image. S606)

The learning method described in FIG. 6 may be performed by the learning device 100 having the configuration illustrated in FIG. 1, but is not limited thereto. That is, the learning method of FIG. 6 may be implemented by a medical image analysis apparatus having a different configuration from that of FIG. 1.

7 shows an analysis result of analyzing 20 brain MRI image data using the analysis model learned by the learning method according to the above-described embodiment. Specifically, in the case of FIG. 7, a table comparing the domainization performance between learning models made from randomly selected data (Random) and data selected by the proposed technique (Entropy). In the case of FIG. 8, a table comparing the domainization performance of the random selection technique and the proposed selection technique (active learning).

For the experiment, 20 data were randomly classified into 2 first medical images, 13 second medical images, and 5 test data frames. It is assumed that the first two medical image frames have labeling and are used to learn the initial segmentation model, and the second medical image frame 13 has no labeling. For comparison, a method of randomly selecting data and updating the model and comparative experiments were conducted. For the method proposed by the present invention, one frame having the highest entropy among 13 medical image frames was selected based on the initial segmentation model, and the learning model was updated by adding labeling of this data. Afterwards, for evaluation of the updated model, the domainization accuracy was calculated for 5 test data frames. In addition, the accuracy was calculated for the test data frame by randomly selecting a data frame and updating the model in the same manner. The accuracy of each segmentation model was calculated through the Dice Similarity coefficient (DSC) measurement method. The number of updates of the learning model was set to a total of 6, and the experiment was repeated until 6 frames out of 13 frames of the second medical image without labeling were added to the training data.

As a result of the experiment, as shown in FIG. 8, when the learning model according to an embodiment of the present invention is used to update the learning model using randomly selected data, about 2 to 5% domainization accuracy is improved. Was able to confirm.

In the above, various embodiments related to a method of learning a medical image analysis apparatus and a learning model for medical image analysis have been described using flowcharts and block diagrams.

The method of learning a learning model for medical image analysis according to various embodiments as described above may be stored and distributed in a recording medium in the form of a program code for performing the method. Specifically, when executed by an electronic device, a program code for learning a learning model for analyzing a medical image using the first medical image and the second medical image may be distributed in a state stored in a recording medium or distributed online.

Here, the program code includes the step of learning an analysis model using the first medical image for which labeling has been completed, and the uncertainty of each frame of the second medical image by applying the learned analysis model to the second medical image before labeling. Calculating; Selecting at least one frame from among the second medical images in which the uncertainty exceeds a preset condition; When labeling for the selected frame is input, generating a third medical image by adding the label inputted frame to the first medical image; And a code for sequentially executing the step of learning the analysis model by using the third medical image.

A device in which a recording medium in which such a program code is stored is mounted may perform operations according to various embodiments described above.

The recording medium may be various types of computer-readable media such as ROM, RAM, memory chips, memory cards, external hard drives, hard drives, CDs, DVDs, magnetic disks or magnetic tapes.

In addition, a device that has downloaded such a program code from an online provider may also perform various operations described above.

Although the present invention has been described with reference to the accompanying drawings, the scope of the present invention is determined by the claims to be described later, and should not be construed as being limited to the above-described embodiments and/or drawings. And it should be clearly understood that the improvements, changes and modifications of the invention described in the claims, which are obvious to those skilled in the art, are included in the scope of the present invention.

100 learning unit 110 processor
120 memory 330 display

Claims (7)

In a learning device for learning an analysis model of a medical image,
A memory storing the analysis model; And
Includes; a processor that analyzes a medical image using the analysis model,
The processor,
The analysis model is trained using the first medical image for which labeling is completed, and the learned analysis model is applied to the second medical image before labeling to generate a probability map of the second medical image, and the probability map is used. To calculate the uncertainty of each frame of the second medical image,
Selecting at least one frame in which the uncertainty exceeds a preset condition among the second medical images, and receiving labeling for the selected frame,
After generating a third medical image by adding the frame to which the labeling has been input to the first medical image, learning the analysis model using the third medical image,
The uncertainty is a learning device, characterized in that the sum of entropy of each voxel of the probability map.
delete The method of claim 1,
It further includes a display;
The processor,
Displaying a UI (User Interface) requesting a labeling input for at least one frame selected as the uncertainty exceeds a preset condition among the second medical images on the display,
When a lesion portion of each of the at least one frame is checked on the UI, a learning device for labeling the at least one frame by adding data on the checked portion to the data of each frame. In the learning method of an analysis model for medical images in which each step is performed by a learning device implemented as a computer,
Learning an analysis model using the first medical image for which labeling has been completed;
Generating a probability map of the second medical image by applying the learned analysis model to a second medical image before labeling;
Calculating an uncertainty of each frame of the second medical image using the probability map;
Selecting at least one frame from among the second medical images in which the uncertainty exceeds a preset condition;
When labeling for the selected frame is input, generating a third medical image by adding the frame to which the labeling is input to the first medical image; And
Including; learning the analysis model by using the third medical image,
The uncertainty is the sum of entropy of each voxel of the probability map.
delete The method of claim 4,
Generating the third medical image,
Displaying a UI (User Interface) for requesting a labeling input for at least one frame selected as the uncertainty exceeds a preset condition among the second medical images;
When a lesion portion of each of the at least one frame is checked on the UI, labeling the at least one frame by adding data on the checked portion to the data of each frame;
And generating the third medical image by adding the labeled frame to the first medical image. In a computer-readable recording medium storing a program for executing a learning method for learning an analysis model for medical images by a computer,
The above learning method,
Learning an analysis model using the first medical image for which labeling has been completed;
Generating a probability map of the second medical image by applying the learned analysis model to a second medical image before labeling;
Calculating an uncertainty of each frame of the second medical image using the probability map;
Selecting at least one frame from among the second medical images in which the uncertainty exceeds a preset condition;
When labeling for the selected frame is input, generating a third medical image by adding the frame to which the labeling is input to the first medical image; And
Including; learning the analysis model by using the third medical image,
The uncertainty is the sum of entropy of each voxel of the probability map.

Images