KR20230062208A - Method and apparatus for sorting medical image for labeling - Google Patents

Abstract

Provided are a method and a device for selecting medical images for labeling according to an embodiment of the present invention. The above method comprises: a step of obtaining a plurality of medical images of a target area of a subject from an imaging device; a step of obtaining prediction result data representing the result of predicting the at least one region from the plurality of medical images using a prediction model learned to predict at least one region corresponding to the target region based on the plurality of medical images; a step of calculating uncertainty data regarding the obtained prediction result data; and a step of selecting a medical image for labeling from the plurality of medical images based on the calculated uncertainty data. Accordingly, the present invention can provide improved prediction results of an active learning-based model.

Description

Method and apparatus for sorting medical images for labeling {METHOD AND APPARATUS FOR SORTING MEDICAL IMAGE FOR LABELING}

The present invention relates to a method and apparatus for selecting medical images for labeling.

In general, imaging medicine tests (e.g., X-ray, ultrasound, CT (Computer Tomography), Angiography, Positron Emission Tomography (PET)) are performed to determine whether or not there is a disease in the target area of the subject. -CT) or MRI (Magnetic Resonance Imaging) examination, etc.) is used to obtain medical images of the target part of the subject.

A specialist can visually check the obtained medical image to identify a target region, thereby confirming the target region and determining whether the target region is a disease or not.

Identification of the target area through the naked eye is highly reliable and accurate because noise exists in the medical image itself due to the performance of the imaging device and/or the motion of the subject, so differences in opinion may occur depending on the skill or experience of specialists. Diagnostic results can be difficult to provide.

Accordingly, a method of predicting a target region from an input medical image using an artificial neural network trained to predict a target region based on a medical image is widely used.

However, there is a problem that the accuracy and reliability of the prediction result are lowered because the medical image itself has noise or there is uncertainty about the prediction result of the predictive model due to the small number of training data.

Accordingly, a method for improving the accuracy and reliability of prediction results in consideration of the above uncertainty is required.

The inventors of the present invention have recognized the fact that, when a specialist visually checks a medical image to determine a target region region, there is uncertainty about the determined region.

In addition, the inventors of the present invention recognized that when using an existing predictive model, if the predictive model is a model trained using a small number of training data, uncertainty about the prediction result exists.

Accordingly, an object to be solved by the present invention is to provide an active learning-based method and apparatus for selecting medical images for labeling.

Specifically, an object to be solved by the present invention is to provide a method and apparatus for selecting medical images for labeling used for model learning in order to improve the performance of an artificial neural network model in consideration of uncertainty in prediction results. .

The tasks of the present invention are not limited to the tasks mentioned above, and other tasks not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the above problems, a method and apparatus for selecting a medical image for labeling are provided.

A method for selecting medical images for labeling according to an embodiment of the present invention includes acquiring a plurality of medical images of a target part of an examinee from a photographing device; obtaining prediction result data indicating a result of predicting at least one region from a plurality of medical images by using a prediction model learned to predict at least one region corresponding to a target region based on a plurality of medical images; Calculating uncertainty data about the obtained prediction result data; and selecting a medical image for labeling from a plurality of medical images based on the calculated uncertainty data.

Uncertainty data according to an embodiment of the present invention includes Aleatoric uncertainty data, and includes both the inherent uncertainty data and epistemic uncertainty data.

The step of selecting a medical image for labeling according to an embodiment of the present invention may mean a step of determining a medical image related to epistemological uncertainty data as a medical image for labeling.

The target site according to an embodiment of the present invention includes the heart, and at least one region includes a left atrium, a left ventricle, a right atrium, and a right ventricle.

A prediction model according to an embodiment of the present invention is configured using a prediction distribution for model parameters of a probability model learned to predict at least one region based on at least one medical image of each of a plurality of subjects.

A probabilistic model according to an embodiment of the present invention may be based on a Bayesian neural network.

A predictive model according to an embodiment of the present invention is configured such that a predictive distribution of model parameters calculated by a probability model is reflected in a layer for estimating a class of at least one region.

A predictive model according to an embodiment of the present invention is composed of u-net, and a predictive distribution of model parameters may be applied to the last layer of the predictive model.

The step of calculating uncertainty data according to an embodiment of the present invention may refer to a step of estimating a variance value of a prediction distribution for uncertainty of prediction result data.

The estimated variance value according to an embodiment of the present invention includes a first value for intrinsic uncertainty and a second value for epistemological uncertainty.

The step of selecting a medical image for labeling according to an embodiment of the present invention may refer to a step of determining at least one medical image having a second value equal to or greater than a preset first threshold as a medical image for labeling. can

In the step of selecting medical images for labeling according to an embodiment of the present invention, when arranging in order from the image having the largest value to the image having the smallest value among at least one medical image corresponding to a first threshold or higher, the maximum value and the image This may refer to a step of determining a medical image in which a difference of is smaller than a preset second threshold as the medical image for the labeling.

An apparatus for selecting medical images for labeling according to an embodiment of the present invention includes a communication unit configured to transmit and receive data; and a control unit configured to be connected to the communication unit, wherein the control unit acquires a plurality of medical images obtained by capturing a target part of the subject from the photographing device through the communication unit, and obtains at least one medical image corresponding to the target part based on the plurality of medical images. Obtaining prediction result data representing a result of predicting at least one area from a plurality of medical images by using a prediction model learned to predict the area, calculating uncertainty data about the obtained prediction result data, and calculating the uncertainty data It is configured to select a medical image for labeling from a plurality of medical images based on.

The controller according to an embodiment of the present invention is configured to determine a medical image including the epistemological uncertainty data as a medical image for labeling.

The control unit according to an embodiment of the present invention is configured to estimate a variance value of a prediction distribution for uncertainty of prediction result data.

The control unit according to an embodiment of the present invention is configured to determine, as the medical image for labeling, a medical image having a second value equal to or greater than a preset threshold.

The control unit according to an embodiment of the present invention may set a difference from the maximum value when the image having the maximum value to the image having the minimum value among at least one medical image corresponding to the first threshold value or higher is arranged in order. and determine a medical image smaller than the value as a medical image for labeling.

Other embodiment specifics are included in the detailed description and drawings.

The present invention can provide an improved prediction result of an active learning-based model by selecting an input image for labeling from an input image for model learning based on uncertainty data.

In addition, the present invention selects a medical image with large epistemic uncertainty among input images for active learning as a medical image for labeling, labels the selected medical image, and learns a predictive model using the selected medical image as an input, thereby improving model performance. , can provide prediction results with improved accuracy and reliability.

In addition, the present invention can minimize human resources and time required to predict each region of the target site using a predictive model.

Effects according to the present invention are not limited by the contents exemplified above, and more various effects are included in the present specification.

1 is a schematic diagram for explaining a system according to an embodiment of the present invention.
2 is a schematic block diagram of an electronic device according to an embodiment of the present invention.
3 is an exemplary diagram for explaining a method of selecting a medical image for labeling using an artificial neural network model according to an embodiment of the present invention.
4 is an exemplary diagram illustrating a predictive model according to an embodiment of the present invention.
5 is a flowchart illustrating a method for selecting medical images for labeling according to an embodiment of the present invention.
6 and 7 are exemplary diagrams illustrating results of quantifying uncertainty of an input image according to an embodiment of the present invention.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and will be implemented in various forms different from each other, only these embodiments make the disclosure of the present invention complete, and common knowledge in the art to which the present invention pertains. It is provided to completely inform the person who has the scope of the invention, and the present invention is only defined by the scope of the claims. In connection with the description of the drawings, like reference numerals may be used for like elements.

In this document, expressions such as "has," "may have," "includes," or "may include" indicate the existence of a corresponding feature (eg, numerical value, function, operation, or component such as a part). , which does not preclude the existence of additional features.

In this document, expressions such as “A or B,” “at least one of A and/and B,” or “one or more of A or/and B” may include all possible combinations of the items listed together. . For example, “A or B,” “at least one of A and B,” or “at least one of A or B” (1) includes at least one A, (2) includes at least one B, Or (3) may refer to all cases including at least one A and at least one B.

Expressions such as “first,” “second,” “first,” or “second,” used in this document may modify various elements, regardless of order and/or importance, and refer to one element as It is used only to distinguish it from other components and does not limit the corresponding components. For example, a first user device and a second user device may represent different user devices regardless of order or importance. For example, without departing from the scope of rights described in this document, a first element may be named a second element, and similarly, the second element may also be renamed to the first element.

A component (e.g., a first component) is "(operatively or communicatively) coupled with/to" another component (e.g., a second component); When referred to as "connected to", it should be understood that the certain component may be directly connected to the other component or connected through another component (eg, a third component). On the other hand, when an element (eg, a first element) is referred to as being “directly connected” or “directly connected” to another element (eg, a second element), the element and the above It may be understood that other components (eg, third components) do not exist between the other components.

As used in this document, the expression "configured to" means "suitable for," "having the capacity to," depending on the circumstances. ," "designed to," "adapted to," "made to," or "capable of." The term "configured (or set) to" may not necessarily mean only "specifically designed to" hardware. Instead, in some contexts, the phrase "device configured to" may mean that the device is "capable of" in conjunction with other devices or components. For example, the phrase "a processor configured (or configured) to perform A, B, and C" may include a dedicated processor (e.g., embedded processor) to perform those operations, or by executing one or more software programs stored in a memory device. , may mean a general-purpose processor (eg, CPU or application processor) capable of performing corresponding operations.

Terms used in this document are only used to describe a specific embodiment, and may not be intended to limit the scope of other embodiments. Singular expressions may include plural expressions unless the context clearly dictates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by a person of ordinary skill in the art described in this document. Among the terms used in this document, terms defined in a general dictionary may be interpreted as having the same or similar meaning as the meaning in the context of the related art, and unless explicitly defined in this document, an ideal or excessively formal meaning. not be interpreted as In some cases, even terms defined in this document cannot be interpreted to exclude the embodiments of this document.

Each feature of the various embodiments of the present invention can be partially or entirely combined or combined with each other, and as those skilled in the art can fully understand, various interlocking and driving operations are possible, and each embodiment can be implemented independently of each other. It may be possible to implement together in an association relationship.

In this specification, an image may be a still image and/or a video, but is not limited thereto, and may be a 2D image or a 3D image (ie, a 3D volume image). there is.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a schematic diagram for explaining a system according to an embodiment of the present invention.

Referring to FIG. 1 , a system 100 is a system for providing a result of predicting at least one region corresponding to a target region based on a medical image of a target region of a subject. Here, the medical image is an image taken of the target area of the subject, such as an x-ray image, an ultrasound image, a CT (Computer Tomography) image, an angiography image, and a positron emission tomography (PET- It may be a CT) image, a Magnetic Resonance Imaging (MRI) image, etc., but is not limited thereto. The target site is a specific body part of the subject, such as the heart, lungs, bronchus, liver, skin, ureter, urethra, testicles, vagina, anus, laryngeal gland, ovary, thyroid gland, esophagus, nasopharyngeal gland, pituitary gland, salivary gland, prostate, pancreas, adrenals, lymph nodes, spleen, brain, varicose veins, musculoskeletal system, paranasal sinuses, spinal cord, kidneys, mouth, lips, throat, oral cavity, nasal cavity, small intestine, colon, parathyroid gland, gallbladder, head and neck, breast, bone, bile duct, cervix, heart, and / or may be the hypopharyngeal gland, etc., but is not limited thereto.

First, the imaging device 110 includes an X-ray imaging device, an ultrasound imaging device, a CT imaging device, an angiography imaging device, a positron emission tomography device, and/or an MRI device for providing a plurality of medical images of a target area of a subject. device or the like.

Next, the electronic device 120 may be at least one of a tablet PC, a laptop computer, and/or a PC for selecting an image for labeling from a plurality of medical images provided from the photographing device 110 .

In detail, the electronic device 120 estimates uncertainty based on a prediction result of predicting at least one region of the subject's target region from a plurality of medical images, and uses the plurality of medical images for labeling based on the estimated uncertainty. Medical images can be selected.

The electronic device 120 may use at least one artificial neural network model or algorithm to predict at least one region of the target region and obtain uncertainty data for the prediction result, but is not limited thereto.

Furthermore, the electronic device 120 may predict a lesion for at least one region of the target site and acquire uncertainty data for the predicted lesion, but is not limited thereto.

In the presented embodiment, the case where the photographing device 110 and the electronic device 120 are implemented as separate devices has been described, but is not limited thereto, and the photographing device 110 and the electronic device 120 may be implemented as one device. may be

In the following, the electronic device 120 will be described in more detail with reference to FIG. 2 .

2 is a schematic block diagram of an electronic device according to an embodiment of the present invention.

Referring to FIG. 2 , the electronic device 200 includes a communication unit 210, a storage unit 220, a display unit 230, and a control unit 240. In various embodiments, the display unit 230 may be selectively provided. In the presented embodiment, the electronic device 200 may mean the electronic device 120 of FIG. 1 .

The communication unit 210 connects the electronic device 200 to enable communication with an external device. The communication unit 210 may be connected to the photographing device 110 using wired/wireless communication to receive at least one medical image of the subject.

The storage unit 220 estimates uncertainty of a result of predicting at least one region of the subject's target region based on a plurality of medical images, and selects various data for labeling based on the estimated uncertainty. can be saved.

In various embodiments, the storage unit 220 may be a flash memory type, a hard disk type, a multimedia card micro type, or a card type memory (eg SD or XD). memory, etc.), RAM (Random Access Memory, RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory, ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory) , a magnetic memory, a magnetic disk, and an optical disk may include at least one type of storage medium. The electronic device 200 may operate in relation to a web storage that performs the storage function of the storage unit 220 on the Internet.

The display unit 230 may display various contents to the user. For example, the display unit 230 may display at least one medical image or prediction result data.

In various embodiments, the display unit 230 may include a touch screen, and for example, a touch using an electronic pen or a part of the user's body, a gesture, a proximity, a drag, or a swipe A swipe or hovering input may be received.

The control unit 240 is operatively connected to the communication unit 210, the storage unit 220, and the display unit 230, and results of predicting at least one region of the subject's target area based on a plurality of medical images of the subject. It is possible to estimate an uncertainty for , and perform various commands for selecting a medical image for labeling based on the estimated uncertainty.

The control unit 240 includes at least one of a central processing unit (CPU), a graphics processing unit (GPU), an application processor (AP), a digital signal processing unit (DSP), an arithmetic logic unit (ALU), and an artificial neural network processor (NPU). It can be configured to include.

Specifically, the control unit 240 obtains at least one medical image of a target part of the subject from the photographing device 110 through the communication unit 210, and obtains at least one medical image corresponding to the target part based on the at least one medical image. Prediction result data obtained by predicting at least one region of a target region from at least one medical image by using a prediction model learned to predict at least one region is acquired. Furthermore, the controller 240 may calculate uncertainty data related to the obtained prediction result data, and select a medical image for labeling from a plurality of medical images based on the calculated uncertainty data. Here, the prediction result data may include a segmented image including a mask region obtained by segmenting at least one region predicted by the prediction model, and furthermore, the class of each predicted region and the predicted segmented image. At least one of accuracies for each region may be further included.

In general, an image may include noise generated by a photographing device itself or motion. When an image including such noise is input to a predictive model, a predictive result may be inaccurate, and thus the reliability of the predictive model may decrease. This can be called Aleatoric uncertainty about prediction.

In addition, the prediction result of the predictive model may be inaccurate due to the small number of reference images for training the predictive model or the parameters of the model, and thus the reliability of the predictive model may decrease. This can be called epistemic uncertainty about prediction.

The noise of the image mentioned above is due to the inherent randomness of the image, so it can be partially reduced by data pre-processing, but in the case of medical images taken through ultrasound imaging devices, CT imaging devices, MRI imaging devices There is a limit to completely removing noise. In particular, the heart is composed of four chambers (left atrium, right atrium, left ventricle, and right ventricle), and between the chambers there are valves that open and close appropriately for blood communication. Since the thickness of such a heart plate is very thin, a medical image may have high noise or a blurred part of the plate may be captured when taking a picture of the plate.

However, since the epistemological uncertainty of the parameters of the predictive model can be reduced through the collection and learning of additional training data, a method for minimizing the epistemological uncertainty has been required to further improve the accuracy and reliability of the prediction results of the predictive model. .

To this end, if all training data is labeled and a predictive model is learned based on the labeled data, the performance of the predictive model can be greatly improved. However, since the number of training data used is very large, labeling all of them not only requires a large number of experts for labeling, but also increases the time and cost required.

Accordingly, there is a need for a method for improving performance of a prediction model by performing labeling on only a portion of training data and learning a prediction model based on the labeled portion of training data.

To this end, the control unit 240 estimates uncertainty data for prediction result data of the predictive model using an active learning method, and labels a plurality of medical images using epistemological uncertainty data among the estimated uncertainty data. medical images can be selected for

In the following, the operation of the controller 240 described above will be described in detail with reference to FIGS. 3 and 4 .

3 is an exemplary diagram for explaining a method of selecting a medical image for labeling using an artificial neural network model according to an embodiment of the present invention. In the presented embodiment, the target site is described as the heart.

Referring to FIG. 3 , the controller 240 predicts a plurality of regions by using a prediction model 310 trained to predict a plurality of regions corresponding to the heart by inputting a plurality of medical images 300, and predicts result data. (320) can be obtained. Here, the plurality of regions corresponding to the heart may be left atrium, left ventricle, right atrium, and right ventricle.

The predictive model 310 is an artificial neural network model trained to predict a plurality of regions corresponding to the heart based on a plurality of medical images 300, which is pre-learned from a plurality of reference images and a plurality of newly input medical images ( 300) to predict a plurality of regions. Here, the plurality of reference images may be a plurality of medical images obtained by photographing the heart of each of a plurality of subjects. Furthermore, the prediction model 310 may be an artificial neural network model for image segmentation.

As mentioned above, in order for the control unit 240 to determine a medical image to be labeled using epistemological uncertainty data, it is necessary to divide the uncertainty data into intrinsic uncertainty data and epistemological uncertainty data.

To this end, in the predictive model 310 , a prediction distribution for model parameters of a probability model learned to predict at least one region of the target region based on a plurality of reference images may be reflected in at least a part.

A probabilistic model may be constructed based on a Bayesian neural network to calculate the predicted distribution of model parameters, but is not limited thereto.

Specifically, when the Bayesian neural network outputs a prediction result of predicting at least one region for a target region based on a plurality of reference images used for training, the posterior of the model parameters representing the prediction distribution for the model parameters used therein. Compute the posterior predictive distribution. To calculate the posterior predictive distribution, a prior predictive distribution representing the predicted distribution of model parameters and a likelihood predictive distribution representing the predicted distribution of prediction results output using the input image and model parameters are used. do.

However, since it is generally difficult to calculate the posterior distribution, an approximate value can be obtained through variational inference. In other words, the posterior predictive distribution can be approximated by a variational distribution.

To obtain an approximation of the posterior prediction distribution for model parameters, the Bayesian neural network calculates the difference between the variance distribution and the posterior prediction distribution, and minimizes the difference, thereby approximating the variance distribution to the posterior prediction distribution. Here, the difference is called the 'Kullback-Leibler divergence'.

In this way, to minimize the difference between the variance distribution and the posterior prediction distribution, the Bayesian neural network uses a variance lower bound. Bayesian neural networks can minimize the difference between the variance distribution and the posterior prediction distribution by maximizing the lower variance limit. Here, the variation lower bound means ELBO (Evidence Lower Bound).

In this way, the lower limit of the variance is maximized to minimize the difference between the variance distribution and the posterior prediction distribution, and through this, an approximate value for the posterior prediction distribution is obtained, so that the posterior prediction distribution of the model parameters can be calculated.

A predictive model 310 to which the calculated posterior predictive distribution is applied may be generated.

This will be described in detail with reference to FIG. 4 .

4 is an exemplary diagram illustrating a predictive model according to an embodiment of the present invention.

Referring to FIG. 4 , a predictive model 400 is configured based on u-net. This predictive model 400 includes a contraction path 405 and an expansion path 410 .

The contraction path 405 is based on general convolution networks, and includes one or more layers for convolution operations, linear functions, and down-sampling on input values.

The extension path 410 includes one or more layers for a linear function and a convolution operation for up-sampling to increase the size of a feature map.

In the contraction path 405, a 3x3 convolution operation is performed a preset number of times for each contraction step (415), and a downsampling operation is performed (420) for feature map contraction. Through this, the size of the feature map 425 is reduced and the number of channels is increased.

In the extension path 410, a 3x3 convolution operation is performed a predetermined number of times in each extension step to extend the feature map (430), and an up-convolution operation for upsampling is performed (435). Through this, the size of the feature map 440 increases and the number of channels decreases.

Moreover, the feature map at each expansion step of the expansion path 410 is merged with the feature map cropped by the size reduced by the convolution operation at each contraction stage of the contraction path 405 at each expansion step (445). .

The last layer of the extension path 410 is a layer for estimating information of each channel as a class score, and a filter 450 for obtaining uncertainty included in all pixels of a medical image is disposed in this layer. This filter 450 reflects the computed prediction distribution for the model parameters (i.e., posterior prediction distribution) to calculate the uncertainty for each predicted region. This last layer may include one or more layers for the 1x1 convolution operation and activation function (sigmoid) to obtain uncertainty about the prediction result.

Referring back to FIG. 3 , the control unit 240 obtains prediction result data obtained by predicting at least one region of the target region using the prediction model 310 in which the prediction distribution of the model parameters of the probability model is reflected (320). , it is possible to estimate uncertainty data for the obtained prediction result data (330).

In order to estimate the uncertainty data, the control unit 240 estimates the probability that the prediction result data is correct using the Bayesian method mentioned above. Specifically, the control unit 240 may estimate the variance of the prediction distribution for the probability that the prediction result data is correct using Equation 1 below.

는 내재적 불확실성에 관한 제1 값이고,

는 인식론적 불확실성에 관한 제2 값을 의미한다. 이어서, x는 입력 영상이고, y*는 원-핫 인코딩된 카테고리형 출력(one-hot encoded categorical output)이며,

는 변분 분포(variational distribution)를 의미한다. 또한,

는 베이지안 신경망의 출력이고, T는 MC 샘플의 개수를 의미한다.here,

is the first value for the inherent uncertainty,

denotes a second value for epistemological uncertainty. Then, x is the input image, y* is the one-hot encoded categorical output,

means a variational distribution. also,

is the output of the Bayesian neural network, and T is the number of MC samples.

In this way, through the variance of the estimated prediction distribution, the inherent uncertainty data and epistemological uncertainty data of the prediction result data can be distinguished.

The controller 240 selects a medical image for labeling from among a plurality of images based on the variance of the predicted distribution estimated by Equation 1 (340).

Specifically, the controller 240 sets at least one medical image for which the second value of the variance of the predicted distribution estimated by Equation 1 is greater than or equal to a preset first threshold value as a medical image for labeling. can decide In other words, the control unit 240 may label a medical image related to a prediction result having epistemological uncertainty data of a predetermined size or more among uncertainty data for a result predicted by the predictive model 310 based on a plurality of medical images. It can be screened with medical imaging.

Furthermore, the controller 240 determines that the difference from the maximum value is less than a preset second threshold value when the second value is arranged in the order from the image having the maximum value to the image having the minimum value among medical images corresponding to a threshold value or higher. A medical image may be determined as a medical image for labeling.

In this way, among the input images for active learning, medical images with large epistemic uncertainty are selected as medical images for labeling, the selected medical images are labeled, and the predictive model is trained with the selected medical images as inputs, thereby improving model performance, accuracy and It can provide prediction results with improved reliability.

In the following, a method for selecting a medical image for labeling based on a prediction result of a predictive model will be described with reference to FIG. 5 .

5 is a flowchart illustrating a method for selecting medical images for labeling according to an embodiment of the present invention.

Referring to FIG. 5 , the controller 240 acquires a plurality of medical images of a target part of the examinee from the photographing device 110 (S500).

The controller 240 generates prediction result data indicating a result of predicting at least one region from a plurality of medical images using a prediction model learned to predict at least one region corresponding to a target region based on a plurality of medical images. Obtain (S510). The predictive model is composed of u-net and is constructed using a predictive distribution for model parameters of a probability model learned to predict at least one region based on at least one medical image of each of a plurality of subjects. For example, the predictive model may be configured such that a predictive distribution of model parameters of the probability model is reflected in a layer for estimating a class among a plurality of layers constituting the predictive model. The probabilistic model is based on a Bayesian neural network.

Specifically, the control unit 240 uses a predictive model in which the predicted distribution of model parameters of the learned probability model is reflected, and uses a segmented image including a mask region obtained by segmenting each region predicted from a plurality of medical images as prediction result data. Acquire

The controller 240 calculates uncertainty data about the obtained prediction result data (S520), and selects a medical image for labeling from a plurality of medical images based on the calculated uncertainty data (S530). Specifically, the controller 240 may estimate a variance value of the prediction distribution for uncertainty of prediction result data, and select a medical image for labeling using a value related to epistemological uncertainty among the estimated variance values. For example, the control unit 240 may estimate the variance of the prediction distribution for uncertainty using a Bayesian method, and as described above, <Equation 1> may be used.

In this way, the controller 240 selects a medical image with epistemic uncertainty as a medical image for labeling, and labels the selected medical image to learn active learning.

Hereinafter, a method for selecting data for labeling in consideration of the size of uncertainty will be described with reference to FIGS. 6 and 7 .

6 and 7 are exemplary diagrams illustrating results of quantifying uncertainty of an input image according to an embodiment of the present invention.

Referring to FIG. 6 , (a) is a medical image of a heart region. In (a), the border of the heart region is unclear due to the large amount of noise.

(b) is an image showing the inherent uncertainty information of (a) obtained using the above-described Equation 1. The boundary portion 600 of the heart region shown in (b) is expressed as if it were smeared by the noise of the image itself. It can be difficult to completely remove this noise.

(c) is an image representing the epistemological uncertainty information of (a) obtained using the above-described Equation 1. In part 610 shown in (c), noise due to uncertainty that may be generated by a small number of training data or model parameters is expressed. Uncertainty of these predictive models can be an important basis for active learning algorithms.

(d) is a segmented image including a masking region of the heart region predicted by the predictive model.

Specifically, (d) is less accurate at the boundary of the predicted region due to inherent uncertainty, and less accurate at the part 620 due to epistemological uncertainty.

Next, referring to FIG. 7 , (a) is a medical image of a heart region. In (a), the boundary of the heart region is ambiguous and unclear, including a large amount of noise.

(b) is an image showing the inherent uncertainty information of (a) obtained using the above-described Equation 1. The boundary portion 700 of the heart region shown in (b) is also expressed like a blur by the noise of the image itself.

(c) is an image showing the epistemological uncertainty information of (a) obtained using the above-described <Equation 1>. (c) has less noise due to epistemological uncertainty than Fig. 6(c). In other words, the portion 710 has a smaller magnitude (or amount) of epistemological uncertainty compared to 610 in FIG. 6 .

(d) is a segmented image including a masking region of the heart region predicted by the predictive model.

Specifically, (d) shows that the accuracy due to the epistemological uncertainty is higher than that of FIG.

In this way, for labeling, it is more preferable to select and label medical images having the highest uncertainty rather than selecting medical images having low uncertainty for labeling in order to improve the model performance of the predictive model.

Therefore, the present invention selects a medical image corresponding to a predetermined threshold value or more of epistemological uncertainty data as a medical image for labeling, labels the medical image, and trains a predictive model using the labeled medical image as an input, thereby improving model performance. It is possible to provide prediction results with improved accuracy and reliability by improving.

Devices and methods according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in computer readable media. Computer readable media may include program instructions, data files, data structures, etc. alone or in combination.

Program instructions recorded on a computer readable medium may be specially designed and configured for the present invention or may be known and usable to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - Includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media and ROM, RAM, flash memory, etc. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

Although the embodiments of the present invention have been described in more detail with reference to the accompanying drawings, the present invention is not necessarily limited to these embodiments, and may be variously modified and implemented without departing from the technical spirit of the present invention. . Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea of the present invention is not limited by these embodiments. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. The protection scope of the present invention should be construed according to the claims below, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

100: system 110: shooting device
120, 200: electronic device 210: communication unit
220: storage unit 230: display unit
240: control unit 300: medical image
310: prediction model 320: prediction result data
330: Uncertainty data estimation for prediction result data
340: Selecting medical images for labeling

Claims (18)

In the method performed by the control unit,
acquiring a plurality of medical images of a target part of the subject from a photographing device;
Obtaining prediction result data indicating a result of predicting the at least one region from the plurality of medical images by using a prediction model learned to predict at least one region corresponding to the target region based on the plurality of medical images. doing;
Calculating uncertainty data about the obtained prediction result data; and
and selecting a medical image for labeling from the plurality of medical images based on the calculated uncertainty data. The method of claim 1, wherein the uncertainty data,
A method for selecting medical images for labeling that contain Aleatoric uncertainty data, and which contain both the Aleatoric uncertainty data and Epistemic uncertainty data. The method of claim 2, wherein selecting the medical image for labeling comprises:
The method of selecting a medical image for labeling, the step of determining a medical image related to the epistemological uncertainty data as the medical image for labeling. The method of claim 1, wherein the predictive model,
A method for selecting medical images for labeling, the method comprising using a prediction distribution for model parameters of a probabilistic model learned to predict the at least one region based on at least one medical image of each of a plurality of subjects. The method of claim 4, wherein the probability model,
A method for screening medical images for labeling based on a Bayesian neural network. The method of claim 5, wherein the predictive model,
The method for selecting a medical image for labeling, configured to reflect a predictive distribution of model parameters calculated by the probabilistic model to a layer for estimating a class of the at least one region. The method of claim 5, wherein the predictive model,
It consists of u-net,
The method of selecting a medical image for labeling, wherein the predictive distribution of the model parameters is applied to the last layer of the predictive model. The method of claim 5, wherein calculating the uncertainty data comprises:
A step of estimating a variance value of a prediction distribution for uncertainty of the prediction result data;
The estimated variance value includes a first value for intrinsic uncertainty and a second value for epistemological uncertainty,
In the step of selecting the medical image for labeling,
and determining, as the medical image for labeling, at least one medical image having the second value equal to or greater than a preset first threshold. The method of claim 8, wherein selecting the medical image for labeling comprises:
The labeling of a medical image having a difference from the maximum value smaller than a preset second threshold when the image having the maximum value and the image having the minimum value are arranged in the order of at least one medical image corresponding to the first threshold or higher. A method for selecting a medical image for labeling, which is a step of determining as a medical image for . a communication unit configured to transmit and receive data; and
Including a control unit configured to connect with the communication unit,
The control unit,
Obtaining a plurality of medical images of a target part of the subject from a photographing device through the communication unit;
Obtaining prediction result data indicating a result of predicting the at least one region from the plurality of medical images by using a prediction model learned to predict at least one region corresponding to the target region based on the plurality of medical images. do,
Calculate uncertainty data regarding the obtained prediction result data;
An apparatus for selecting a medical image for labeling, configured to select a medical image for labeling from the plurality of medical images based on the calculated uncertainty data. 11. The method of claim 10, wherein the uncertainty data,
An apparatus for selecting a medical image for labeling, comprising Aleatoric uncertainty data, or comprising both the inherent uncertainty data and Epistemic uncertainty data. The method of claim 11, wherein the control unit,
An apparatus for selecting a medical image for labeling, configured to determine a medical image including the epistemological uncertainty data as the medical image for labeling. The method of claim 10, wherein the predictive model,
An apparatus for selecting medical images for labeling, configured using a prediction distribution for model parameters of a probabilistic model learned to predict the at least one region based on at least one medical image of each of a plurality of subjects. The method of claim 13, wherein the probabilistic model,
An apparatus for screening medical images for labeling, based on a Bayesian neural network. The method of claim 14, wherein the predictive model,
The apparatus for selecting medical images for labeling, configured to reflect a predictive distribution of model parameters calculated by the probabilistic model to a layer for estimating a class of the at least one region. The method of claim 14, wherein the predictive model,
It consists of u-net,
Apparatus for selecting a medical image for labeling, wherein the predictive distribution of the model parameters is applied to the last layer of the predictive model. The method of claim 14, wherein the control unit,
configured to estimate a variance value of a prediction distribution for uncertainty of the prediction result data;
The estimated variance value includes a first value for intrinsic uncertainty and a second value for epistemological uncertainty,
The control unit,
The apparatus for selecting a medical image for labeling, configured to determine a medical image for which the second value is greater than or equal to a preset first threshold value as the medical image for labeling. The method of claim 14, wherein the control unit,
The labeling of a medical image having a difference from the maximum value smaller than a preset second threshold when the image having the maximum value and the image having the minimum value are arranged in the order of at least one medical image corresponding to the first threshold or higher. An apparatus for selecting a medical image for labeling, configured to determine a medical image for

Images