KR102328198B1 - Method and apparatus for measuring volume of organ using artificial neural network - Google Patents

Abstract

A method and apparatus for measuring the volume of an organ are provided. The organ volume measurement method according to an embodiment of the present invention acquires a plurality of images and imaging metadata of the organs, and pre-processes the plurality of images to obtain a plurality of image patches of a specified size Step, inputting the plurality of image patches into a 3D Convolutional Neural Network (CNN)-based neural network model to estimate an organ region corresponding to each of the plurality of image patches, the area of the estimated organ region and the imaging meta Measuring the volume of the organ using data, measuring the uncertainty value of the neural network model and the uncertainty value of a plurality of images based on the estimation result of the neural network model, based on the uncertainty value of the plurality of images changing at least one image among a plurality of images; and changing a labeling policy of the neural network model based on an uncertainty value of the neural network model.

Description

Method and device for measuring organ volume using artificial neural network

The present invention relates to a method and apparatus for measuring the volume of an organ using an artificial neural network. More particularly, it relates to a method and apparatus for detecting an organ region through an artificial neural network using a 3D image of the organ, and measuring the volume of the organ based on the detected organ region.

Accurately measuring the volume of the kidney and liver is a very important step for pathophysiology, analysis of treatment mechanisms, and evaluation of the effects of therapeutic agents. In particular, the polycystic kidney is characterized by anatomical and morphological inconsistency, so it is very difficult for non-professionals to understand the shape of the organ. Performing work on all of the hundreds of CT image slides requires a lot of hard labor (repetition of simple labor) of doctors for a very long time. Accordingly, although the development of open surgery for this problem is being actively carried out through multi-center studies at global units such as the HALT-PKD cohort, CRISP, and SUISSE study, the scientific reliability of the organ volume itself measured by each institution is currently 86%. stay at the level.

SUMMARY Embodiments of the present invention provide a method and an apparatus for accurately detecting an organ region using an image acquired in a hospital and accurately measuring the volume of the organ.

The organ volume measurement method according to an embodiment of the present invention acquires a plurality of images and imaging metadata of the organs, and pre-processes the plurality of images to obtain a plurality of image patches of a specified size Step, inputting the plurality of image patches into a 3D Convolutional Neural Network (CNN)-based neural network model to estimate an organ region corresponding to each of the plurality of image patches, the area of the estimated organ region and the imaging meta Measuring the volume of the organ using data, measuring the uncertainty value of the neural network model and the uncertainty value of a plurality of images based on the estimation result of the neural network model, based on the uncertainty value of the plurality of images It may include changing at least one image among a plurality of images and changing a labeling policy of the neural network model based on an uncertainty value of the neural network model.

In one embodiment, the plurality of images of the organs include a CT image obtained from a DICOM (Digital Imaging and Communications in Medicine) file and a labeling image of the organs, and the imaging metadata includes the It may include pixel spacing data for each of the plurality of images and depth data of the image.

In an embodiment, the obtaining of the plurality of image patches includes performing data augmentation on a first image included in the plurality of images to generate a plurality of images from the first image, and Pre-processing the generated plurality of images to obtain a plurality of image patches, wherein the data augmentation may include one or more of spatial enlargement, color enhancement, noise enhancement, and cropping of the image. .

In an embodiment, the plurality of images are a plurality of 3D images obtained by photographing the organs, and the obtaining of the plurality of image patches includes sliding the plurality of 3D images in a depth direction and having a specified size. It may include obtaining the plurality of image patches.

In one embodiment, the neural network model performs dropout in the learning step and the inference step, and the uncertainty value of the neural network model is a variance value for the probability distribution of the result data in the inference step of the neural network model. can be measured on the basis of

In an embodiment, the uncertainty value of the plurality of images may be measured based on a variance estimate of the result data in the inference step of the neural network model.

In an embodiment, the changing of at least one image among the plurality of images based on the uncertainty values of the plurality of images may include: detecting one or more images in which the uncertainty values of the plurality of images are equal to or greater than a reference value; and changing the detected image based on a user input to the organ region of the image.

In an embodiment, the method may further include setting weights of a plurality of images according to the changed labeling policy, and training the neural network model by giving the changed image a greater weight than the image before the change.

An apparatus for measuring the volume of an organ according to another embodiment of the present invention includes a processor, wherein the processor obtains a plurality of images of the organs and imaging metadata, and pre-processes the plurality of images to obtain a plurality of images of a specified size. Obtaining an image patch, inputting the plurality of image patches into a 3D convolutional neural network (CNN)-based neural network model to estimate an organ region corresponding to each of the plurality of image patches, and Measuring the volume of the organ using the area and the imaging metadata, measuring the uncertainty value of the neural network model and the uncertainty value of a plurality of images based on the estimation result of the neural network model, and measuring the uncertainty value of the plurality of images Based on at least one image among the plurality of images, the labeling policy of the neural network model may be changed based on the uncertainty value of the neural network model.

1 is a diagram illustrating an example of a polycystic kidney discrimination system including an organ volume measurement device according to an embodiment of the present invention.
2 is a hardware configuration diagram for explaining the internal configuration of an organ volume measuring apparatus for measuring the volume of an organ of an object according to an embodiment of the present invention.
Figure 3 is a hardware block diagram for explaining the configuration and operation of the organ volume measuring apparatus according to an embodiment of the present invention.
4 is a flowchart of a method for measuring the volume of an organ according to an embodiment of the present invention.
5 is a diagram illustrating an example of neural network input data according to an embodiment of the present invention.
6 to 8 are diagrams for explaining a method of pre-processing input data according to an embodiment of the present invention.
9 to 10 are diagrams for explaining the structure of an artificial neural network according to an embodiment of the present invention.
11 is a view for explaining a method of measuring the volume of an organ using a detected organ region according to an embodiment of the present invention.
12 is a diagram for explaining a method of updating input data according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0023] Reference is made to the accompanying drawings, which show by way of illustration specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein may be implemented with changes from one embodiment to another without departing from the spirit and scope of the present invention. In addition, it should be understood that the location or arrangement of individual components within each embodiment may be changed without departing from the spirit and scope of the present invention. Accordingly, the following detailed description is not to be taken in a limiting sense, and the scope of the present invention should be taken as encompassing the scope of the claims and all equivalents thereto. In the drawings, like reference numerals refer to the same or similar elements throughout the various aspects. Artificial intelligence (AI) disclosed in the present invention may refer to a field researching artificial intelligence or a methodology that can make it, and machine learning is a field of computing as a field of artificial intelligence technology. It may be an algorithm that allows a device to learn from data to understand a specific object or condition, or it may be an algorithm that enables a computer to analyze data as a technical way to find and classify patterns in data. Machine learning disclosed in the present invention may be understood as meaning including an operation method for learning an artificial intelligence model.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings in order to enable those of ordinary skill in the art to easily practice the present invention.

Hereinafter, a polycystic kidney identification system including an organ volume measuring device according to an embodiment of the present invention will be described in detail with reference to FIG. 1 .

The apparatus 200 for measuring organ volume according to an embodiment may use a plurality of images 110 obtained by photographing an organ and a plurality of images 120 indicating an organ region as input data 100 . The organ volume measurement apparatus 200 may receive the input data 100 from an external server through the network 10 . In this case, the external server is, for example, a hospital server, and the organ volume measurement apparatus 200 may acquire the input data 100 that is a standard medical image format from the hospital server. In this case, the input data 100 including a plurality of images of the organs included in the input data 100 may be in a DICOM (Digital Imaging and Communications in Medicine) format and include a CT image, patient information, and measurement. It may include data and photographing information. However, this is only an example, and it goes without saying that the form of the input data 100 according to some embodiments of the present invention may be in the form of various image files such as JPG, PNG, and TIF. In an exemplary embodiment, the input data 100 may include 3D image data, and in this case, each voxel constituting the volume of the image may be in the form of an arrangement of pixels. Also, the plurality of images 120 indicating the organ region included in the input data 100 may be correct answer data indicating the organ region to be detected using an artificial neural network. That is, the plurality of images 120 indicating the organ region may be annotation data (or labeling data) for the organ region. An artificial neural network that detects a long-term region according to some embodiments of the present invention using the aforementioned input data 100 may perform supervised learning.

The organ volume measuring apparatus 200 receiving the input data 100 may detect an organ region included in each image using a 3D CNN-based artificial neural network. By using a 3D CNN-based rather than a 2D CNN, accurate analysis of CT data acquired in the form of a three-dimensional image becomes possible. However, an additional pre-processing process may be performed by a person skilled in the art to perform the organ volume measurement method according to some embodiments of the present invention using a 2D CNN-based neural network. In more detail, the organ volume measurement apparatus 200 obtains a plurality of images and photographing metadata of an organ, pre-processes the plurality of images to obtain a plurality of image patches of a specified size, and a plurality of patches By inputting an image into a 3D Convolutional Neural Network (CNN)-based neural network model, the organ region corresponding to each of the plurality of image patches is estimated, and the organ volume is obtained using the estimated area of the organ region and the imaging metadata. measure, measure the uncertainty value of the neural network model and the uncertainty value of the plurality of images based on the estimation result of the neural network model, and change at least one image among the plurality of images based on the uncertainty value of the input data, The labeling policy of the neural network model may be changed based on the uncertainty value of the model.

Also, according to an embodiment, a 2D image or a 3D model 300 corresponding to an organ may be generated and displayed on the display by using the data on the detected organ region. In this case, data on an organ region detected in response to a user input to the 2D image or 3D model 300 displayed on the display may be corrected, and the corrected data may be used again as input data of an artificial neural network.

Thereafter, the polycystic kidney determining apparatus 400 may measure polycystic kidney using volume data of the organ region. However, this is only an application example when the detected organ region is the kidney or liver, and it goes without saying that the disease or symptom to be discriminated may vary depending on the detected organ region according to some embodiments of the present invention.

Hereinafter, the internal configuration of the organ volume measuring apparatus 200 will be described in detail with reference to FIG. 2 .

In an embodiment, the organ volume measuring apparatus 200 for measuring the volume of an organ may include an input/output interface 201 , a memory 202 , a processor 203 , and a communication module 204 . The memory 202 is a computer-readable recording medium and may include a random access memory (RAM), a read only memory (ROM), and a permanent mass storage device such as a disk drive. In addition, the memory 202 may temporarily or permanently store program codes and settings for controlling the organ volume measurement apparatus 200 , a camera image, and pose data of an object.

The processor 203 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. Instructions may be provided to the processor 203 by the memory 202 or the communication module 204 . For example, the processor 203 may be configured to execute instructions received according to program code stored in a recording device, such as the memory 202 .

The communication module 204 may provide a function for communicating with an external server through the network 10 . The above-described external server may be, for example, a hospital server that provides medical images. For example, a request generated by the processor 203 of the organ volume measurement device 200 according to a program code stored in a recording device such as the memory 202 is externally transmitted through the network 10 under the control of the communication module 204 . can be sent to the server. Conversely, a control signal, command, content, file, etc. provided under the control of the processor of the external server may be received by the organ volume measurement apparatus 200 through the network 10 through the communication module 204 . For example, a control signal or command of an external server received through the communication module 204 may be transmitted to the processor 203 or the memory 202 , and the content or file may be transmitted by the organ volume measurement device 200 further. It may be stored in a storage medium that may include. In addition, although the communication method of the communication module 204 is not limited, the network 10 may be a local area wireless communication network. For example, the network 10 may be a Bluetooth (Bluetooth), BLE (Bluetooth Low Energy), or Wifi communication network.

Also, the input/output interface 201 may receive a user input and display output data. The input/output interface 201 according to an embodiment may display a 2D image or a 3D model corresponding to the organ on the display. Alternatively, feedback information may be input to the detected organ region from the user.

Although not shown, the organ volume measuring apparatus 200 according to an embodiment may further include a camera module. The camera module may be a camera module including one or more individual cameras. For example, the camera module may be a camera module built into the organ volume measurement apparatus 200 or a module connected to a separately provided camera device. In this case, the image of the organ acquired from the camera module may be stored in the memory 202 or a storage medium further included in the organ volume measurement device 200 .

Also, in other embodiments, the organ volume measurement apparatus 200 may include more components than those of FIG. 2 . However, there is no need to clearly show most of the prior art components. For example, the organ volume measurement apparatus 200 may include a battery and a charging device for supplying power to internal components of the user terminal, and is implemented to include at least some of the above-described input/output devices or a transceiver (transceiver). ), a Global Positioning System (GPS) module, various sensors, and other components such as a database may be further included.

Figure 3 is a hardware block diagram for explaining the configuration and operation of the organ volume measuring apparatus according to an embodiment of the present invention.

In one embodiment, the organ volume measuring apparatus 200 includes a data acquisition unit 210 , a data preprocessor 211 , an artificial neural network 220 , an uncertainty measurement unit 230 , an artificial neural network update unit 240 , and an organ volume It may include an operation unit 250 .

In an embodiment, the data acquisition unit 210 may acquire a plurality of images of organs and imaging metadata. The plurality of images of the organs may include a CT image obtained from a Digital Imaging and Communications in Medicine (DICOM) file and a labeling image of the organ. However, this is only an example, and it goes without saying that the form of the input data 100 according to some embodiments of the present invention may be in the form of various image files such as JPG, PNG, and TIF. The labeling image is an image in which the organ region is displayed, and may be correct answer data for the organ region to be estimated through the artificial neural network. Accordingly, the artificial neural network for estimating the organ region according to some embodiments of the present invention may perform supervised learning using the above-described labeling data.

In an embodiment, the data preprocessor 211 may perform data augmentation to secure sufficient data required for learning of the artificial neural network. Also, according to an embodiment, the data preprocessor 211 may preprocess a plurality of images to obtain a plurality of image patches of a specified size. In an embodiment, the data preprocessor 211 for increasing the learning data may perform at least one of spatial augmentation, color augmentation, noise augmentation, and cropping. . Also, the data preprocessor 211 for generating a plurality of image patches may perform image re-sizing for generating a plurality of image patches having the same size and extracting an image patch suitable for an artificial neural network. Also, in an embodiment, the data preprocessor 211 may acquire a plurality of image patches by sliding in a depth direction with respect to the 3D CT image.

In an embodiment, the artificial neural network unit 220 may include a neural network learning unit 221 and a long-term region inference unit 222 . In an embodiment, the artificial neural network unit 220 may perform a learning step and an inference step using an artificial neural network based on a 3D CNN (Convolitional Neural Network). In an embodiment, the neural network learning unit 221 may perform the above-described learning of the artificial neural network using an image patch generated based on the plurality of CT images and the plurality of labeling images acquired by the data acquiring unit 210 . . Also, in an embodiment, the organ region inference unit 222 may infer an organ region corresponding to each image patch by using a plurality of image patches generated based on a plurality of CT images.

Meanwhile, the aforementioned 3D CNN-based artificial neural network according to some embodiments of the present invention may perform dropout in order to prevent an overfitting problem. According to an embodiment, the above-described artificial neural network may perform dropout in both the learning stage and the inference stage. In this case, the artificial neural network may randomly perform an inference step based on a sample of a probability distribution.

In an embodiment, the uncertainty measurement unit 230 may measure the uncertainty value of the input data and the artificial neural network model. The uncertainty value of the input data and the artificial neural network model is a number indicating whether a specific cause of the input data and the artificial neural network model in which an error has occurred can be explored. Therefore, the uncertainty measuring unit 230 according to an embodiment may measure the uncertainty of the neural network model due to a lack of data or a structure in which it is difficult to analyze all data. In this case, the uncertainty measurement unit 230 may measure the uncertainty value of the neural network based on the variance value of the probability distribution of the result data in the inference step of the artificial neural network model described above. In addition, the uncertainty measuring unit 230 according to an embodiment may measure the uncertainty of data due to the case in which different labeling is performed despite having the same uncertainty value due to the random data. In this case, the uncertainty measuring unit 230 may measure the uncertainty values of the plurality of images based on the estimated variance of the result data in the inference step of the artificial neural network model described above.

In an embodiment, the artificial neural network updater 240 may correct the organ region estimated for each of the plurality of images by the organ region inference unit 222 . In more detail, the artificial neural network updater 240 may correct the organ region detected in the image based on the uncertainty of the artificial neural network model. In this case, the organ volume measuring apparatus 200 may obtain information on the corrected organ region from the user using the input/output interface.

Also, the artificial neural network updater 240 may change the labeling policy of the aforementioned artificial neural network model based on the uncertainty value of the input data. In addition, the artificial neural network updater 240 sets a weight for each of the plurality of images according to the changed labeling policy, and assigns a greater weight to the image in which the estimated long-term region is corrected than other images, thereby providing training data for the artificial neural network. is available as

In an embodiment, the organ volume calculating unit 250 estimates the organ volume by using the area of the organ region estimated using the artificial neural network unit 220 for each of the plurality of image patches and imaging metadata for the plurality of images. can do.

4 is a flowchart of a method for measuring the volume of an organ according to an embodiment of the present invention.

In step S110, the organ volume measuring apparatus may acquire a plurality of images of organs and imaging metadata.

The plurality of images of the organs may include a CT image obtained from a Digital Imaging and Communications in Medicine (DICOM) file and a labeling image of the organ. However, this is only an example, and it goes without saying that the form of the input data 100 according to some embodiments of the present invention may be in the form of various image files such as JPG, PNG, and TIF. In addition, the photographing metadata may include pixel spacing data for each of the plurality of images and depth data of the image slice. In an embodiment, the plurality of images described above may be a plurality of 3D images obtained by photographing the organs. The labeling image is an image in which the organ region is displayed, and may be correct answer data for the organ region to be estimated through the artificial neural network. Accordingly, the artificial neural network for estimating the organ region according to some embodiments of the present invention may perform supervised learning using the above-described labeling data. A more detailed description will be given later with reference to FIG. 5 .

In step S120, the organ volume measurement apparatus may pre-process a plurality of images to obtain a plurality of image patches of a specified size. In more detail, the device volume measuring apparatus may perform data augmentation on each of a plurality of images to generate a plurality of images, and preprocess the plurality of images to obtain a plurality of image patches. In an embodiment, the device volume measuring device may perform data augmentation when training data is insufficient. In this case, the organ volume measuring apparatus may perform one or more of space enlargement, color enhancement, noise enhancement, and cropping. In an embodiment, the spatial expansion may include at least one of mirroring for each of a plurality of images, channel transformation for registration error simulation, elastic deformation, rotation, scaling, and resampling. However, it should be noted that this is only an example of space expansion, and embodiments according to the present invention are not limited thereto. In an embodiment, the color enhancement may include at least one of adding brightness, multiplying brightness, adjusting contrast, and adjusting gamma. However, it should be noted that this is only an example of color enhancement, and embodiments according to the present invention are not limited thereto. In an embodiment, the noise enhancement may use at least one of Gaussian noise and Rician noise. However, it should be noted that this is only an example of noise enhancement, and embodiments according to the present invention are not limited thereto. In an embodiment, the cropping may be one of random cropping and center cropping. However, it should be noted that this is only an example of cropping, and embodiments according to the present invention are not limited thereto.

In addition, the organ volume measuring apparatus may perform image re-sizing to generate a plurality of image patches having the same size and image patch extraction in a form suitable for an artificial neural network. Also, in an embodiment, the apparatus for measuring organ volume may acquire a plurality of image patches by sliding in a depth direction with respect to the 3D CT image. Also, the organ volume measuring apparatus may generate a plurality of image patches by overlapping a predetermined patch size by a predetermined ratio. In addition, the organ volume measuring apparatus may add padding for generating an image patch of a specified size to prevent the size of data from being reduced through a convolution operation. For example, the organ volume measurement device may add symmetric padding for width and/or height, and add zero padding for depth to fit the input data shape of a 3D CNN-based artificial neural network. have. In this case, the added padding data may be '0' data. A more detailed description will be given later with reference to FIGS. 6 to 8 .

In step S130, the organ volume measurement apparatus may input a plurality of patch images to a 3D convolutional neural network (CNN)-based neural network model. The neural network model may perform dropout in the learning phase and the inference phase in order to prevent the occurrence of an overfitting problem. Also, the neural network model according to an embodiment may include an encoding layer including a reduction cell and a decoding layer including an expansion cell. A more detailed description will be given later with reference to FIGS. 9 to 10 .

In step S140 , the organ volume measuring apparatus may estimate an organ region corresponding to each of the plurality of image patches using a 3D CNN-based neural network. According to an embodiment, the artificial neural network for estimating the organ region may estimate the organ region for each of a plurality of image patches. In addition, the organ volume measurement apparatus may further perform data post-processing on the result data. For example, the organ volume measuring apparatus may perform at least one of depth spline, depth stitching, and unpadding. In an embodiment, the organ volume measuring apparatus may perform a depth spline on the result data to give a lower inference weight as the distance from the center of each image patch increases. Through this, a natural stitching result can be obtained using the result data. Also, according to an embodiment, the apparatus for measuring organ volume may configure a new volume by performing depth stitching on result data corresponding to each of the plurality of image patches. And in one embodiment, the organ volume measuring apparatus may perform un-padding for controlling the padding added in the data pre-processing step.

In operation S150, the organ volume measuring apparatus may measure the organ volume using the estimated area of the organ region and imaging metadata. In more detail, the organ volume measuring apparatus may calculate the organ volume by using the pixel interval included in the imaging metadata, the thickness of the image slice, and the number of voxels included in the image patch. A more detailed description will be given later with reference to FIG. 11 .

In step S160 , the apparatus for measuring the organ volume may measure the uncertainty value of the neural network model and the uncertainty value of each of the plurality of image patches based on the estimation result of the neural network model. In an embodiment, the uncertainty value of the neural network model may be measured based on a variance value of the probability distribution of the result data in the inference step of the neural network model. In addition, the uncertainty value of each of the plurality of image patches may be measured based on a variance estimate of the result data in the inference step of the neural network model.

In step S170, the organ volume measuring apparatus may change at least one image among a plurality of images based on the uncertainty value of the artificial neural network model. The long-term volume measurement device can select data that needs to be corrected based on the uncertainty value of the model. In addition, the long-term volume measurement device may establish a label policy based on the uncertainty value of the data. Through this, the neural network according to some embodiments of the present invention may perform neural network model update (active learning) using the corrected data. That is, the organ volume measuring apparatus may detect one or more images having an uncertainty value of input data equal to or greater than a reference value, and may change the detected image based on a user input to the organ region of the detected image.

In step S180, the organ volume measurement device may change the labeling policy of the neural network model based on the uncertainty value of the input data. The apparatus for measuring organ volume according to an embodiment performs labeling on each input data according to the changed labeling policy, and assigns a greater weight to the input data in which the label is corrected than the input data in which the label is not corrected, and the neural network model can be learned

5 is a diagram illustrating an example of neural network input data according to an embodiment of the present invention.

(a) is image data obtained by photographing organs, and a plurality of image data as in (a) may be obtained from a DICOM (Digital Imaging and Communications in Medicine) file. However, this is only an example, and it goes without saying that the form of the input data 100 according to some embodiments of the present invention may be in the form of various image files such as JPG, PNG, and TIF. Also, in an embodiment, the apparatus for measuring organ volume may further acquire imaging metadata for each of the plurality of images from the DYCOM file. The photographing metadata may further include, for example, the size of each pixel of the plurality of images, information about the depth of the image, and information about the size and number of voxels.

(b) is data indicating the organ region. (b) is correct answer data for a long-term region to be inferred through an artificial neural network, and may be labeling data in which a long-term region is indicated. The neural network model according to an embodiment may perform supervised learning using the labeling data as shown in (b). However, it goes without saying that the neural network model for detecting the organ region in another embodiment may perform unsupervised learning using only the image as shown in (a).

6 to 9 are diagrams for explaining a method of pre-processing input data according to an embodiment of the present invention.

Referring to FIG. 6 , in an embodiment, the apparatus for measuring organ volume may perform data augmentation in order to secure a sufficient amount of learning data. The organ volume measurement device may perform one or more of spatial enlargement, color enhancement, noise enhancement, and cropping. In an embodiment, the spatial expansion may include at least one of mirroring for each of a plurality of images, channel transformation for registration error simulation, elastic deformation, rotation, scaling, and resampling. In an embodiment, the color enhancement may include at least one of adding brightness, multiplying brightness, adjusting contrast, and adjusting gamma. In an embodiment, the noise enhancement may use at least one of Gaussian noise and Rician noise. In an embodiment, the cropping may be one of random cropping and center cropping. For example, the image shown in (b) is an example of a case in which rotation is performed with respect to (a), and the image shown in (c) is an example of a case where the brightness of (a) is lowered, and shown in (d) The cropped image is an example of center cropping. As described above, the apparatus for measuring organ volume according to an embodiment may acquire a sufficient amount of training data required for an artificial neural network by transforming the acquired image in various ways.

In addition, referring to FIG. 7 , in an embodiment, the apparatus for measuring organ volume may perform a data preprocessing process for obtaining an image patch having a specified size. First, the device volume measuring device may extract an image patch at the same scale by adjusting the sizes of a plurality of images. Thereafter, the device volume measuring device may extract a 3D image of a specified size by scanning in one direction through the 3D sliding window. In an embodiment, the organ volume measuring apparatus may extract a plurality of image patches by overlapping a specified ratio of a specified patch size. For example, (b) and (c) are examples of the 3D image patch extracted from (a).

Referring to FIG. 8 , the apparatus for measuring organ volume according to an embodiment may add padding to input data in order to adjust the size of result data of the artificial neural network. The artificial neural network for estimating the organ region according to some embodiments of the present invention may be a 3D CNN-based model. Therefore, as the operation in the neural network is performed, a plurality of convolutional operations are performed on the input data, and the size of the result data is inevitably reduced. Therefore, the organ volume measuring apparatus may add padding to each of width, height, and depth in order to obtain an image patch having a specified size. For example, FIG. (b) is an example of data to which padding for depth is added with respect to (a). Without being limited thereto, the image patch to which padding is added according to some embodiments of the present invention may be an image in which a specific image is symmetrically duplicated. According to an embodiment, when padding of a specified size is added, the added data may be '0'. For example, padding in the depth direction may be added to a 2D image to be used as input data of a 3D CNN-based neural network, and in this case, padding having a specified depth may consist of '0' data. In another embodiment, the size of the added padding may be '0'. As described above, since the neural network model according to some embodiments of the present invention uses a 3D image as input data, formal depth information may be added to the input data, and in this case, the depth may be '0'.

9 to 10 are diagrams for explaining the structure of an artificial neural network according to an embodiment of the present invention.

Referring to FIG. 9 , the neural network 500 receiving image patches 501a and 501b generated based on a plurality of images may extract organ regions 502a and 502b corresponding to each image patch. . For example, the organ region 502a corresponding to the first image patch 501a may be detected based on the first image patch 501a, and the second image patch 501b may be detected based on the second image patch 501b. ) may be extracted from the organ region 502b.

Also, in an embodiment, the apparatus for measuring organ volume may measure uncertainty values for the neural network model and input data. The uncertainty of the model arises from the lack of data or the neural network with a structure that makes it difficult to represent all of the data. Accordingly, the artificial neural network model according to some embodiments of the present invention may perform dropout in the learning phase and the inference phase in order to prevent the overfitting problem from occurring. In more detail, the apparatus for measuring organ volume may perform random prediction based on a sample of a probability distribution, and the aforementioned random prediction may be performed by sampling k thinned neural nets and using an average value of each predicted value. In this case, the probability distribution may be the probability distribution of the result data obtained in the inference stage of the neural network model, the average value at this time may be obtained as a predicted value, and the variance value at this time may be regarded as an uncertainty value.

The uncertainty of the input data is caused by the randomness that occurs in the data generation process. In more detail, for example, when the uncertainty values of the first data and the second data included in the randomness data that are difficult to express with probability are the same, even though the uncertainty values of the above-described first data and the second data are the same, the Since the first data and the second data are not data obtained according to a specified probability, different labels may be applied. When the uncertainty of the input data occurs as described above, the apparatus for measuring the organ volume according to some embodiments of the present invention may measure the uncertainty value of the above-described input data through variance estimation.

10 is an exemplary diagram of the structure of an artificial neural network model according to an embodiment of the present invention.

In an embodiment, the neural network model 500 for extracting the organ region is a 3D CNN-based neural network model, and may include an encoding layer 510 including a reduction cell and a decoding layer 520 including an expansion cell. However, it should be noted that the structure of the neural network model according to some embodiments of the present invention is not limited to that shown in FIG. 10 , and may have various structures that can be adopted and modified by those skilled in the art. In addition, the neural network model according to an embodiment of the present invention is not limited thereto, and an additional preprocessing process is performed by a person skilled in the art to perform the organ volume measurement method according to some embodiments of the present invention using a 2D CNN-based neural network. might do

11 is a view for explaining a method of measuring the volume of an organ using a detected organ region according to an embodiment of the present invention.

According to an exemplary embodiment, the volume of an organ to be photographed may be measured using result data of a neural network for detecting an organ region. The organ volume measuring apparatus may measure the organ volume using the estimated area of the organ region and imaging metadata. In more detail, the organ volume measuring apparatus may calculate the organ volume by using the pixel interval, the thickness of the image slice, and the number of voxels included in the image patch included in the imaging metadata. 11, for example, assuming that the spacing (or size) of square-shaped pixels is 0.6 mm, the thickness of the image slice is 1 mm, and the number of voxels is 2,000,000, in this case, the The volume is 0.6*0.6*1*2,000,000 = 0.36mm ² * 2,000,000 = 720ml. As another example, the x-coordinate of the voxel

12 is a diagram for explaining a method of updating input data according to an embodiment of the present invention.

According to an embodiment, a 3D model 602 of a corresponding organ region may be generated using data 502 on a plurality of organ regions estimated using a plurality of image patches. In an embodiment, the organ volume measurement device may display data 502 of the organ region or a 3D model 602 of the organ region described above. Thereafter, the organ volume measuring apparatus may change at least one image among the plurality of images based on the uncertainty value of the artificial neural network model. That is, the long-term volume measuring apparatus may select the data 601 , 602 , and 603 that need to be corrected based on the uncertainty value of the model. That is, the organ volume measuring apparatus may detect one or more images having an uncertainty value of input data equal to or greater than a reference value, and may change the detected image based on a user input to the organ region of the detected image. In addition, the organ volume measurement apparatus may change the labeling policy of the neural network model based on the uncertainty value of the input data. The apparatus for measuring organ volume according to an embodiment performs labeling on each input data according to the changed labeling policy, and assigns a greater weight to the input data in which the label is corrected than the input data in which the label is not corrected, and the neural network model can be learned Through this, the neural network according to some embodiments of the present invention may perform neural network model update (active learning) based on the modified data and the new labeling policy.

In addition, the apparatus for measuring organ volume according to an embodiment of the present invention may verify the effect of a drug using the above-described method for measuring organ volume. For example, a diagnosis of polycystic kidney may be performed by measuring the volume of a patient's kidney and/or liver, and a therapeutic effect according to drug administration may also be determined. In more detail, first, the patient's kidney volume segmentation may be performed using the artificial neural network for estimating the above-described organ region. Then, after receiving the patient data before and after dosing and performing kidney volume segmentation (kidney volume segmentation), the volume of the kidney is measured. In this case, it goes without saying that the volume of the kidney can be measured using the size of the pixel and the thickness of the image slice included in the dicom file for the kidney region of the patient. The drug efficacy can be measured based on the volume increase/decrease rate of the kidney measured according to the above-described method. Furthermore, it goes without saying that the apparatus for measuring organ volume according to an embodiment of the present invention can provide an interface for checking the change trend of organ volume according to time for each individual.

The device described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA). , a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

The software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and carry out program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible by those skilled in the art from the above description. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc., are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (10)

A method for measuring the volume of an organ performed by a computing device, the method comprising:
obtaining a plurality of images of the organs and photographing metadata, and pre-processing the plurality of images to obtain a plurality of image patches of a specified size;
estimating an organ region corresponding to each of the plurality of image patches by inputting the plurality of image patches to a 3D convolutional neural network (CNN)-based neural network model;
measuring the volume of the organ using the estimated area of the organ region and the imaging metadata;
measuring an uncertainty value of the neural network model and an uncertainty value of a plurality of images based on the estimation result of the neural network model;
changing at least one image among the plurality of images based on the uncertainty values of the plurality of images; and
Including; changing the labeling policy of the neural network model based on the uncertainty value of the neural network model;
Changing at least one image among the plurality of images based on the uncertainty value of the plurality of images comprises:
detecting one or more images in which uncertainty values of the plurality of images are greater than or equal to a reference value;
changing the detected image based on a user input for the organ region of the detected image,
Methods for measuring the volume of organs.
According to claim 1,
A plurality of images of the organs are
Contains an image obtained from a DICOM (Digital Imaging and Communications in Medicine) file and a labeling image for the organ,
The shooting metadata is
including pixel spacing data for each of the plurality of images and depth data of the image,
Methods for measuring the volume of organs.
According to claim 1,
The step of obtaining the plurality of image patches comprises:
Performing data augmentation on the first image included in the plurality of images, generating a plurality of images from the first image, and pre-processing the generated plurality of images to obtain a plurality of image patches comprising steps,
The data augmentation is
comprising one or more of spatial enlargement, color enhancement, noise enhancement and cropping of the image;
Methods for measuring the volume of organs.
According to claim 1,
The plurality of images,
A plurality of 3D images of the organs,
The step of obtaining the plurality of image patches comprises:
Comprising the step of sliding in a depth direction with respect to the plurality of 3D images and obtaining the plurality of image patches of a specified size,
Methods for measuring the volume of organs.
According to claim 1,
The neural network model is
Dropout is performed in the training phase and inference phase,
The uncertainty value of the neural network model is,
Measured based on the variance value for the probability distribution of the result data in the inference step of the neural network model,
Methods for measuring the volume of organs.
6. The method of claim 5,
The uncertainty value of the plurality of images is,
Measured based on the variance estimate of the result data in the reasoning step of the neural network model,
Methods for measuring the volume of organs.
delete According to claim 1,
Setting the weights of a plurality of images according to the changed labeling policy, further comprising the step of training the neural network model by giving the changed image a weight greater than that of the image before the change,
Methods for measuring the volume of organs.
processor; including;
The processor is
Obtaining a plurality of images of organs and imaging metadata, preprocessing the plurality of images to obtain a plurality of image patches of a specified size, and converting the plurality of image patches to a 3D Convolutional Neural Network (CNN)-based Input to the neural network model to estimate the organ region corresponding to each of the plurality of image patches, measure the organ volume using the estimated area of the organ region and the imaging metadata, and calculate the estimation result of the neural network model Measure the uncertainty value of the neural network model and the uncertainty value of a plurality of images based on the basis, detect one or more images in which the uncertainty value of the plurality of images is equal to or greater than a reference value, and based on the user input for the organ region of the detected image , changing the detected image, and changing the labeling policy of the neural network model based on the uncertainty value of the neural network model,
A device for measuring the volume of an organ.
A computer program stored in a recording medium for executing the method of any one of claims 1 to 6 and 8 using a computer.

Images