KR20230019477A - Method and apparatus for diagnosis of kidney cancer using artificial neural network - Google Patents

Abstract

A method and an apparatus for diagnosing renal cancer are provided. The method for diagnosing renal cancer according to an embodiment of the present invention comprises: a step of acquiring a plurality of images of a cross-section of the kidney and imaging meta-data, and pre-processing the plurality of images to acquire a plurality of image patches of a designated size; a step of inputting the plurality of image patches into a neural network model based on an edge gated convolutional neural network (EG-CNN) to estimate a tumor region corresponding to each of the plurality of image patches; a step of determining whether the tumor is benign or malignant using the estimated tumor region and the imaging meta-data; a step of measuring an uncertainty value of the neural network model and an uncertainty value of the plurality of images based on the estimation results of the neural network model; a step of changing at least one image among the plurality of images based on the uncertainty value of the plurality of images; and a step of changing a labeling policy of the neural network model based on the uncertainty value of the neural network model. The method for diagnosing renal cancer can accurately detect the area of the tumor, and accurately determine whether the tumor is malignant or benign, thereby diagnosing renal cancer.

Description

Method and apparatus for diagnosis of kidney cancer using artificial neural network

The present invention relates to a method and apparatus for diagnosing renal cancer using an artificial neural network.

In accordance with the academic trend of not recommending a biopsy for renal cancer due to the risk of metastasis, surgery is decided only with preoperative radiographs. However, there is a disadvantage in that the predictive power of radiographic examination is relatively low compared to pathology examination, and the result can be confirmed only after the pathology result after surgery. Since surgery is decided based on the diagnosis made in radiographic examination before surgery, nephrectomy is performed for benign diseases, and about 20% of patients undergo unnecessary surgery, and these results are due to the uncertainty of radiographic examination. Therefore, there is a need for an artificial intelligence technology to improve diagnostic prediction capabilities based on CT images.

Under such a technical background, research on a diagnosis method based on CT images is being actively conducted (Patent Registration 10-2020-010661), but it is still incomplete.

Embodiments of the present invention provide a method and apparatus for diagnosing renal cancer by accurately detecting a tumor region using an acquired image and accurately determining whether the tumor is malignant or benign.

In one aspect, in a method for diagnosing kidney cancer performed by a computing device, a plurality of images obtained by photographing a cross section of a kidney and photographic metadata are acquired, and a plurality of image patches ( obtaining a patch); estimating a tumor region corresponding to each of the plurality of image patches by inputting the plurality of image patches to a neural network model based on an edge-gated convolutional neural network (CNN); and determining whether the tumor is malignant or benign using the estimated tumor region and the imaging metadata.

In one embodiment, the plurality of images of the cross section of the kidney include a CT image obtained from a Digital Imaging and Communications in Medicine (DICOM) file and a labeling image of the kidney, and the photographic metadata Including pixel interval data and depth data of the image for each of the plurality of images,

It is to provide a method for diagnosing renal cancer.

In one embodiment, the acquiring of the plurality of image patches may include generating a plurality of images from the first images by performing data augmentation on a first image included in the plurality of images; Preprocessing the generated plurality of images to obtain a plurality of image patches, wherein the data augmentation includes at least one of spatial expansion, color enhancement, noise enhancement, and cropping of the image, It is to provide a method for diagnosing renal cancer.

In one embodiment, the plurality of images are a plurality of 3D images obtained by photographing cross sections of the kidney, and the obtaining of the plurality of image patches may include sliding in a depth direction with respect to the plurality of 3D images. It is to provide a method for diagnosing renal cancer, comprising obtaining the plurality of image patches having a designated size.

In one embodiment, measuring the uncertainty value of the neural network model and the uncertainty value of the plurality of images based on the estimation result of the neural network model; changing at least one image among the plurality of images based on uncertainty values of the plurality of images; and changing a labeling policy of the neural network model based on the uncertainty value of the neural network model.

In one embodiment, the neural network model performs dropout in a learning step and an inference step, and the uncertainty value of the neural network model is a variance value of a probability distribution of resultant data in an inference step of the neural network model. It is to provide a method for diagnosing renal cancer, which is measured based on.

In one embodiment, a method for diagnosing renal cancer is provided in which uncertainty values of the plurality of images are measured based on a variance estimation value of result data in an inference step of the neural network model.

In an embodiment, the changing of at least one image among the plurality of images based on uncertainty values of the plurality of images may include: detecting one or more images having uncertainty values of the plurality of images equal to or greater than a reference value; and changing the detected image based on a user input for the tumor region of the detected image, and providing a method for diagnosing renal cancer.

In one embodiment, diagnosis of renal cancer further comprising setting weights of a plurality of images according to the changed labeling policy and assigning a greater weight to the changed images than images before the change to train the neural network model. is to provide a way

Another aspect includes a processor, wherein the processor obtains a plurality of images of a cross-section of the kidney and photographic metadata, and pre-processes the plurality of images to obtain a plurality of image patches having a designated size. and inputs the plurality of image patches to an edge-gated convolutional neural network (CNN)-based neural network model to estimate a tumor region corresponding to each of the plurality of image patches, and the estimated tumor region and the photographing metadata determining whether the tumor is benign or malignant by using a tumor, measuring an uncertainty value of a neural network model and uncertainty values of a plurality of images based on an estimation result of the neural network model, and determining whether the tumor is benign or malignant based on the uncertainty values of the plurality of images. It is to provide an apparatus for diagnosing renal cancer, which changes at least one of the images of , and changes a labeling policy of the neural network model based on an uncertainty value of the neural network model.

Another aspect is to provide a computer program stored in a recording medium in order to execute the method of any one of claims 1 to 8 using a computer.

1 is a diagram illustrating an example of a system for determining a tumor including an apparatus for diagnosing renal cancer according to an embodiment.
2 is a hardware configuration diagram illustrating an internal configuration of an apparatus for diagnosing kidney cancer according to an exemplary embodiment.
3 is a hardware block diagram illustrating the configuration and operation of an apparatus for diagnosing kidney cancer according to an exemplary embodiment.
4 is a flow chart of a method for diagnosing renal cancer according to an embodiment.
5 is a diagram illustrating an example of neural network input data according to an embodiment.
6 is a diagram for explaining a method of preprocessing input data according to an exemplary embodiment.
7 is a diagram for explaining the structure of an artificial neural network according to an embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The detailed description of the present invention which follows refers to the accompanying drawings which illustrate, by way of illustration, specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable any person skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention are different from each other but are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented from one embodiment to another without departing from the spirit and scope of the present invention. It should also be understood that the location or arrangement of individual components within each embodiment may be changed without departing from the spirit and scope of the present invention. Therefore, the detailed description to be described later is not performed in a limiting sense, and the scope of the present invention should be taken as encompassing the scope claimed by the claims and all scopes equivalent thereto. Like reference numbers in the drawings indicate the same or similar elements throughout the various aspects. Artificial intelligence (AI) disclosed in the present invention may mean a field of researching artificial intelligence or a methodology for creating it, and machine learning is a field of artificial intelligence technology, and computing It can be an algorithm that allows a device to learn from data to understand a specific object or condition, or a computer to analyze data in a technical way to find and classify patterns in data. Machine learning disclosed in the present invention can be understood as meaning including an operating method for learning an artificial intelligence model.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily practice the present invention.

Referring to FIG. 1 , a benign or malignant tumor determination system including an apparatus for diagnosing renal cancer according to an embodiment of the present invention will be described in detail.

The apparatus 200 for diagnosing kidney cancer according to an embodiment may use a plurality of images 110 obtained by photographing a cross section of a kidney as input data 100 . The renal cancer diagnosis apparatus 200 may receive input data 100 from an external server through the network 10 . In this case, the external server is, for example, a hospital server, and the renal cancer diagnosis apparatus 200 may obtain input data 100 that is a standard medical image format from the hospital server. In this case, the input data 100 including a plurality of images of cross-sections of the kidneys included in the input data 100 may be in a Digital Imaging and Communications in Medicine (DICOM) format, and may include CT images and patient information. , measurement data and photographing information. However, this is only an example, and the form of the input data 100 according to some embodiments of the present invention may be in the form of various image files such as JPG, PNG, TIF, of course. In one embodiment, the input data 100 may include 3D image data, and in this case, each voxel constituting the volume of the image may be in the form of an array of pixels. An artificial neural network that detects a tumor region using the above-described input data 100 according to some embodiments of the present invention may perform supervised learning.

Upon receiving the input data 100, the renal cancer diagnosis apparatus 200 may detect a tumor region included in each image using an artificial neural network based on an edge-gated CNN (EG-CNN). By using EG-CNN as a basis, it is possible to accurately analyze CT data acquired in the form of a 3D image. However, the method for diagnosing renal cancer according to some embodiments of the present invention may be performed using other types of CNN-based neural networks by performing an additional preprocessing process by a person skilled in the art. More specifically, the apparatus for diagnosing kidney cancer 200 acquires a plurality of images of a cross-section of the kidney and photographic metadata, pre-processes the plurality of images to obtain a plurality of image patches having a designated size, , A plurality of patch images are input to a neural network model based on Edge Gated Convolutional Neural Network (EG-CNN), a tumor area corresponding to each of the plurality of image patches is estimated, and the estimated tumor area and the shooting metadata are used. to determine whether the tumor is benign or malignant, measure the uncertainty value of the neural network model and the uncertainty value of a plurality of images based on the estimation result of the neural network model, and at least one of the plurality of images based on the uncertainty value of the input data One image may be changed, and a labeling policy of the neural network model may be changed based on an uncertainty value of the neural network model.

In addition, in one embodiment, a 2D image or 3D model 300 corresponding to the tumor may be generated using data on the detected tumor region and displayed on a display. In this case, data on the detected tumor region may be corrected in response to a user input on the 2D image or 3D model 300 displayed on the display, and the corrected data may be used as input data of the artificial neural network.

The internal configuration of the renal cancer diagnosis apparatus 200 will be described in detail with reference to FIG. 2 .

In one embodiment, the renal cancer diagnosis apparatus 200, which is a device for diagnosing renal cancer, may include an input/output interface 201, a memory 202, a processor 203, and a communication module 204. The memory 202 is a computer-readable recording medium, and may include a random access memory (RAM), a read only memory (ROM), and a permanent mass storage device such as a disk drive. In addition, the memory 202 may temporarily or permanently store program codes and settings for controlling the renal cancer diagnosis apparatus 200 , camera images, and pose data of an object.

The processor 203 may be configured to process commands of a computer program by performing basic arithmetic, logic, and input/output operations. Instructions may be provided to processor 203 by memory 202 or communication module 204 . For example, processor 203 may be configured to execute instructions received according to program codes stored in a recording device such as memory 202 .

The communication module 204 may provide functionality for communicating with an external server via the network 10 . The aforementioned external server may be, for example, a hospital server providing medical images. For example, a request generated by the processor 203 of the renal cancer diagnosis apparatus 200 according to a program code stored in a recording device such as the memory 202 is sent to the outside through the network 10 under the control of the communication module 204. can be forwarded to the server. Conversely, control signals, commands, contents, files, etc. provided under the control of the processor of the external server may be received by the renal cancer diagnosis apparatus 200 through the communication module 204 via the network 10 . For example, a control signal or command from an external server received through the communication module 204 may be transmitted to the processor 203 or memory 202, and content or files may be further transferred to the renal cancer diagnosis apparatus 200. It can be stored in a storage medium that can include. In addition, the communication method of the communication module 204 is not limited, but the network 10 may be a local area wireless communication network. For example, the network 10 may be a Bluetooth, Bluetooth Low Energy (BLE), or Wifi communication network.

Also, the input/output interface 201 may receive a user's input and display output data. The input/output interface 201 according to an embodiment may display a 2D image or a 3D model corresponding to a tumor on a display. Alternatively, feedback information on the detected tumor region may be input from the user.

Although not shown, the apparatus 200 for diagnosing kidney cancer according to an embodiment may further include a camera module. A camera module may be a camera module that includes one or more individual cameras. For example, the camera module may be a camera module built into the renal cancer diagnosis apparatus 200 or may be a module connected to a separately provided camera device. In this case, the image of the cross section of the kidney obtained from the camera module may be stored in the memory 202 or a storage medium further included in the renal cancer diagnosis apparatus 200 .

Also, in other embodiments, the renal cancer diagnosis apparatus 200 may include more components than those shown in FIG. 2 . However, there is no need to clearly show most of the conventional technical components. For example, the renal cancer diagnosis apparatus 200 may include a battery and a charging device for supplying power to internal components of a user terminal, implemented to include at least some of the above-described input/output devices, or a transceiver. ), a Global Positioning System (GPS) module, various sensors, and other components such as a database may be further included.

3 is a hardware block diagram illustrating the configuration and operation of an apparatus for diagnosing kidney cancer according to an exemplary embodiment.

In an embodiment, the renal cancer diagnosis apparatus 200 includes a data acquisition unit 210, a data pre-processing unit 211, an artificial neural network unit 220, an uncertainty measuring unit 230, an artificial neural network updating unit 240, and positive and negative A malignant tumor determination operation unit 250 may be included.

In an embodiment, the data acquisition unit 210 may acquire a plurality of images of a kidney cross-section and photographing metadata. The plurality of images obtained by photographing the cross section of the kidney may include a CT image obtained from a Digital Imaging and Communications in Medicine (DICOM) file and a labeling image of the cross section of the kidney. However, this is only an example, and the form of the input data 100 according to some embodiments of the present invention may be in the form of various image files such as JPG, PNG, TIF, of course. The labeling image is an image in which a tumor region is displayed, and may be correct answer data for a tumor region to be estimated through an artificial neural network. Accordingly, an artificial neural network for estimating a tumor area according to some embodiments of the present invention may perform supervised learning using the aforementioned labeling data.

In one embodiment, the data pre-processing unit 211 may perform data augmentation to secure sufficient data required for learning of the artificial neural network. Also, in one embodiment, the data preprocessing unit 211 may obtain a plurality of image patches having a designated size by preprocessing a plurality of images. In an embodiment, the data pre-processor 211 that augments learning data may perform at least one of spatial augmentation, color augmentation, noise augmentation, and cropping. . In addition, the data pre-processing unit 211 generating a plurality of image patches may perform image re-sizing to generate a plurality of image patches having the same size and image patch extraction in a form suitable for an artificial neural network.

In one embodiment, the artificial neural network unit 220 may include a neural network learning unit 221 and a tumor region inference unit 222 . In one embodiment, the artificial neural network unit 220 may perform a learning step and an inference step using an artificial neural network based on an edge gated convolitional neural network (EG-CNN). In an embodiment, the neural network learning unit 221 may perform the aforementioned artificial neural network learning using an image patch generated based on a plurality of CT images and a plurality of labeling images acquired by the data acquisition unit 210. . Also, in an embodiment, the tumor region inference unit 222 may infer a tumor region corresponding to each image patch using a plurality of image patches generated based on a plurality of CT images.

Meanwhile, the above-described EG-CNN-based artificial neural network according to some embodiments of the present invention may perform dropout to prevent an overfitting problem. According to an embodiment, the above-described artificial neural network may perform dropout in both a learning step and an inference step. At this time, the artificial neural network may randomly perform an inference step based on a sample of a probability distribution.

In one embodiment, the uncertainty measurer 230 may measure uncertainty values of the input data and the artificial neural network model. The uncertainty value of the input data and the artificial neural network model is a value indicating whether a specific cause of the input data and artificial neural network model in which an error occurred can be searched for. Accordingly, the uncertainty measurement unit 230 according to an embodiment may measure the uncertainty value of the neural network model due to lack of data or a structure in which it is difficult to analyze all data. In this case, the uncertainty measurement unit 230 may measure the uncertainty value of the neural network based on the variance value of the probability distribution of result data in the inference step of the artificial neural network model described above. In addition, the uncertainty measurer 230 according to an embodiment may measure the uncertainty value of the data due to the case where different labeling is performed even though the random data has the same uncertainty value. In this case, the uncertainty measurer 230 may measure the uncertainty values of the plurality of images based on the estimated variance of the resulting data in the inference step of the artificial neural network model described above.

In one embodiment, the artificial neural network updating unit 240 may correct the tumor area estimated by the tumor area inference unit 222 for each of a plurality of images. In more detail, the artificial neural network updating unit 240 may correct the tumor region detected in the image based on the uncertainty of the artificial neural network model. In this case, the renal cancer diagnosis apparatus 200 may obtain information on the corrected tumor region from the user using an input/output interface.

In addition, the artificial neural network updating unit 240 may change the above-described labeling policy of the artificial neural network model based on the uncertainty value of the input data. In addition, the artificial neural network updating unit 240 sets a weight for each of a plurality of images according to the changed labeling policy, and assigns a higher weight to an image in which the estimated tumor region is corrected than other images so as to obtain learning data of the artificial neural network. can be used as

In one embodiment, the benign and malignant tumor determination operation unit 250 determines whether a tumor is positive or malignant by using the tumor area estimated using the artificial neural network 220 for each of a plurality of image patches and the metadata for a plurality of images. It can determine malignancy and diagnose kidney cancer.

4 is a flow chart of a method for diagnosing renal cancer according to an embodiment of the present invention.

In step S110, the apparatus for diagnosing kidney cancer may obtain a plurality of images obtained by photographing a cross-section of the kidney and photographing metadata.

The plurality of images obtained by photographing the cross section of the kidney may include a CT image obtained from a Digital Imaging and Communications in Medicine (DICOM) file and a labeling image of the cross section of the kidney. However, this is only an example, and the form of the input data 100 according to some embodiments of the present invention may be in the form of various image files such as JPG, PNG, TIF, of course. Also, the shooting metadata may include pixel interval data for each of a plurality of images and depth data of an image slice.

In one embodiment, the plurality of images described above may be a plurality of 3D images obtained by photographing a cross section of the kidney. The labeling image is an image in which a cross section of a kidney is displayed, and may be correct answer data for a tumor region of a cross section of a kidney to be estimated through an artificial neural network. Accordingly, an artificial neural network for estimating a tumor area according to some embodiments of the present invention may perform supervised learning using the aforementioned labeling data. A more detailed description will be given later with reference to FIG. 5 .

In step S120, the apparatus for diagnosing kidney cancer may obtain a plurality of image patches having a designated size by pre-processing the plurality of images. In more detail, the apparatus for diagnosing kidney cancer may perform data augmentation on each of a plurality of images to generate a plurality of images and pre-process the generated plurality of images to obtain a plurality of image patches. In one embodiment, the apparatus for diagnosing renal cancer may perform data augmentation when learning data is insufficient. In this case, the renal cancer diagnosis apparatus may perform at least one of space expansion, color enhancement, noise enhancement, and cropping. In an embodiment, spatial expansion may include at least one of mirroring for each of a plurality of images, channel conversion for registration error simulation, elastic deformation, rotation, scaling, and resampling. However, note that this is only an example of space expansion, and embodiments according to the present invention are not limited thereto. In an embodiment, color enhancement may include at least one of brightness addition, brightness multiplication, contrast adjustment, and gamma adjustment. However, it should be noted that this is only an example of color enhancement, and embodiments according to the present invention are not limited thereto. In one embodiment, noise enhancement may use at least one of Gaussian noise and Rician noise. However, note that this is only one example of noise enhancement, and embodiments according to the present invention are not limited thereto. Cropping in one embodiment may be one of random cropping and center cropping. However, note that this is only an example of cropping, and embodiments according to the present invention are not limited thereto.

In addition, the apparatus for diagnosing kidney cancer may perform image re-sizing to generate a plurality of image patches having the same size and image patch extraction in a form suitable for an artificial neural network. Also, in one embodiment, the apparatus for diagnosing kidney cancer may obtain a plurality of image patches by sliding in a depth direction with respect to the 3D CT image. Also, the apparatus for diagnosing renal cancer may generate a plurality of image patches by performing overlapping at a specified ratio with respect to a determined patch size. In addition, the apparatus for diagnosing renal cancer may add padding to generate an image patch having a designated size in order to prevent data size from being reduced through a convolution operation. For example, a renal cancer diagnosis device may add symmetric padding for width and/or height, and add zero padding for depth to match the shape of input data of an artificial neural network based on EG CNN. there is. In this case, the added padding data may be '0' data.

In step S130, the apparatus for diagnosing renal cancer may input a plurality of patch images to a neural network model based on an EG convolutional neural network (EG CNN). The neural network model may perform dropout in a learning step and an inference step in order to prevent an overfitting problem from occurring. Also, a neural network model according to an embodiment may include an encoding layer including reduction cells and a decoding layer including expansion cells.

In step S140, the apparatus for diagnosing renal cancer may estimate a tumor region corresponding to each of a plurality of image patches using an EG CNN-based neural network. According to an embodiment, an artificial neural network for estimating a tumor area may estimate a tumor area for each of a plurality of image patches. Also, the apparatus for diagnosing renal cancer may further perform data post-processing on result data. For example, the renal cancer diagnosis apparatus may perform at least one of depth spline, depth stitching, and unpadding. In an embodiment, the apparatus for diagnosing renal cancer may perform a depth spline on the resulting data and assign a lower inference weight as the distance from the center of each image patch increases. Through this, it is possible to obtain a natural stitching result using result data. Also, according to an embodiment, the apparatus for diagnosing kidney cancer may construct a new volume by performing depth stitching on result data corresponding to each of a plurality of image patches. In one embodiment, the renal cancer diagnosis apparatus may perform un-padding to control the padding added in the data pre-processing step.

In step S150, the apparatus for diagnosing renal cancer may determine whether the tumor is malignant or benign using the estimated tumor area and imaging metadata. In more detail, the apparatus for diagnosing kidney cancer may calculate whether the tumor is malignant or benign by using the pixel interval, the thickness of the image slice, and the number of voxels included in the image patch included in the photographing metadata.

In step S160, the apparatus for diagnosing renal cancer may measure an uncertainty value of the neural network model and an uncertainty value of each of the plurality of image patches based on the estimation result of the neural network model. In an embodiment, the uncertainty value of the neural network model may be measured based on a variance value of a probability distribution of result data in an inference step of the neural network model. In addition, the uncertainty value of each of the plurality of image patches may be measured based on a variance estimation value of result data in the inference step of the neural network model.

In step S170, the apparatus for diagnosing kidney cancer may change at least one image among a plurality of images based on the uncertainty value of the artificial neural network model. The renal cancer diagnosis device can select data that needs correction based on the uncertainty value of the model. In addition, the renal cancer diagnosis apparatus may establish a label policy based on the uncertainty value of the data. Through this, the neural network according to some embodiments of the present invention can perform active learning using the modified data. That is, the apparatus for diagnosing renal cancer may detect one or more images in which the uncertainty value of the input data is greater than or equal to a reference value, and change the detected image based on a user input for the tumor region of the detected image.

In step S180, the apparatus for diagnosing renal cancer may change the labeling policy of the neural network model based on the uncertainty value of the input data. An apparatus for diagnosing renal cancer according to an embodiment performs labeling on each input data according to a changed labeling policy, assigns a greater weight to input data with corrected labels than input data without corrected labels, and obtains the neural network model. can be learned.

5 is a diagram showing an example of neural network input data according to an embodiment of the present invention.

(a) is image data of a cross section of a kidney, and a plurality of image data such as (a) can be obtained from a DICOM (Digital Imaging and Communications in Medicine) file. However, this is only an example, and the form of the input data 100 according to some embodiments of the present invention may be in the form of various image files such as JPG, PNG, TIF, of course. In addition, in an embodiment, the apparatus for diagnosing kidney cancer may further obtain photographic metadata for each of a plurality of images from the DICOM file. The photographing metadata may further include, for example, pixel size of each of a plurality of images, depth information of the image, and information on the size and number of voxels.

(b) is data showing the tumor area in green and the kidney area in red. (b) is answer data for a tumor region to be inferred through an artificial neural network, and may be labeling data in which the tumor region is indicated. The neural network model according to an embodiment may perform supervised learning using labeling data as shown in (b). However, it goes without saying that in another embodiment, the neural network model for detecting the tumor region can perform un-supervised learning using only the image shown in (a).

6 is a diagram for explaining a method of pre-processing input data according to an embodiment of the present invention. In one embodiment, the apparatus for diagnosing kidney cancer may perform data augmentation to secure a sufficient amount of learning data. The apparatus for diagnosing kidney cancer may perform one or more of space expansion, color enhancement, noise enhancement, and cropping. In an embodiment, spatial expansion may include at least one of mirroring for each of a plurality of images, channel conversion for registration error simulation, elastic deformation, rotation, scaling, and resampling. In an embodiment, color enhancement may include at least one of brightness addition, brightness multiplication, contrast adjustment, and gamma adjustment. In one embodiment, noise enhancement may use at least one of Gaussian noise and Rician noise. Cropping in one embodiment may be one of random cropping and center cropping. For example, the image shown in (b) is an example of performing rotation with respect to (a), the image shown in (c) is an example of lowering the brightness of (a), and the image shown in (d) The image shown is an example of performing center cropping. As described above, the apparatus for diagnosing kidney cancer according to an embodiment may acquire a sufficient amount of learning data necessary for an artificial neural network by transforming an acquired image in various ways.

7 is a diagram for explaining the structure of an artificial neural network according to an embodiment of the present invention. In an embodiment, the neural network 500 receiving image patches 501a and 501b generated based on a plurality of images may extract tumor regions 502a and 502b corresponding to the respective image patches. For example, the tumor region 502a corresponding to the first image patch 501a may be detected based on the first image patch 501a, and the second image patch 501b may be detected based on the second image patch 501b. ), the tumor region 502b corresponding to may be extracted.

In addition, in one embodiment, the apparatus for diagnosing kidney cancer may measure an uncertainty value for a neural network model and input data. Uncertainty in the model is caused by the lack of data or the structure of the neural network, which makes it difficult to represent all of the data. Accordingly, the artificial neural network model according to some embodiments of the present invention may perform dropout in the learning phase and the inference phase in order to prevent an overfitting problem from occurring. In more detail, the apparatus for diagnosing kidney cancer may perform random prediction based on a probability distribution sample, and may perform the above-described random prediction by sampling k thinned neural nets and using an average value of each predicted value. In this case, the probability distribution may be a probability distribution of result data obtained in the inference step of the neural network model, an average value at this time may be obtained as a predicted value, and a variance value at this time may be regarded as an uncertainty value.

Uncertainty in input data is caused by randomness in the data generation process. In more detail, taking, for example, the case where the uncertainty values of the first data and the second data included in the random data that are difficult to express in terms of probability are the same, even though the uncertainty values of the aforementioned first data and the second data are the same, Since the first data and the second data are not data obtained according to a specified probability, different labels may be applied to each other. When such uncertainty of the input data occurs, the apparatus for diagnosing kidney cancer according to some embodiments of the present invention may measure the uncertainty value of the above-described input data through variance estimation.

The devices described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA) , a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. The device can be commanded. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. Computer readable media may include program instructions, data files, data structures, etc. alone or in combination. Program commands recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (9)

A method for providing information necessary for diagnosis of renal cancer performed by a computing device,
obtaining a plurality of images of a cross-section of the kidney and photographing meta data, and pre-processing the plurality of images to obtain a plurality of image patches having a designated size;
estimating a tumor region corresponding to each of the plurality of image patches by inputting the plurality of image patches to an artificial neural network model based on an edge-gated convolutional neural network (CNN); and
artificial intelligence using the shooting metadata including the estimated tumor region and pixel interval data, image depth data, thickness of an image slice, and number of voxels included in an image patch for each of the plurality of images. determining whether the tumor is malignant or benign by
As a method for providing information necessary for the diagnosis of renal cancer,
measuring uncertainty values of a neural network model and uncertainty values of a plurality of images based on estimation results of the neural network model;
changing at least one image among the plurality of images based on uncertainty values of the plurality of images; and
Further comprising: changing a labeling policy of the neural network model based on the uncertainty value of the neural network model;
Changing at least one image among the plurality of images based on the uncertainty values of the plurality of images
detecting at least one image having an uncertainty value of the plurality of images equal to or greater than a reference value;
Based on a user input for the tumor region of the detected image, changing the detected image,
A method for providing information necessary for the diagnosis of renal cancer. The method according to claim 1, wherein a plurality of images obtained by photographing the cross section of the kidney,
Including a CT image obtained from a Digital Imaging and Communications in Medicine (DICOM) file and a labeling image for the kidney,
A method for providing information necessary for the diagnosis of renal cancer. The method of claim 1,
Acquiring the plurality of image patches,
Performing data augmentation on a first image included in the plurality of images to generate a plurality of images from the first image, and pre-processing the generated plurality of images to obtain a plurality of image patches contains steps,
The data augmentation,
Including one or more of spatial enlargement, color enhancement, noise enhancement and cropping of the image,
A method for providing information necessary for the diagnosis of renal cancer. The method of claim 1,
The plurality of images,
A plurality of 3D images of a cross section of the kidney,
Acquiring the plurality of image patches,
Sliding in a depth direction with respect to the plurality of 3D images and acquiring the plurality of image patches of a specified size,
A method for providing information necessary for the diagnosis of renal cancer. The method of claim 4,
The neural network model,
Perform dropout in the learning phase and inference phase,
The uncertainty value of the neural network model is,
Measured based on the variance value of the probability distribution of the resulting data in the inference step of the neural network model,
A method for providing information necessary for the diagnosis of renal cancer. The method of claim 5,
The uncertainty values of the plurality of images are,
Measured based on the estimated variance of the resulting data in the inference step of the neural network model,
A method for providing information necessary for the diagnosis of renal cancer. The method of claim 1,
Providing information necessary for diagnosing renal cancer, further comprising setting weights of a plurality of images according to the changed labeling policy and assigning a greater weight to the changed images than images before the change to train the neural network model. How to. Including; processor;
the processor,
Acquiring a plurality of images of a kidney cross-section and photographing metadata, pre-processing the plurality of images to obtain a plurality of image patches of a designated size, and edge-gated CNN (Convolutional Neural Network) based artificial neural network model to estimate a tumor region corresponding to each of the plurality of image patches,
artificial intelligence using the shooting metadata including the estimated tumor region and pixel interval data, image depth data, thickness of an image slice, and number of voxels included in an image patch for each of the plurality of images. determining whether the tumor is malignant or benign, measuring an uncertainty value of the neural network model and uncertainty values of a plurality of images based on an estimation result of the neural network model, and determining the plurality of images based on the uncertainty values of the plurality of images. An apparatus for diagnosing renal cancer, which changes at least one of the images and changes a labeling policy of the neural network model based on an uncertainty value of the neural network model.
Changing at least one image of the plurality of images based on the uncertainty value of the plurality of images
detecting one or more images in which an uncertainty value of the plurality of images is greater than or equal to a reference value;
and changing the detected image based on a user input for the tumor region of the detected image. A computer program stored in a recording medium to execute the method of any one of claims 1 to 7 using a computer.

Images