KR20230147589A - Presence or absence of lesion prediction method using 2 dimensional images extracted from 3 dimensional computed tomography image - Google Patents

Abstract

A method of predicting a lesion using 2D images extracted from a 3D CT image includes the steps of an analysis device receiving a 3D CT image of a subject's region of interest, and the analysis device separating anatomically separate sub-regions from the region of interest. A step of calculating, by the analysis device, a plurality of 2D images by projecting from different starting points for each of the sub-areas, by the analysis device a plurality of images calculated from each of the sub-areas among the plurality of 2D images. Combining the images and inputting the combined images into a classification model by the analysis device to predict lesions in the region of interest.

Description

Method for predicting lesions using 2D images extracted from 3D CT images {PRESENCE OR ABSENCE OF LESION PREDICTION METHOD USING 2 DIMENSIONAL IMAGES EXTRACTED FROM 3 DIMENSIONAL COMPUTED TOMOGRAPHY IMAGE}

The technology described below relates to a technique for predicting lesions using 2D images generated from 3D (dimensional) CT (Computed Tomography) images.

Traditionally, medical institutions use patients' CT images to diagnose diseases. Recently, various studies are being conducted to diagnose lesions by analyzing medical images using AI (Artificial Intelligence).

Korean Patent Publication No. 10-2021-0035381

CT images are provided as three-dimensional images by stacking multiple slide images. Conventional AI models extract features from two-dimensional images and produce certain information. The AI model that received CT images evaluated lesions by analyzing one or multiple slides. Therefore, in the prior art, it was not easy to analyze when some of the slides among a plurality of slides had lesions. In addition, although organs such as the lungs have a three-dimensional structure, the prior art had limitations in utilizing the entire information in a three-dimensional space.

The technology described below generates 2D images from different viewpoints from 3D CT images and seeks to convey comprehensive information given in 3D space to a machine learning model. The technology described below seeks to provide a technique for predicting a subject's lesion using multiple 2D images.

A method of predicting a lesion using 2D images extracted from a 3D CT image includes the steps of an analysis device receiving a 3D CT image of a subject's region of interest, and the analysis device separating anatomically separate sub-regions from the region of interest. A step of calculating, by the analysis device, a plurality of 2D images by projecting from different starting points for each of the sub-areas, by the analysis device a plurality of images calculated from each of the sub-areas among the plurality of 2D images. Combining the images and inputting the combined images into a classification model by the analysis device to predict lesions in the region of interest.

In another aspect, a method of predicting a lesion using 2D images extracted from a 3D CT image includes the steps of an analysis device receiving a 3D CT image of a subject's region of interest, and the analysis device Separating regions, calculating a plurality of 2D images by projecting the analysis device from different starting points for each of the sub-regions, and calculating a plurality of 2D images by the analysis device in each of the sub-regions among the plurality of 2D images. The method includes inputting at least one 2D image into different classification models and predicting a lesion in the region of interest based on values output by the analysis device from the different classification models.

The technology described below enables robust lesion prediction by using 2D images from various viewpoints rather than a single fixed viewpoint. Furthermore, the technology described below enables more accurate lesion prediction by using an ensemble model that processes multiple 2D images.

Figure 1 is an example of a lesion prediction system using 2D images extracted from 3D CT images.
Figure 2 is an example of a lesion prediction process using a 2D image extracted from a 3D CT image.
Figure 3 is an example of the process of separating the lung region from a 3D CT image.
Figure 4 is an example of acquiring a 2D image through projection of a specific viewpoint from a 3D CT image.
Figure 5 is an example of a 2D image generation process according to truncation maximum projection from a 3D CT image.
Figure 6 is an example of a classification model that predicts lesions using 2D CT images.
Figure 7 is another example of a classification model that predicts lesions using 2D CT images.
Figure 8 is an example of an analysis device that predicts lesions using 2D images extracted from 3D CT images.

The technology described below may be subject to various changes and may have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the technology described below to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the technology described below.

Terms such as first, second, A, B, etc. may be used to describe various components, but the components are not limited by the terms, and are only used for the purpose of distinguishing one component from other components. It is used only as For example, without departing from the scope of the technology described below, the first component will be referred to as the second component, and the technology described below may be modified in various ways and may have several embodiments. Examples will be illustrated in the drawings and explained in detail. However, this is not intended to limit the technology described below to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the technology described below.

Terms such as first, second, A, B, etc. may be used to describe various components, but the components are not limited by the terms, and are only used for the purpose of distinguishing one component from other components. It is used only as For example, a first component may be named a second component without departing from the scope of the technology described below, and similarly, the second component may also be named a first component. The term and/or includes any of a plurality of related stated items or a combination of a plurality of related stated items.

In terms used in this specification, singular expressions should be understood to include plural expressions, unless clearly interpreted differently from the context, and terms such as “including” refer to the described features, numbers, steps, operations, and components. , it means the existence of parts or a combination thereof, but should be understood as not excluding the possibility of the presence or addition of one or more other features, numbers, step operation components, parts, or combinations thereof.

Before providing a detailed description of the drawings, it would be clarified that the division of components in this specification is merely a division according to the main function each component is responsible for. That is, two or more components, which will be described below, may be combined into one component, or one component may be divided into two or more components for more detailed functions. In addition to the main functions it is responsible for, each of the components described below may additionally perform some or all of the functions handled by other components, and some of the main functions handled by each component may be performed by other components. Of course, it can also be carried out exclusively by .

In addition, when performing a method or operation method, each process forming the method may occur in a different order from the specified order unless a specific order is clearly stated in the context. That is, each process may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the opposite order.

The technology described below is a technique that extracts 2D image(s) at a specific point in time from a 3D CT image and predicts lesions using the extracted 2D image.

3D CT images are produced by CT equipment and provided as digital data in a certain format. The algorithms or models by which CT equipment reconstructs 3D images can vary. The following description assumes that a certain 3D CT image is provided, and does not limit the method of configuring the 3D CT image.

Below, it is explained that the analysis device predicts lesions in the image using a certain machine learning model. The analysis device can be implemented as a variety of devices capable of processing data. For example, an analysis device can be implemented as a PC, a server on a network, a smart device, or a chipset with a dedicated program embedded therein.

Machine learning models include decision trees, random forest, KNN (K-nearest neighbor), Naive Bayes, SVM (support vector machine), and ANN (artificial neural network). In particular, the technology described below can predict lesions in CT images using ANN.

ANN is a statistical learning algorithm that mimics biological neural networks. Various neural network models are being studied. Recently, deep learning network (DNN) has been attracting attention. DNN is an artificial neural network model consisting of several hidden layers between the input layer and the output layer. DNN, like general artificial neural networks, can model complex non-linear relationships. Various types of DNN models have been studied. For example, there are Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), Generative Adversarial Network (GAN), and Relation Networks (RL).

Figure 1 is an example of a lesion prediction system 100 using a 2D image extracted from a 3D CT image. Figure 1 shows an example in which the analysis device is a computer terminal 130 and a server 140.

The CT equipment 110 generates CT images for the subject. At this time, the final image generated by the CT equipment 110 is a 3D CT image. The subject's 3D CT image can be stored in EMR (Electronic Medical Record, 120).

In FIG. 1, a user (A) can predict a lesion using a subject's image using the computer terminal 130. The computer terminal 130 receives the 3D CT image of the subject. The computer terminal 130 can receive 3D CT images from the CT equipment 110 or the EMR 120 through a wired or wireless network. In some cases, the computer terminal 130 may be a device physically connected to the CT equipment 110.

The computer terminal 130 generates 2D image(s) from the 3D CT image of the subject. At this time, the 2D image(s) correspond to an image in a certain direction in three-dimensional space. The computer terminal 130 analyzes 2D images and predicts lesions using a machine learning model. User A can check the analysis results.

The server 140 may receive a 3D CT image of a subject from the CT equipment 110 or the EMR 120. The server 140 generates 2D image(s) from the 3D CT image of the subject. At this time, the 2D image(s) correspond to an image in a certain direction in three-dimensional space. The server 140 analyzes 2D images and predicts lesions using a machine learning model. The server 140 may transmit the analysis results to user A's terminal. User A can check the analysis results.

The computer terminal 130 and/or server 140 may store the analysis results in the EMR 120.

The technology described below is a technology that predicts or confirms a subject's lesion using 2D image(s) extracted from 3D CT images. Therefore, the technology described below can be applied to 3D CT images regardless of the type of specific anatomical region. For example, it can be applied to images of various areas such as the brain, lungs, and abdomen. However, for convenience of explanation, the explanation will be based on the lungs.

Figure 2 is an example of a lesion prediction process 200 using a 2D image extracted from a 3D CT image. It is assumed that the analysis device performs the entire process. However, depending on the embodiment, the device that generates a 2D image from a 3D CT image and the device that classifies the lesion based on the 2D image may be different.

The analysis device receives 3D CT images of the lung. The analysis device separates the lung region from the 3D CT image (210). The analysis device can separate lung areas from 3D images using a model (segmentation model) that separates specific objects in the image.

Various segmentation models have been studied. For example, segmentation models include FCN (Fully Convolutinal Network), DeebLab, U-net, etc. Figure 2 illustrates the structure of an FCN-based model. The process of segmenting the lung region is explained in detail in FIG. 3.

The analysis device separates the lung region from the lung 3D CT and projects it to a specific viewpoint in 3D space to obtain the desired 2D image(s) (220). The specific point in time can be set by the user. The 2D image at a specific viewpoint desired by the user is named the 2D image of interest. The process of generating a 2D image of interest is explained in detail in FIGS. 4 and 5.

The analysis device predicts the subject's lesion based on the 2D image(s) of interest (230). The predicted result may be normal or lesion detection. The analysis device can input 2D image(s) of interest into a pre-trained classification model and predict lesions based on the probability value calculated by the classification model. A classification model is a model that calculates output values based on the characteristics of the input image. Representative classification models include CNN. For example, in CNN, the convolutional layer extracts features of the input image, and the pre-connection layer calculates a certain probability value from the input features.

Figure 3 is an example of the process of separating the lung region from a 3D CT image. First, the analysis device creates a mask to separate the area of interest (lung area) from the 3D CT image. Afterwards, the analysis device can separate the lung area from the 3D CT image using a mask.

Figure 3(A) is an example of the process of creating a waste mask. The analysis device can generate a lung mask by inputting the lung 3D CT image into the segmentation model. FCN detects lung areas through a process where the convolution layer (Conv) extracts features from the input image and the deconvolution layer (Deconv) restores the image from the features. Meanwhile, since the input image is a 3D CT image, a layer that samples the 3D image in 2D form before the convolution layer and a layer that configures the 2D image into a 3D image after the deconvolution layer may be further used. Furthermore, the analysis device can calculate the lung mask using any one of various neural network models for segmentation of 3D images.

Figure 3(B) is an example of separating the lung area from a lung 3D CT image using a lung mask. The analysis device can separate lung areas from lung 3D CT images based on the lung mask. The lungs are made up of the left lung and right lung. In order to prevent both lungs from overlapping during the projection process of generating a 2D image, the analysis device can separate the separated lung area into left lung and right lung.

Figure 3 is an example explaining lungs with left and right structures. In the case of a brain consisting of a left brain and a right brain, the analysis device can separate the left brain and the right brain through a process similar to FIG. 3. Meanwhile, in the case of anatomical areas that do not have a symmetrical structure, the analysis device may not perform the process of separating the left and right sides.

Assume that the analysis device has separated the left lung and right lung in the lung 3D CT. Currently, the left lung and right lung are each 3D image objects. Therefore, projection is possible based on a certain direction (viewpoint) in three-dimensional space.

Figure 4 is an example of acquiring a 2D image through projection of a specific viewpoint from a 3D CT image.

Figure 4(A) is an example of setting projection points in three-dimensional space for the left and right lungs. Each lung region has a sphere surrounding it. At this time, the location of the lung area within the oral cavity may vary. For example, the center of gravity of the lung area may coincide with the center of the sphere. In Figure 4, a point indicates the projection start position. The projection can proceed in a straight line from the starting position towards the center of the sphere. Alternatively, the projection may proceed in a straight line from the starting position toward the center of gravity of the lung region.

Figure 4(B) is an example of acquiring various 2D images of interest by setting various projection start points based on one lung area.

The analysis device can output the separated lung area on the screen. In this case, the analysis device can output a projection starting point that moves between the lung area and the surface of a virtual sphere. The user can set the projection start point while viewing the output screen using an input device. Through this, the user can secure image(s) in the desired direction. Users can set the projection start point so that images in a specific direction overlap. Alternatively, users can set the projection start point in various directions to prevent 2D images from overlapping.

The fluoroscopy starting point may be set in a certain direction, such as front, upper side, right side, left side, and back, considering the anatomical structure of the lung. In this case, the analysis device may automatically analyze the shape or form of the lung area and set the projection point so that a 2D image in a specific direction is generated.

There are various techniques for generating 2D images by projecting 3D CT images. If you start projection from the starting point, it will pass through a number of pixels in a single straight line. A 2D image must express multiple pixels on this straight line as a single pixel value. Some techniques are explained as follows. The techniques below correspond to guidelines for selecting or calculating the values of multiple pixels. The value of a pixel in a CT image may be a value representing brightness and darkness.

(1) Average projection technique

Each pixel value of a 2D image generated by projection is given as the average of the values corresponding to the pixel vector array of the projection angle within the 3D closed area for generating the corresponding pixel. The array consists of pixels located in a straight line.

(2) max projection technique

Each pixel value of a 2D image generated by projection is given as the largest value (max) among the values in the pixel vector array of the corresponding projection angle within the 3D closed area for generating the corresponding pixel.

(3) Truncated max projection technique

This technique uses only pixel values that satisfy certain conditions. This technique uses only pixels with values in a certain range [a,b] in the pixel vector array. Each pixel value of a 2D image generated by projection is given as the largest value among the values between a and b in the pixel vector array of the corresponding projection angle within the 3D closed area for generating the pixel.

(4) truncated average projection technique

This technique uses only pixel values that satisfy certain conditions. This technique uses only pixels with values in a certain range [a,b] in the pixel vector array. Each pixel value of a 2D image generated by projection is given as the average value of pixels with values between a and b in the pixel vector array of the corresponding projection angle within the 3D closed area for generating the pixel.

FIG. 5 is an example of a 2D image generation process 300 based on truncation maximum projection from a 3D CT image. Figure 5 shows a case where the specific range among the values located in the pixel vector array is set to -800 to 0. At this time, the range can be selected by the user. Alternatively, the range may be set to a preset value depending on the CT equipment or anatomical region characteristics.

The analysis device selects (310) a pixel vector array to determine one pixel value. A pixel vector array is set for all pixels relative to the 2D plane parallel to the projection direction. The analysis device sets the largest value among pixels with values between -800 and 0 among the selected pixel vector array as the value of the corresponding pixel in the 2D image (320). The analysis device performs the same process for all pixels in the 2D plane.

Furthermore, the analysis device can regularly normalize the 2D image calculated through this process (330). For example, the analysis device may perform minimum-maximum (Min-Max) normalization on the calculated 2D image. As is widely known, minimum-maximum normalization is a method of normalizing other values by uniformly scaling them based on the largest and smallest pixel values in the entire image. Of course, the analysis device may apply other normalization techniques to the 2D image.

As described above, the analysis device can generate multiple 2D images from a 3D CT of the lung. The analysis device classifies 2D images using a classification model. At this time, the analysis device can utilize a method of inputting multiple 2D images into an individual classification model (ensemble model). For convenience of explanation below, it is assumed that images of the front of the left lung, the side of the left lung, the front of the right lung, and the side of the right lung were created for the lung region.

Figure 6 is an example of a classification model that predicts lesions using 2D CT images. Figure 6 shows three models. Model 1, model 2, and model 3 each correspond to individual classification models. Meanwhile, the individual models in FIG. 6 must be trained in advance to calculate probability values for normal or abnormal (presence of lesion) using training data. The researcher learned the model in Figure 6 using both normal and patient data.

In the case of the front view of the lungs, the left lung and right lung can be displayed in one image. Therefore, the frontal lung image can be composed of one image combining the left lung and right lung. Model 1 is an example of a model that uses a single frontal image. Model 1 may be useful for classifying patients with lesions in only one of the left and right lungs. Model 1 must be trained in advance using training data consisting of frontal lung images and the label values of the corresponding images. In the prediction process, Model 1 receives the subject's frontal lung image (left lung + right lung), extracts features, and calculates a probability value for the lesion (e.g., a value between 0 and 1) based on the extracted features. The analysis device predicts lesions on the subject's image based on the value output by Model 1. For example, the analysis device can predict that if the value output by Model 1 is close to 0, it is normal, and if it is close to 1, it can be predicted that it is a lesion.

For lateral lung images, it is difficult to combine information from both lungs into one image. Therefore, separate classification models can be used for the left and right lungs. Model 2 is an example of a model that uses two side images. Model 2 must be trained in advance using training data consisting of two side lung images (left lung and right lung) of the same subject and the label values of the corresponding images. Model 2 receives the left lung side image and the right lung side image and calculates the probability value for the lesion. When using two input images, the model can be implemented in various structures. Model 2 shows a model structure with two input layers and a convolution layer. The left lung side image is input from one input layer and features are extracted through a convolutional layer. In another input layer, the right lung side image is input and features are extracted through a convolution layer. A series of feature vectors can be constructed by combining the features of the left lung lateral image and the features of the right lung lateral image. The pre-connection layer receives feature vectors and calculates a probability value for the lesion (e.g., a value between 0 and 1). The analysis device predicts lesions on the subject's image based on the value output by Model 2. For example, the analysis device can predict that if the value output by Model 2 is close to 0, it is normal, and if it is close to 1, it is a lesion.

Furthermore, the subject's lesion can be predicted using an ensemble model that utilizes both Model 1 and Model 2. Model 3 can use both the first value output by Model 1 and the second value output by Model 2 to finally output a predicted value for the subject's lesion. For example, Model 3 can predict lesions based on the smaller or larger value of the first value or the second value. Alternatively, Model 3 may predict lesions based on the average of the first and second values. Alternatively, Model 3 may be classified as having a lesion if either the first or second value is abnormal.

The analysis device inputs a combined image (e.g., frontal image) of some of the plurality of 2D images into one model, calculates the value, and at least one image (e.g., left lung lateral image, right lung image) of the plurality of 2D images. The lesion may be predicted using all values calculated by inputting the lateral image (lateral image or left lung lateral image and right lung lateral image) into the second classification model.

The researcher constructed a model as shown in Figure 6 and verified the model. The researcher verified the results using CT data from 500 normal people and 255 tuberculosis patients. Model 3 was set to predict that the image was abnormal if either Model 1 or Model 2 was judged to be abnormal. Table 1 below shows the verification results for the models.

Accuracy TB-Sensitivity model 1 90% 79% model 2 92% 79% model 3 92% 87%

The results of Model 1 showed an accuracy of 90% and a tuberculosis sensitivity of 79%. Model 2 showed an accuracy of 92% and a sensitivity of 79% for tuberculosis. Model 3 showed an accuracy of 92% and a sensitivity of 87% for tuberculosis. Model 3, an ensemble model, showed better performance than individual models.

Figure 7 is another example of a classification model that predicts lesions using 2D CT images. Figure 7 is an example of an ensemble model using four images (left lung front, left lung lateral, right lung front, and right lung lateral). Figure 7 shows four classification models (Model 1, Model 2, Model 3, and Model 4) each processing four individual images.

Model 1 is an example of a model using the right lung front. Model 1 must be trained in advance using training data consisting of one frontal lung image and the label values of the corresponding images. In the prediction process, Model 1 receives the subject's right lung front as input, extracts features, and calculates a probability value for the lesion (e.g., a value between 0 and 1) based on the extracted features. The value calculated by Model 1 can be called a certain score.

Model 2 is an example of a model using the right lung side. Model 2 must be trained in advance using training data consisting of one side lung image and the label values of the corresponding images. In the prediction process, Model 2 receives the subject's right lung side as input, extracts features, and calculates a probability value for the lesion (e.g., a value between 0 and 1) based on the extracted features. The value calculated by Model 2 can be called a certain score.

Model 3 is an example of a model using the left lung front. Model 3 must be trained in advance using training data consisting of one frontal lung image and the label values of the corresponding images. In the prediction process, Model 3 receives the subject's left lung front as input, extracts features, and calculates a probability value for the lesion (e.g., a value between 0 and 1) based on the extracted features. The value calculated by Model 3 can be called a certain score.

Model 4 is an example of a model using the left lung aspect. Model 4 must be trained in advance using training data consisting of one side lung image and the label values of the corresponding images. In the prediction process, Model 4 receives the left lung side of the subject as input, extracts features, and calculates a probability value for the lesion (e.g., a value between 0 and 1) based on the extracted features. The value calculated by Model 4 can be called a certain score.

The analysis device can finally determine whether the subject is normal or abnormal by combining the scores output by Model 1, Model 2, Model 3, and Model 4. The analysis device can predict lesions by comparing the sum of the scores output by the four models and the threshold. Alternatively, the analysis device may predict lesions by comparing the average of the scores output by the four models with the threshold. Alternatively, the analysis device may predict lesions by individually assigning different weights according to the values output by Models 1 to 4 and comparing the summed value with the threshold.

In Figures 6 and 7, an ensemble model for classifying lesions using 2D CT images is explained. The ensemble model may have a structure different from that of Figure 6 or Figure 7. The ensemble model may receive input images from different viewpoints (angles), and the number of input images may be different.

The individual models in Figure 7 must also be trained in advance using training data, and the model learning methods in Figures 6 and 7 may vary. For example, the individual model in FIG. 6 or FIG. 7 may be learned to classify normal and abnormal using normal data and patient data or only normal data. The researcher constructed a model with the structure shown in Figure 7 and learned individual models using only normal data. In other words, individual models were learned to calculate specific values (scores) for normal data. In this case, the individual model produces different values for abnormal data than for normal data.

Meanwhile, individual models can output values between 0 and 1 with a probability of being close to normal for input data (close to 0 is normal, and if close to 1 is abnormal). The researcher finally classified the results as normal based on the average value of the values output by individual models. Of course, the criteria for classifying input data using the values output by the ensemble model may vary.

The researcher also verified the performance of the ensemble model shown in Figure 7. The ensemble model verification in Figure 7 used the same data as the data used in the model verification in Figure 6. The researcher added data from pneumonia patients as well as tuberculosis patients to the model verification data in Figure 6 and verified the results of distinguishing between normal and abnormal (tuberculosis + pneumonia). As a result of the verification, the ensemble model with the structure shown in Figure 7 showed significantly high reliability with AUC (Area under the Curve) = 0.948.

Figure 8 is an example of an analysis device 400 that predicts a lesion using a 2D image extracted from a 3D CT image. The analysis device 400 corresponds to the above-described analysis devices (130 and 140 in FIG. 1). The analysis device 400 may be physically implemented in various forms. For example, the analysis device 400 may take the form of a computer device such as a PC, a network server, or a chipset dedicated to data processing.

The analysis device 400 may include a storage device 410, a memory 420, an arithmetic device 430, an interface device 440, a communication device 450, and an output device 460.

The storage device 410 can store a reinforcement learning model for detecting measurement points in cephalometric radiology images.

The storage device 410 can store a 3D CT image of a subject.

The storage device 410 may store other programs or codes for image processing. The storage device 410 may store a program for extracting 2D CT image(s) from a 3D CT image. The storage device 410 may store a neural network model that separates 2D CT image(s) from 3D CT images. The storage device 410 may store a segmentation model for creating a mask that separates a region of interest in a 3D CT image.

The storage device 410 can store 2D CT image(s) extracted from 3D CT images.

The storage device 410 may receive 2D CT image(s) and store a classification model that predicts the subject's lesion. The classification model may be an ensemble model including multiple models as described above.

The storage device 410 can store analysis results (video, text, etc.).

The memory 420 can store data and information generated during the process of the analysis device 400 generating a 2D CT image(s) from a 3D CT image and the process of predicting a lesion using the 2D CT image(s). there is.

The interface device 440 is a device that receives certain commands and data from the outside. The interface device 440 may receive a 3D CT image of the analysis target from a physically connected input device or an external storage device. The interface device 440 may transmit analysis results (video, text, etc.) to an external object.

The communication device 450 refers to a configuration that receives and transmits certain information through a wired or wireless network. The communication device 450 may receive a 3D CT image of the analysis target from an external object. Alternatively, the communication device 450 may transmit the analysis results (video, text, etc.) to an external object such as a user terminal.

Since the interface device 440 and the communication device 450 are components that exchange certain data with a user or other physical object, they can also be collectively referred to as input/output devices. If limited to the function of receiving 3D CT images, the interface device 440 and communication device 450 may be referred to as input devices.

The output device 460 is a device that outputs certain information. The output device 460 can output interfaces, analysis results, etc. required for the data processing process.

The computing device 430 can receive the 3D CT image and predict the subject's lesion using commands or program codes stored in the storage device 410.

The computing device 430 may generate a mask for detecting a region of interest by inputting the 3D CT image into a pre-trained segmentation model. The computing device 430 may separate a region of interest (eg, lung region) from the 3D CT image by applying a mask to the 3D CT image. Furthermore, the computing device 430 may separate the region of interest into sub-regions (eg, left lung and right lung), if necessary.

As described in FIG. 4, the computing device 430 may generate 2D image(s) by projecting the separated region of interest or sub-region at a certain viewpoint (angle).

The computing device 430 may input the generated 2D image(s) into a classification model to predict the subject's lesion. The computing device 430 may predict the subject's lesion using an ensemble model such as the model described in FIG. 6 or FIG. 7 .

The computing device 430 may be a device such as a processor that processes data and performs certain operations, an AP, or a chip with an embedded program.

Additionally, the method of generating a 2D image from a 3D CT image and the method of classifying lesions using a 2D CT image as described above may be implemented as a program (or application) including an executable algorithm that can be executed on a computer. The program may be stored and provided in a temporary or non-transitory computer readable medium.

A non-transitory readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as registers, caches, and memories. Specifically, the various applications or programs described above include CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM (read-only memory), PROM (programmable read only memory), and EPROM (Erasable PROM, EPROM). Alternatively, it may be stored and provided in a non-transitory readable medium such as EEPROM (Electrically EPROM) or flash memory.

Temporarily readable media include Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), and Enhanced SDRAM (Enhanced RAM). It refers to various types of RAM, such as SDRAM (ESDRAM), Synchronous DRAM (Synclink DRAM, SLDRAM), and Direct Rambus RAM (DRRAM).

This embodiment and the drawings attached to this specification only clearly show some of the technical ideas included in the above-described technology, and those skilled in the art can easily understand them within the scope of the technical ideas included in the specification and drawings of the above-described technology. It is self-evident that all inferable variations and specific embodiments are included in the scope of rights of the above-mentioned technology.

Claims (6)

A step where the analysis device receives a 3D (dimensional) CT (Computed Tomography) image of the subject's area of interest;
separating, by the analysis device, anatomically separate sub-regions from the region of interest;
generating, by the analysis device, a plurality of 2D images by projecting from different starting points for each of the sub-areas;
combining, by the analysis device, a plurality of images calculated from each of the sub-areas among the plurality of 2D images; and
A method of predicting a lesion using 2D images extracted from a 3D CT image, including the step of the analysis device inputting the combined image into a classification model to predict a lesion in the region of interest. According to paragraph 1,
The step of separating the sub-regions is
generating, by the analysis device, a mask for each of the sub-regions in the 3D CT image using a segmentation model; and
A method of predicting a lesion using 2D images extracted from a 3D CT image, including the step of the analysis device separating the sub-regions from the 3D CT image using each mask. According to paragraph 1,
The analysis device projects, for each of the sub-regions, a direction from a starting point located on the surface of the sphere including the sub-region to the center of the sphere, and calculates each pixel value of the plane in an average projection technique, maximum value projection. A method of predicting a lesion using 2D images extracted from a 3D CT image that calculates a plurality of 2D images using any one of a truncated average projection technique and a truncated maximum projection technique. A step where the analysis device receives a 3D (dimensional) CT (Computed Tomography) image of the subject's area of interest;
separating, by the analysis device, anatomically separate sub-regions from the region of interest;
generating, by the analysis device, a plurality of 2D images by projecting from different starting points for each of the sub-areas;
inputting, by the analysis device, at least one 2D image for each of the sub-regions among the plurality of 2D images into different classification models; and
A method of predicting a lesion using 2D images extracted from a 3D CT image, comprising predicting a lesion in the region of interest based on values output by the analysis device from the different classification models. According to paragraph 4,
The step of separating the sub-regions is
generating, by the analysis device, a mask for each of the sub-regions in the 3D CT image using a segmentation model; and
A method of predicting a lesion using 2D images extracted from a 3D CT image, including the step of the analysis device separating the sub-regions from the 3D CT image using each mask. According to paragraph 4,
The analysis device projects, for each of the sub-regions, a direction from a starting point located on the surface of the sphere including the sub-region to the center of the sphere, and calculates each pixel value of the plane in an average projection technique, maximum value projection. A method of predicting a lesion using 2D images extracted from a 3D CT image that calculates a plurality of 2D images using any one of a truncated average projection technique and a truncated maximum projection technique.

Images