KR102393390B1 - Target data prediction method using correlation information based on multi medical image - Google Patents

Abstract

A method for predicting target data using correlation information based on different medical images is provided. According to an embodiment of the present invention, a method for predicting target data using correlation information based on different medical images includes: acquiring a plurality of medical images using different methods; extracting a feature value for each medical image from the plurality of medical images; generating a joint feature matrix based on the feature values obtained as a result of the extraction; calculating correlation information based on the generated combined feature matrix; and predicting target data using the calculated correlation information and the acquired feature values.

Description

TARGET DATA PREDICTION METHOD USING CORRELATION INFORMATION BASED ON MULTI MEDICAL IMAGE using correlation information based on different medical images

The present invention relates to a method for predicting target data using correlation information based on different medical images. It relates to a method of predicting target data.

It is a recent trend for a diagnostician to predict various target data related to a patient by learning a medical image using a deep learning technique, rather than predicting target data related to a patient while watching a medical image as in the past. Here, the target data means the ground truth to derive the machine learning results using medical images. In order to accurately predict target data using medical images, a learning method using two different medical images is applied, but there are advantages and disadvantages for analyzing target data for each medical image, so there is a limit in increasing the prediction probability. .

On the other hand, cancer is the most common cause of death due to disease. In the case of a patient receiving radiation therapy, which is one of the methods for treating cancer, it is important to predict tumor malignancy, which is one of the aforementioned target data, in order to evaluate the prognosis and treatment response of cancer.

For example, among several types of tumors, prostate cancer is a common cancer that occurs in the West. An accurate and reliable test method is important for prostate cancer patients. A system commonly used to predict cancer malignancy is the Gleason scoring method. The Gleason scoring method is based on microscopic tumor patterns assessed by a pathologist while interpreting biopsy specimens from the patient's prostate. Specifically, the Gleason scoring method is based on the degree of loss of normal gland tissue structure (ie, gland shape, size, and differentiation). A typical Gleason scoring plot presents five basic tissue patterns called tumor “grades”. Subjective microscopic measurements of cancer-induced loss of normal gland structure are abbreviated as a number ranging from 1 to 5, with 5 being the worst possible rating. The Gleason score (GS) and Gleason sum are one and the same. However, a "Gleason rating" and a "Gleason score" (also called a "Gleason sum") are different. The Gleason score is the sum of primary grade (representing multiple tumors) and secondary grade (for few tumors), a number ranging from 2 to 10. According to current practice, it is generally thought that a higher Gleason score indicates that the tumor is more likely to be aggressive and the patient's prognosis is poor.

Such a Gleason scoring method must be accompanied by tissue collection and examination through a biopsy, and there is a problem in that pain and bleeding from the tumor occur due to the biopsy.

Meanwhile, in consideration of the above problems, there has been an attempt to predict the malignancy of cancer like the Gleason evaluation score through medical images such as magnetic resonance imaging. For example, in FIG. 1 , (a) the Gleason evaluation score (GS) is 1st grade 3 + 2nd grade 4, and the score is 7, and (b) 1st grade 4 + 2nd grade 3, where the score is 7 T2-weighted MR images (yellow arrows point to tumors) and ADC maps (yellow arrows point to tumors) of the case are shown. Here, the case of (b), which has a higher primary grade score, is a case of higher cancer malignancy than the case (a), but the size or shape of the prostate cancer shown on medical images is not related to the malignancy and bleeding from the tumor Because it affects the texture, it was difficult to immediately match the Gleason evaluation score with the tumor on the medical image.

In addition, T2-weighted MR imaging (T2wMR), which is commonly used in diagnosing prostate cancer through imaging, can distinguish prostate cancer by using the fact that the signal intensity of the prostate cancer region is lower than that of normal prostate tissue. Although it has the advantage of having anatomical information due to its high resolution, there is a limitation in that the signal strength of normal tissues in the prostate is lowered due to bleeding caused by the biopsy performed for the diagnosis of prostate cancer, which lowers the sensitivity, and the diffusion-weighted image ( The apparent diffusion coefficient map (ADC map), which is an imaging of diffusion-weighted imaging (DWI) values, uses an exponential function to calculate the difference between the low and high b-values of diffusion-weighted images. The signal intensity contrast between prostate cancer and normal tissue is maximized, and the signal intensity is low in prostate cancer. However, compared to T2wMR, ADC maps have less anatomical information, more noise, and low image resolution, making it difficult to use alone.

Therefore, there is a need for a method capable of predicting the malignancy of a tumor and increasing its accuracy, such as the Gleason evaluation score, using only a medical image without a biopsy.

(Patent Document 0001) US 10401457 (Patent Document 0002) US 9619882 (Patent Document 0003) US 8965089

An object of the present invention is to provide a method of accurately predicting target data based on a plurality of medical images.

Another object of the present invention is to provide a method for predicting the malignancy of cancer according to correlation information calculated using a plurality of medical images obtained by different methods without performing a biopsy.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the above problem, a method for predicting target data using correlation information based on different medical images includes: acquiring a plurality of medical images using different methods; extracting a feature value for each medical image from the plurality of medical images; generating a joint feature matrix based on the feature values obtained as a result of the extraction; calculating correlation information based on the generated combined feature matrix; and predicting target data using the calculated correlation information and the acquired feature values.

Other specific details of the invention are included in the detailed description and drawings.

According to the present invention as described above, it has various effects as follows.

According to the present invention, the prediction probability of target data can be increased by additionally using correlation information rather than simply extracting and combining feature values from a plurality of medical images.

The present invention can predict the malignancy of cancer only with a plurality of medical images without performing a tumor biopsy.

In addition, the present invention can increase the accuracy of predicting the degree of cancer malignancy by using correlation information calculated from medical images obtained by different methods without using only one medical image together with feature values of a plurality of medical images. there is.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is an exemplary diagram illustrating a T2-weighted MR image and an ADC map according to the Gleason evaluation score.
2 is a block diagram illustrating an apparatus for predicting target data according to an embodiment of the present invention.
3 is a flowchart illustrating a method for predicting target data according to an embodiment of the present invention.
4 is a flowchart illustrating a medical image preprocessing method according to an embodiment of the present invention.
5 is an exemplary diagram for explaining a medical image preprocessing method according to an embodiment of the present invention.
6 is a table showing characteristic values according to an embodiment of the present invention.
7 is a block diagram illustrating a method for predicting target data according to an embodiment of the present invention.
8 is a table for explaining an effect of a method for predicting target data according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the present embodiments allow the disclosure of the present invention to be complete, and those of ordinary skill in the art to which the present invention pertains. It is provided to fully understand the scope of the present invention to those skilled in the art, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

Spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe the correlation between a component and other components. Spatially relative terms should be understood as terms including different directions of components during use or operation in addition to the directions shown in the drawings. For example, when a component shown in the drawing is turned over, a component described as “beneath” or “beneath” of another component may be placed “above” of the other component. can Accordingly, the exemplary term “below” may include both directions below and above. Components may also be oriented in other orientations, and thus spatially relative terms may be interpreted according to orientation.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

2 is a block diagram illustrating an apparatus for predicting target data according to an embodiment of the present invention.

Referring to FIG. 2 , the apparatus 100 for predicting target data according to an embodiment of the present invention may predict target data (eg, tumor malignancy) using a plurality of acquired medical images. For example, the target data prediction apparatus 100 may calculate correlation information using feature values extracted from a plurality of medical images obtained by different methods, and predict target data using the extracted feature values and correlation information. there is. Here, the target data may be a ground truth to be derived by repeatedly machine learning the steps of extracting feature values, generating a combined feature matrix, calculating correlation information, and using the calculated correlation information and acquired feature values. Accordingly, the target data may include a tumor malignancy, and more specifically, a prostate cancer malignancy.

In an embodiment, the target data prediction apparatus 100 may be an electronic device such as a server or a PC, and a dedicated program for setting a target data prediction method may be installed. For example, the target data prediction apparatus 100 includes an image processing unit 110 capable of obtaining a medical image, preprocessing a medical image, and setting a medical image, and extracting image feature values for extracting radiomic features from the medical image. The unit 120, the correlation information generating unit 130 that generates a combined feature matrix and correlation information using the feature values, performs machine learning by using the combined feature value that combines the feature values and the correlation information as an input value, and targets A deep learning unit 140 for generating a data prediction value, a tumor determination unit 150 for determining target data based on a target data prediction value, and a plurality of medical images, extracted feature values, correlation information, combined feature values, machine The learning result, the structure and filter values of the deep neural network, the patient data, the target data prediction result, etc. may be converted into big data and stored in the database 160 may be included.

3 is a flowchart illustrating a method for predicting target data according to an embodiment of the present invention. 4 is a flowchart illustrating a medical image preprocessing method according to an embodiment of the present invention. 5 is an exemplary diagram for explaining a medical image pre-processing method according to an embodiment of the present invention. 6 is a table showing characteristic values according to an embodiment of the present invention. 7 is a block diagram illustrating a method for predicting target data according to an embodiment of the present invention. 8 is a table for explaining an effect of a method for predicting target data according to an embodiment of the present invention.

Meanwhile, as described above, the target data may be a ground truth to be derived through deep learning. That is, the ground truth to be derived is the target data by repeatedly machine learning the steps of extracting feature values, generating the combined feature matrix, calculating the correlation information, and using the calculated correlation information and the acquired feature values, which will be described below. can be Accordingly, the target data may be any data that can be diagnosed based on feature values extracted using medical images. That is, for example, the target data may be one of numerous medical data, such as tumor malignancy, a patient's fracture, organ damage, and the like. However, for convenience of explanation, below, the degree of tumor malignancy will be described as an example, and more specifically, prostate cancer will be additionally described as an example. Accordingly, the present invention is not limited to tumor malignancy or prostate cancer, and may include all types of data that can be diagnosed through medical imaging.

2 to 8 , in an embodiment, in operation 31 , the image processing unit 110 may acquire a plurality of medical images in different ways. That is, the plurality of medical images utilized in the present invention may each include a tumor located in the same organ, but may have different imaging devices or may have the same imaging device but different imaging conditions.

For example, the medical image includes an X-ray image, a CT image, and an MRI image, and there is no particular limitation if it is a medical image. "Image" means a subject collected by computed tomography (CT), magnetic resonance imaging (MRI), ultrasound or any other medical imaging system known in the art. may be a medical image of A medical image is voxel data, and may include a plurality of slices, that is, a plurality of unit images.

For example, each of the plurality of medical images may be acquired in the same imaging device under different conditions or may be acquired through different imaging devices. That is, the plurality of medical images is a plurality of images taken of the same organ with different imaging devices such as MR, CT, PET, or a plurality of images obtained with the same device (MRI) but under different imaging conditions (T2-weighted MR image, ADC map). It can be a video. In addition, in the case of a medical image such as CT or MRI, the brightness and contrast of the image can be adjusted through window setting, and the window setting consists of a window level and a window width, A plurality of medical images to which different window setting values are applied may be acquired. That is, for example, one window setting may be a case in which the window width and window level are adjusted so that the entire region of the organ is clearly visible in the medical image, and the other window setting is between the area where the tumor is located and the area surrounding the tumor in the medical image. It may be the case that the window width and window level are adjusted so that the boundary (or the critical region where the tumor is located) is clearly visible.

4 and 5 , the method for predicting tumor malignancy of the present invention may further include pre-processing a plurality of medical images obtained by different methods to increase prediction accuracy.

In an embodiment, the image processing unit 110 may acquire a plurality of medical images using different methods in operation 41 , and may perform intensity normalization on the plurality of medical images in operation 42 .

Specifically, there is a limitation in that it is difficult to extract consistent feature values because the range of signal intensity between MR images of the same parameter is different depending on the protocol used in the patient's medical image acquisition process. By doing so, a consistent feature vector can be extracted. To use the same range of signal strength between patients, the histogram stretching is performed through Equation 1 below to normalize the signal strength range, and T2wMR can be normalized to have a range from 0 to 500 and ADC map to have a range from 500 to 3000 there is.

[Equation 1]

In this case, I represents the signal intensity of the original image, Imax is the maximum value of the signal intensity in the original image, Imin represents the minimum value of the signal intensity in the original image, and I' max is the maximum value of the signal intensity range to be normalized, I ' min represents the minimum value of the signal intensity range to be normalized.

In an embodiment, in operation 43 of the image processing unit 110 , a plurality of medical images may be cropped to segment an ROI from the plurality of medical images. For example, as shown in FIG. 5 , the image processing unit 110 may acquire a T2-weighted MR image (T2wMR in (a)) and an ADC map (ADC in (b)) and derive it by machine learning in advance. Using the segmented tumor mask (segmented tumor mask in (c)), the ROI images of the T2-weighted MR image (Tumor ROI of T2wMR in (d)) and the ROI images of the ADC map (Tumor ROI of in (e)) ADC) can be obtained.

In an embodiment, in operation 32 , the image feature value extractor 120 may extract a feature value for each medical image from the plurality of medical images. Here, the type and number of feature values are not limited to the table shown in FIG. 6 . That is, for example, a total of 68 feature values shown in FIG. 6 may not be extracted for each medical image, the number may be changed, and types are not limited to those shown in FIG. 6 . In the present invention, rather than the type and number of feature values, it is important to extract feature values from each of a plurality of medical images and utilize them.

For example, the plurality of feature values may include at least one of a histogram feature value, a texture feature value, and a shape feature value. The histogram feature value may include first order statistics and histogram percentile intensities shown in FIG. 6 , and the texture feature value is a gray level co-occurrence matrix (GLCM), a gray level run length matrix (GLRLM), and a local binary pattern (LBP). ), and the shape feature values may include area/perimeter ratio, convex area, eccentricity, Euler number, major-minor axis ratio, major axis length, minor axis length, area, and perimeter.

Specifically, histogram feature vectors include mean, minimum (min), maximum (max), variance, standard deviation, skewness, kurtosis, and entropy. ) can be extracted. The histogram feature vector can be calculated through Equation 2 below.

[Equation 2]

는 전립선 PZ 내에 위치하는 윈도우의 i번째 화소에서 영상의 신호강도를 의미하고,

는 윈도우 내 영상의 신호강도 평균값을 의미한다.

는 영상 히스토그램에서

의 신호강도 확률을 의미한다. 한편, 최소값(min)은 최소 신호강도 값, 최대값(max)은 최대 신호강도 값일 수 있고, 표준편차(standard deviation)는 분산에 제곱근을 적용하여 연산할 수 있다.In this case, n means the size of the window of the medical image,

is the signal intensity of the image in the i-th pixel of the window located within the prostate PZ,

is the average value of the signal intensity of the image within the window.

is in the image histogram.

is the signal strength probability of . Meanwhile, the minimum value (min) may be a minimum signal intensity value, the maximum value (max) may be a maximum signal intensity value, and the standard deviation may be calculated by applying a square root to the variance.

Meanwhile, for the histogram percentage intensity, an average value, a maximum value, a minimum value, etc. may be extracted as feature values based on 5%, 25%, 50%, 75%, and 95%, respectively.

For example, GLCM, GLRLM, and LBP can be extracted as texture feature vectors. GLCM is a probability-based matrix that analyzes the number of simultaneous occurrences of two pixel pairs to express texture information in an image. 3, a feature value of a medical image can be extracted. Meanwhile, after obtaining the following 7 values, a total of 14 feature values may be utilized by calculating the average value (min) and standard deviation of each value. Of course, this number is exemplary and other numbers may be applied. On the other hand, ASM (Angular Second Moment) can be calculated through the following energy equation.

[Equation 3]

In this case, i and j mean two pixel pairs having signal intensity information, d is the distance between the two pixels used to calculate the intensity co-occurrence matrix, θ is the direction, and p(i, j) is Contrast may mean the probability value of the (i, j) position of the co-occurrence matrix, and N may mean the size of the intensity/dark co-occurrence matrix.

For example, GLRLM is a method of expressing texture information through images. It is a probability-based matrix that analyzes the lengths of two or more pixels. It means the continuity of pixels with the same signal intensity. The feature vector of the image can be calculated as in Equation 4 below through the GLRLM obtained in the direction of the figure. Meanwhile, after obtaining 11 values through the following 11 equations, a total of 22 feature values can be utilized by calculating the average value (min) and standard deviation of each value. Of course, this number is exemplary and other numbers may be applied.

[Equation 4]

In this case, i is the signal strength of the pixel, j is the number of pixels having the same signal strength appearing continuously as i, GLRLM(i, j) is the probability value of the (i, j) position of the GLRLM, and H is the size of the intensity working length matrix.

For example, LBP is a feature vector representing the texture characteristics around a pixel. It compares the signal intensity of one pixel with the signal intensity of the neighboring pixel and expresses the result as a threshold value to align. It is possible to calculate a feature vector having , as in Equation 5. Meanwhile, after obtaining the following three values, the average value (min), minimum value (min), maximum value (max), variance, standard deviation, etc. of each value are calculated to obtain a total of 10 feature values. can be used Of course, this number is exemplary and other numbers may be applied.

[Equation 5]

In this case, gc denotes a central pixel, gp denotes a neighboring pixel, and m0 and m1 denote an average between signal intensities of neighboring pixels converted to a threshold value.

In an embodiment, in operation 33 , the correlation information generator 130 may generate a joint feature matrix based on the feature values obtained as a result of extraction. For example, a histogram feature value, a texture feature value, and a shape feature value may be extracted for each medical image, and a matrix may be generated by utilizing each feature value. For example, for convenience of explanation, if two medical images are used as shown in FIG. 7 , a first feature value and a second feature value are respectively obtained from the first medical image and the second medical image obtained by different methods. can be extracted. Here, each of the first feature value and the second feature value may include 68 feature values shown in FIG. 6 . And, for example, the correlation information generator 130 may generate a combined feature matrix in which the first feature value is set as the x-axis data and the second feature value is set as the y-axis data. Here, the combined feature matrix may be a combination of feature values of a plurality of medical images in a matrix form. Accordingly, the manner in which the combined feature matrix is generated may vary.

In addition, when there is a third medical image obtained by a method different from the first medical image and the second medical image, a third feature value may be extracted from the third medical image, and the first feature value, the second feature value, and A combined feature matrix) may be generated based on the third feature value. For example, the correlation information generator 130 may generate a combined feature matrix in which the first feature value is set as the x-axis data, the second feature value is set as the y-axis data, and the third feature value is set as the z-axis data. there is. That is, as described above, the combined feature matrix may be generated based on three medical images, and, of course, may be generated by being expanded to four or more.

In an embodiment, in operation 34 , the correlation information generator 130 may calculate correlation information based on the generated combined feature matrix. Here, the correlation information may be calculated, for example, through a combined feature matrix including the first feature value and the second feature value as shown in FIG. 7 . This may mean that the correlation information becomes information capable of representing all feature values of a plurality of medical images.

For example, the correlation information may be calculated based on a gray level co-occurrence matrix (GLCM) that considers pixel brightness values and position information.

For example, the intensity co-occurrence matrix (GLCM) is a statistical method for examining textures in consideration of spatial relationships of pixels, and is a spatial intensity-dependent matrix. The GLCM function is a method of identifying a medical image by calculating how often a pixel pair having a specific value in an image occurs in a specific spatial relationship, generating a GLCM, and extracting statistical measurements from this matrix as feature values.

Accordingly, it is possible to convert to GLCM based on a combined feature matrix generated by combining respective feature values of a plurality of medical images, and extract a feature value from the converted GLCM, and these feature values are the relationship between the plurality of medical images. may be correlation information indicating Of course, a method of calculating correlation information using GLCM is only an example, and correlation information may be extracted using another function or algorithm such as GLRLM based on the combined feature matrix.

In an embodiment, in operation 35, the correlation information generator 130 may generate a combined feature value by concatenating the calculated correlation information and the acquired feature values. For example, as shown in FIG. 7 , a combined feature value is generated by concatenating (eg, simple sum, average, maximum, etc.) the first feature value, the second feature value, and correlation information (eg, GLCM feature value). can do. For example, as shown in FIG. 6 , when the first feature value is 68 and the second feature value is 68, and as described above, when 14 feature values are extracted using GLCM for correlation information, the combined feature value can be 150. there is. Of course, this number is only an example, and the type and number of feature values may be variously modified.

In an embodiment, in operation 36, the deep learning unit 140 may input the combined feature value into a previously learned deep neural network and may calculate a result value through the deep neural network. For example, the deep neural network may be a classifier and may be a random forest classifier (RFC). The classifier can change the ground truth depending on the type of tumor to learn, and when predicting the malignancy of prostate cancer, the tissue biopsy result can be used as the ground truth. Of course, as the classifier, logistic regression analysis (LR), k nearest neighbors (kNN), support vector machine (SVM), etc. may be used.

In an embodiment, in operation 37, the tumor determiner 150 may predict the malignancy of a tumor corresponding to a plurality of medical images by using a result value calculated through the deep neural network. For example, the result value may be a quantified value such as the Gleason evaluation score, and the degree of malignancy may be predicted like the Gleason evaluation score according to the numerical value of the score.

That is, as described above, according to the present invention, not only feature values of each medical image are used, but also correlation information between medical images can be used to increase the prediction accuracy of tumor malignancy.

On the other hand, the data set used to confirm the effect of the present invention utilized 210 cases of the low Gleason score group and 156 cases of the high Gleason score group. The low Gleason score group included 14 cases with a score of 3+3 and 196 cases with a score of 3+4 as cases lower than the primary grade 3 and secondary grade 4. The high Gleason score group was a case higher than the primary grade 4 and secondary grade 3, with 116 cases with a score of 4+3, 16 cases with a score of 4+4, 21 cases with a score of 4+5, and 3 cases with a score of 5+4. includes

Referring to FIG. 8 , A is a result of learning 68 feature values shown in FIG. 6 of a T2-weighted MR image (T2wMR) to the classifier, and B is an ADC map showing 68 feature values shown in FIG. 6 to the classifier. It is the learning result, C is the concatenated features of A and B features, and 136 feature values are learned by the classifier, D is the average value of A and B features (Averaged features, 68) is the result of learning in the classifier, E is the result of learning the multiplied feature value (Multiplied features, 68) of the A feature value and B feature value, and F is the correlation information (feature value based on the combined feature matrix, 14 features) ) is the result of learning by the classifier, and G is the result of learning the concatenated joint feature matrix features (150) of the present invention by the classifier. As shown in FIG. 8 , in the seven methods, six evaluation criteria were determined from the classifier learning results. The six evaluation criteria are accuracy (acc), sensitivity (sen), specificity (spe), positive predictive value (PPV), negative predictive value (NPV), and classifier (AUC). operating characteristic curve). Sensitivity and PPV are the true positive ratios for actual Gleason score cases and predicted cases, respectively. Specificity and NPV are correspondingly the ratios of true negatives for actual Gleason score cases and predicted cases. Finally, AUC represents the probability of a classifier that accurately discriminates tumor malignancy, such as the actual Gleason score. Referring to FIG. 8 , method G of the present invention showed the best performance in AUC, accuracy, sensitivity, and positive predictive value, and showed above-average performance even in negative predictive value. It was confirmed that excellent performance was shown.

According to an embodiment of the present invention, a method for predicting target data using correlation information based on different medical images includes: acquiring a plurality of medical images using different methods; extracting a feature value for each medical image from the plurality of medical images; generating a joint feature matrix based on the feature values obtained as a result of the extraction; calculating correlation information based on the generated combined feature matrix; and predicting target data using the calculated correlation information and the acquired feature values.

According to various embodiments, the target data is to be derived by repeatedly machine learning the steps of extracting the feature value, generating the combined feature matrix, calculating the correlation information, and using the calculated correlation information and the acquired feature values. It could be the ground truth.

According to various embodiments, the target data may be tumor malignancy.

According to various embodiments, the target data may be prostate cancer.

According to various embodiments, each of the plurality of medical images may be acquired under different conditions in the same imaging device or may be acquired through different imaging devices.

According to various embodiments, predicting the target data may include generating a combined feature value by concatenating the calculated correlation information and the acquired feature values.

According to various embodiments, the predicting of the target data may include inputting the combined feature value into a previously learned deep neural network, and using the result value calculated through the deep neural network to correspond to the plurality of medical images It may further include; predicting the target data.

According to various embodiments, the plurality of medical images include a first medical image and a second medical image obtained by different methods, respectively, and a first feature value and a first feature value in the first medical image and the second medical image extracting a second feature value; generating the combined feature matrix based on the first feature value and the second feature value; and calculating the correlation information based on the combined feature matrix.

According to various embodiments of the present disclosure, the method may include acquiring a third medical image using a method different from that of the first medical image and the second medical image; extracting a third feature value from the third medical image; generating the joint feature matrix based on the first feature value, the second feature value, and the third feature value; and calculating the correlation information based on the combined feature matrix.

According to various embodiments, the correlation information may be calculated based on a gray level co-occurrence matrix (GLCM) that considers the pixel brightness value and position information.

According to various embodiments, the feature values may include at least one of a histogram feature value, a texture feature value, and a shape feature value.

According to various embodiments, the method may further include preprocessing the plurality of medical images, wherein the preprocessing may include: histogram normalization of the plurality of medical images; and cropping the plurality of medical images to segment the ROI from the plurality of medical images.

According to various embodiments, a tumor malignancy prediction program using correlation information based on different medical images may be stored in a medium in order to execute the method of any one of claims 1 to 9 by being combined with a computer that is hardware. there is.

The steps of a method or algorithm described in relation to an embodiment of the present invention may be implemented directly in hardware, as a software module executed by hardware, or by a combination thereof. A software module may contain random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present invention pertains.

As mentioned above, although embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains know that the present invention may be embodied in other specific forms without changing the technical spirit or essential features thereof. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

100: target data prediction device
110: image processing unit
120: image feature value extraction unit
130: correlation information generating unit
140: deep learning unit
150: target data determination unit
160: database

Claims (10)

A method for predicting target data using correlation information based on different medical images, the method comprising:
acquiring a plurality of medical images using different methods, respectively;
extracting a feature value for each medical image from the plurality of medical images;
generating a joint feature matrix by combining the feature values obtained as a result of the extraction in a matrix form;
calculating correlation information based on the generated combined feature matrix; and
generating a combined feature value by concatenating the calculated correlation information and the acquired feature values;
Predicting target data corresponding to the plurality of medical images by inputting the combined feature values into a previously learned deep neural network and using the result values calculated through the deep neural network;
Target data prediction method using correlation information based on different medical images, wherein the plurality of medical images each include a tumor located in the same organ, and are acquired under different conditions in the same imaging device or are acquired through different imaging devices . The method of claim 1, wherein the target data is to be derived by repeatedly machine learning the steps of extracting the feature value, generating the combined feature matrix, calculating the correlation information, and using the computed correlation information and the acquired feature values. Target data prediction method using correlation information based on different medical images, characterized in that it is the ground truth. The method of claim 2 , wherein the target data is a tumor malignancy level using correlation information based on different medical images. The method of claim 3 , wherein the target data is prostate cancer, using correlation information based on different medical images. delete delete delete The correlation based on different medical images according to claim 1, wherein the correlation information is calculated based on a gray level co-occurrence matrix (GLCM) that considers pixel brightness values and position information. A method of predicting target data using information. The method of claim 1, wherein the feature values include at least one of a histogram feature value, a texture feature value, and a shape feature value. In combination with a computer that is hardware, the computer program for predicting target data using correlation information based on different medical images, stored in a medium to execute the method of any one of claims 1 to 4, 8 and 9 .

Images