KR102490461B1 - Target data prediction method using correlation information based on multi medical image - Google Patents

Abstract

A method of predicting target data using correlation information based on different medical images is provided. A method of predicting target data using correlation information based on different medical images according to an embodiment of the present invention includes acquiring a plurality of medical images using different methods; pre-processing the plurality of medical images to improve prediction accuracy; extracting feature values for each medical image from the plurality of medical images; generating a joint feature matrix by combining feature values obtained as a result of the extraction in a matrix form; calculating correlation information based on the generated combined feature matrix; and generating a combined feature value by concatenating the computed correlation information and the acquired feature values. and inputting the combined feature values into a pre-learned deep neural network and predicting target data corresponding to the plurality of medical images using result values calculated through the deep neural network.

Description

Target data prediction method using correlation information based on different medical images

The present invention relates to a method for predicting target data using correlation information based on different medical images, and more particularly, by using correlation information calculated based on feature values extracted from a plurality of medical images obtained by different methods. It relates to a method of predicting target data.

It is a recent trend that a diagnoser predicts various target data related to a patient by learning the medical image using a deep learning technique, instead of predicting target data related to the patient by viewing the medical image as in the past. Here, the target data means the ground truth to be derived from machine learning results using medical images. In order to accurately predict target data using medical images, a learning method using two different medical images is applied. .

On the other hand, cancer is the most common cause of death by disease. In the case of a patient who has received radiation therapy, which is one of the methods for treating cancer, it is important to predict tumor aggressiveness, which is one of the aforementioned target data, in order to evaluate the prognosis and treatment response of cancer.

For example, among several types of tumors, prostate cancer is a common cancer that occurs in the West. Recently, prostate cancer is increasing rapidly in Korea due to the increase in the elderly population and westernization of diet. Accurate and reliable test methods for prostate cancer are important. A commonly used system for predicting the malignancy of prostate cancer patients is the Gleason scoring method. The Gleason scoring method is based on the microscopic tumor pattern evaluated by a pathologist while interpreting a biopsy specimen from a patient's prostate. Specifically, the Gleason scoring is based on the loss of normal glandular tissue structure (i.e., gland shape, size, and differentiation). A typical Gleason scoring chart presents five basic histological patterns called tumor "grades". Subjective microscopic measures of cancer-induced loss of normal glandular structure are outlined by grading as a number ranging from 1 to 5, with 5 being the worst possible rating. The Gleason score (GS) and the Gleason sum are one and the same. However, a "Gleason scale" and a "Gleason score" (also called a "Gleason sum") are different. The Gleason score is a number ranging from 2 to 10 as the sum of the primary grade (representing a majority of tumors) and the secondary grade (representing a minority of tumors). According to current practice, it is generally thought that the higher the Gleason score, the more likely the tumor is aggressive and the poorer the patient's prognosis.

This Gleason score evaluation method should be accompanied by tissue collection and examination through biopsy, and there were problems such as pain and bleeding from the tumor due to biopsy.

Meanwhile, in consideration of the above problems, attempts have been made to predict the malignancy of cancer through medical images such as magnetic resonance imaging, like the Gleason evaluation score. For example, in FIG. 1 (a) when the Gleason evaluation score (GS) is 7 as the first grade 3 + second grade 4 and (b) when the score is 7 as the first grade 4 + second grade 3 A T2-weighted MR image of the case (yellow arrow points to the tumor) and ADC map (yellow arrow points to the tumor) are shown. Here, the case (b) with a higher first grade score is a case where the malignancy of the cancer is higher than that of the case (a), but the size or shape of the prostate cancer shown in the medical images is not related to the malignancy and bleeding from the tumor is low. Because it affects texture, it was difficult to immediately match the Gleason evaluation score with the tumor shown in the medical image.

In addition, T2-weighted MR imaging (T2wMR), which is commonly used when diagnosing prostate cancer through imaging, can distinguish prostate cancer by using the fact that the signal intensity of the prostate cancer area is lower than that of normal prostate tissue. Although it has the advantage of having anatomical information due to its high resolution, there is a limitation in that the sensitivity is lowered because the signal intensity of normal tissues in the prostate is also lowered due to bleeding caused by the biopsy performed for the diagnosis of prostate cancer, and the diffusion-weighted image ( The apparent diffusion coefficient map (ADC map), which visualizes the phenomenon diffusion coefficient value of diffusion-weighted imaging (DWI), uses an exponential function to determine the difference between the low b-value and the high b-value of the diffusion-weighted image. The signal intensity contrast between prostate cancer and normal tissue has a maximized value, and there is a characteristic of having low signal intensity in prostate cancer. However, compared to T2wMR, the ADC map has problems in that it has less anatomical information, a lot of noise, and a low image resolution, making it difficult to use alone.

Therefore, there is a need for a method capable of predicting the malignancy of a tumor and increasing its accuracy, such as the Gleason evaluation score, using only medical images without biopsy.

(Patent Document 0001) US 10401457 (Patent Document 0002) US 9619882 (Patent Document 0003) US 8965089

An object to be solved by the present invention is to provide a method of accurately predicting target data based on a plurality of medical images.

In addition, an object to be solved by the present invention is to provide a method for predicting the malignancy of cancer according to correlation information calculated using a plurality of medical images obtained by different methods without performing a biopsy.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

A method for predicting target data using correlation information based on different medical images according to an embodiment of the present invention for solving the above problems includes acquiring a plurality of medical images using different methods; pre-processing the plurality of medical images to improve prediction accuracy; extracting feature values for each medical image from the plurality of medical images; generating a joint feature matrix by combining feature values obtained as a result of the extraction in a matrix form; calculating correlation information based on the generated combined feature matrix; and generating a combined feature value by concatenating the computed correlation information and the acquired feature values. and inputting the combined feature values into a pre-learned deep neural network and predicting target data corresponding to the plurality of medical images using result values calculated through the deep neural network.

Meanwhile, an apparatus for predicting target data using correlation information based on different medical images according to an embodiment of the present invention acquires a plurality of medical images using different methods and pre-processes the plurality of medical images to improve prediction accuracy. image processing unit; an image feature value extraction unit extracting feature values for each medical image from the plurality of medical images and generating a joint feature matrix by combining the feature values acquired as a result of the extraction in a matrix form; a correlation information generating unit calculating correlation information based on the generated combined feature matrix; a deep learning unit inputting a concatenated feature value obtained by concatenating the computed correlation information and the acquired feature values to a pre-learned deep neural network, and calculating a resultant value through the deep neural network; and a target data determiner configured to predict target data corresponding to the plurality of medical images using result values calculated through the deep neural network.

Other specific details of the invention are included in the detailed description and drawings.

According to the present invention as described above, it has various effects as follows.

The present invention does not stop at simply extracting and combining feature values from a plurality of medical images, and can increase the prediction probability of target data by additionally using correlation information.

The present invention can predict the malignancy of cancer only with a plurality of medical images without performing a tumor biopsy.

In addition, the present invention can improve the accuracy of predicting cancer malignancy by using correlation information calculated from medical images acquired in different methods together with feature values of a plurality of medical images, rather than using only one medical image. there is.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is an exemplary view showing a T2-weighted MR image and an ADC map according to Gleason evaluation scores.
2 is a block diagram for explaining a target data prediction apparatus according to an embodiment of the present invention.
3 is a flowchart illustrating a method for predicting target data according to an embodiment of the present invention.
4 is a flowchart illustrating a method for preprocessing a medical image according to an embodiment of the present invention.
5 is an exemplary diagram for explaining a medical image preprocessing method according to an embodiment of the present invention.
6 is a table showing feature values according to an embodiment of the present invention.
7 is a block diagram for explaining a method for predicting target data according to an embodiment of the present invention.
8 is a table for explaining an effect of a method for predicting target data according to an embodiment of the present invention.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, only these embodiments are intended to complete the disclosure of the present invention, and are common in the art to which the present invention belongs. It is provided to fully inform the person skilled in the art of the scope of the invention, and the invention is only defined by the scope of the claims.

Terminology used herein is for describing the embodiments and is not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. As used herein, "comprises" and/or "comprising" does not exclude the presence or addition of one or more other elements other than the recited elements. Like reference numerals throughout the specification refer to like elements, and “and/or” includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various components, these components are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first element mentioned below may also be the second element within the technical spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used with meanings commonly understood by those skilled in the art to which the present invention belongs. In addition, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless explicitly specifically defined.

The spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe a component's correlation with other components. Spatially relative terms should be understood as including different orientations of elements in use or operation in addition to the orientations shown in the drawings. For example, if you flip a component that is shown in a drawing, a component described as "below" or "beneath" another component will be placed "above" the other component. can Thus, the exemplary term “below” may include directions of both below and above. Components may also be oriented in other orientations, and thus spatially relative terms may be interpreted according to orientation.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

2 is a block diagram for explaining a target data prediction apparatus according to an embodiment of the present invention.

Referring to FIG. 2 , the target data prediction apparatus 100 according to an embodiment of the present invention may predict target data (eg, tumor malignancy) using a plurality of acquired medical images. For example, the target data prediction apparatus 100 may calculate correlation information using feature values extracted from a plurality of medical images acquired in different ways, and predict target data using the extracted feature values and the correlation information. there is. Here, the target data may be a ground truth to be derived by repeatedly machine learning steps of extracting feature values, generating a combined feature matrix, calculating correlation information, and using the calculated correlation information and acquired feature values. Accordingly, the target data may include tumor malignancy, and more specifically, prostate cancer malignancy.

In one embodiment, the target data prediction device 100 may be an electronic device such as a server or a PC, and a dedicated program capable of setting a method of predicting target data may be installed. For example, the target data prediction apparatus 100 includes an image processing unit 110 capable of obtaining medical images, pre-processing medical images, and setting medical images, and extracting image feature values for extracting radiomic features from medical images. A unit 120, a correlation information generation unit 130 that generates a combined feature matrix and correlation information using feature values, performs machine learning using a combined feature value obtained by combining feature values and correlation information as an input value, and performs machine learning, A deep learning unit 140 that generates data prediction values, a tumor determination unit 150 that determines target data based on the target data prediction values, and a plurality of medical images, extracted feature values, correlation information, combined feature values, and machine It may include a database 160 capable of converting learning results, structures and filter values of deep neural networks, patient data, target data prediction results, and the like into big data and storing them.

3 is a flowchart illustrating a method for predicting target data according to an embodiment of the present invention. 4 is a flowchart illustrating a method for preprocessing a medical image according to an embodiment of the present invention. 5 is an exemplary diagram for explaining a medical image preprocessing method according to an embodiment of the present invention. 6 is a table showing feature values according to an embodiment of the present invention. 7 is a block diagram for explaining a method for predicting target data according to an embodiment of the present invention. 8 is a table for explaining an effect of a method for predicting target data according to an embodiment of the present invention.

Meanwhile, as described above, the target data may be ground truth to be derived through deep learning. That is, the ground truth to be derived is the target data by repeatedly machine learning the steps of extracting feature values, generating the combined feature matrix, calculating the correlation information, and using the calculated correlation information and the acquired feature values, which will be described below. can be Accordingly, the target data may be any data that can be diagnosed based on feature values extracted using medical images. That is, for example, the target data may be one of numerous medical data, such as tumor malignancy, patient's bone fracture, and organ damage. However, for convenience of explanation, while the tumor malignancy will be described below as an example, prostate cancer will be additionally described as an example in more detail. Accordingly, the present invention is not limited to tumor malignancy or prostate cancer, and may include all types of data that can be diagnosed through medical images.

Referring to FIGS. 2 to 8 , in one embodiment, in operation 31, the image processing unit 110 may obtain a plurality of medical images using different methods. That is, the plurality of medical images used in the present invention may each include a tumor located in the same organ but may use different imaging devices, or may use the same imaging device but have different imaging conditions.

For example, medical images include X-ray images, CT images, and MRI images, and there is no particular limitation as long as they are medical images. "Image" means a subject collected by computed tomography (CT), magnetic resonance imaging (MRI), ultrasound, or any other medical imaging system known in the art. It may be a medical image of Medical images are voxel data, and may include a plurality of slices, that is, a plurality of unit images.

For example, a plurality of medical images may be acquired by the same imaging device under different conditions or through different imaging devices. That is, a plurality of medical images are a plurality of images obtained by taking the same organ with different imaging devices such as MR, CT, PET, or the same device (MRI) but with different imaging conditions (T2-weighted MR image, ADC map). it could be a video. In addition, in the case of medical images such as CT or MRI, the brightness and contrast of the image can be adjusted through the window setting, and the window setting consists of a window level and a window width. A plurality of medical images to which different window setting values are applied may be obtained. That is, for example, one window setting may be a case where the window width and window level are adjusted so that the entire region of the organ can be clearly seen in the medical image, and the other window setting is between the area where the tumor is located and the surrounding area of the tumor in the medical image. It may be the case that the window width and window level are adjusted so that the boundary (or the important region where the tumor is located) is clearly visible.

Referring to FIGS. 4 and 5 , the method for predicting tumor malignancy according to the present invention may further include pre-processing a plurality of medical images obtained by different methods to increase prediction accuracy.

In an embodiment, the image processing unit 110 may acquire a plurality of medical images by different methods in operation 41 and perform intensity normalization on the plurality of medical images in operation 42 .

Specifically, since the range of signal intensity between MR images of the same parameter differs according to the protocol used in the process of acquiring medical images of patients, it is difficult to extract consistent feature values. Normalize MR images of the same parameters between patients through histogram stretching By doing so, a consistent feature vector can be extracted. In order to use the same range of signal intensity between patients, histogram stretching is performed through Equation 1 below to normalize the signal intensity range. there is.

[Equation 1]

At this time, I represents the signal intensity of the original image, Imax represents the maximum value of the signal intensity in the original image, Imin represents the minimum value of the signal intensity in the original image, I' max represents the maximum value of the signal intensity range to be normalized, I ' min represents the minimum value of the signal strength range to be normalized.

In an embodiment, in operation 43 of the image processing unit 110, a plurality of medical images may be cropped to divide a region of interest in the plurality of medical images. For example, as shown in FIG. 5, the image processing unit 110 may acquire a T2-weighted MR image (T2wMR in (a)) and an ADC map (ADC in (b)), and derive it through machine learning in advance. Using the segmented tumor mask in (c), the region of interest (Tumor ROI of T2wMR in (d)) images of the T2-weighted MR image and the region of interest images of the ADC map (Tumor ROI of (e)) ADC) can be obtained.

In an embodiment, in operation 32, the image feature value extractor 120 may extract a feature value for each medical image from a plurality of medical images. Here, the type and number of feature values are not limited to the table shown in FIG. 6 . That is, for example, a total of 68 feature values shown in FIG. 6 may not be extracted for each medical image, the number may be changed, and the type is not limited to that shown in FIG. 6 . In the present invention, it is important to extract and utilize feature values for each of a plurality of medical images rather than the type and number of feature values.

For example, the plurality of feature values may include at least one of a histogram feature value, a texture feature value, and a shape feature value. The histogram feature values may include first order statistics and histogram percentile intensities shown in FIG. 6, and the texture feature values include Gray Level Co-occurrence Matrix (GLCM), Gray Level Run Length Matrix (GLRLM), and Local Binary Pattern ), and the shape feature values may include area/perimeter ratio, convex area, eccentricity, Euler number, major-minor axis ratio, major axis length, minor axis length, area, and perimeter.

Specifically, histogram feature vectors include mean, minimum value (min), maximum value (max), variance, standard deviation, skewness, kurtosis, and entropy ) can be extracted. The histogram feature vector can be calculated through Equation 2 below.

[Equation 2]

는 전립선 PZ 내에 위치하는 윈도우의 i번째 화소에서 영상의 신호강도를 의미하고,

는 윈도우 내 영상의 신호강도 평균값을 의미한다.

는 영상 히스토그램에서

의 신호강도 확률을 의미한다. 한편, 최소값(min)은 최소 신호강도 값, 최대값(max)은 최대 신호강도 값일 수 있고, 표준편차(standard deviation)는 분산에 제곱근을 적용하여 연산할 수 있다.At this time, n means the size of the window of the medical image,

Means the signal intensity of the image at the ith pixel of the window located in the prostate PZ,

Means the average value of the signal intensity of the image within the window.

in the image histogram

means the signal strength probability of Meanwhile, the minimum value (min) may be the minimum signal strength value, the maximum value (max) may be the maximum signal strength value, and the standard deviation (standard deviation) may be calculated by applying a square root to the variance.

Meanwhile, for the histogram percentage intensity, an average value, a maximum value, a minimum value, etc. may be extracted as feature values based on 5%, 25%, 50%, 75%, and 95%, respectively.

For example, GLCM, GLRLM, and LBP can be extracted as texture feature vectors. GLCM is a probability-based matrix that analyzes the number of simultaneous occurrences of two pixel pairs to express texture information in an image. As in 3, feature values of medical images can be extracted. Meanwhile, after obtaining the following 7 values, a total of 14 feature values may be utilized by calculating the average value (min) and standard deviation (standard deviation) of each value. Of course, this number is exemplary and other numbers may be applied. Meanwhile, ASM (Angular Second Moment) may be calculated through the following energy equation.

[Equation 3]

At this time, i and j mean two pixel pairs having signal strength information, d is the distance between two pixels used to calculate the contrast co-occurrence matrix, θ means the direction, and p(i, j) is It means the probability value of the position (i, j) of the intensity co-occurrence matrix, and N may mean the size of the intensity co-occurrence matrix.

For example, GLRLM is a probability-based matrix that analyzes the length of two or more pixels as a method of expressing texture information through an image. Through the GLRLM obtained in the degree direction, the feature vector of the image can be calculated as shown in Equation 4 below. Meanwhile, after obtaining 11 values through the following 11 equations, a total of 22 feature values may be utilized by calculating the average value (min) and standard deviation of each value. Of course, this number is exemplary and other numbers may be applied.

[Equation 4]

At this time, i is the signal strength of the pixel, j is the number of pixels with the same signal strength appearing consecutively with i, and GLRLM (i, j) is the probability value of the (i, j) position of the GLRLM, and H denotes the size of the intensity action length matrix.

For example, LBP is a feature vector representing texture characteristics around a pixel. It compares a pixel with the signal intensity of neighboring pixels, expresses the result as a threshold value, and aligns it. A feature vector having can be calculated as shown in Equation 5. On the other hand, after obtaining the following three values, the average value (min), minimum value (min), maximum value (max), variance, standard deviation, etc. of each value are calculated to obtain a total of 10 feature values. can be utilized Of course, this number is exemplary and other numbers may be applied.

[Equation 5]

At this time, gc means the central pixel, gp means the neighboring pixel, and m0 and m1 mean the average of the signal intensities of the neighboring pixels converted to the threshold.

In one embodiment, in operation 33, the correlation information generating unit 130 may generate a joint feature matrix based on feature values obtained as a result of the extraction. For example, a histogram feature value, a texture feature value, and a shape feature value may be extracted for each medical image, and a matrix may be generated using each feature value. For example, if the description is made with two medical images as shown in FIG. 7 for convenience of description, a first feature value and a second feature value are obtained from a first medical image and a second medical image obtained by different methods, respectively. this can be extracted. Here, the first feature value and the second feature value may each include 68 feature values shown in FIG. 6 . And, for example, the correlation information generating unit 130 may generate a combined feature matrix in which the first feature value is set as x-axis data and the second feature value is set as y-axis data. Here, the combined feature matrix may be a combination of feature values of a plurality of medical images in a matrix form. Thus, the manner in which the joint feature matrix is generated can vary.

In addition, when there is a third medical image obtained by a method different from the first medical image and the second medical image, a third feature value may be extracted from the third medical image, and the first feature value, the second feature value, and the A combined feature matrix) may be generated based on the third feature value. For example, the correlation information generator 130 may generate a combined feature matrix in which the first feature value is set as x-axis data, the second feature value is set as y-axis data, and the third feature value is set as z-axis data. there is. That is, in this way, the combined feature matrix may be generated based on three medical images and, of course, may be extended to four or more.

In one embodiment, in operation 34, the correlation information generating unit 130 may calculate correlation information based on the generated combination feature matrix. Here, the correlation information may be calculated through a combined feature matrix including a first feature value and a second feature value, for example, as shown in FIG. 7 . This may mean that the correlation information becomes information capable of representing all feature values of a plurality of medical images.

For example, the correlation information may be calculated based on a gray level co-occurrence matrix (GLCM) considering a brightness value and location information of a pixel.

For example, the intensity co-occurrence matrix (GLCM) is a statistical method for examining a texture by considering the spatial relationship of pixels, and is a spatial intensity dependence matrix. The GLCM function is a method for identifying medical images by generating GLCM by calculating how often pixel pairs having a specific value occur in a specific spatial relationship in an image, and extracting statistical measurement values from this matrix as feature values.

Therefore, it is possible to convert each feature value of a plurality of medical images into a GLCM based on a combined feature matrix generated by combining, and extract feature values from the converted GLCM, and these feature values determine the relationship between the plurality of medical images. It can be correlation information representing . Of course, the method of calculating correlation information using GLCM is only an example, and correlation information may be extracted using other functions or algorithms such as GLRLM based on the joint feature matrix.

In an embodiment, in operation 35, the correlation information generating unit 130 may generate a combined feature value by concatenating the computed correlation information and the acquired feature values. For example, as shown in FIG. 7, a combined feature value is generated by connecting (eg, simple sum, average, maximum value, etc.) the first feature value, the second feature value, and correlation information (eg GLCM feature value) can do. For example, as shown in FIG. 6, if the first feature value is 68 and the second feature value is 68, and 14 feature values are extracted using GLCM for correlation information as described above, the combined feature value can be 150. there is. Of course, these numbers are just examples, and the type and number of feature values may be variously modified.

In one embodiment, in operation 36, the deep learning unit 140 may input the combined feature value to the deep neural network that has been previously learned and calculate a resultant value through the deep neural network. For example, a deep neural network may be a classifier and may be a random forest classifier (RFC). The classifier may have different ground truth depending on the type of tumor to be learned, and in the case of predicting the malignancy of prostate cancer, the result of a tissue biopsy may be used as the ground truth. Of course, logistic regression analysis (LR), k nearest neighbor (kNN), support vector machine (SVM), and the like may be used as classifiers.

In an embodiment, in operation 37, the tumor determiner 150 may predict the malignancy of the tumor corresponding to the plurality of medical images using the result value calculated through the deep neural network. For example, the resultant value may be a quantified value such as the Gleason evaluation score, and malignancy may be predicted according to the numerical value of the score, such as the Gleason evaluation score.

That is, in this way, the present invention can improve the prediction accuracy of tumor malignancy by utilizing not only feature values of each medical image but also correlation information between medical images.

On the other hand, the data set used to confirm the effect of the present invention utilized 210 cases of the low Gleason evaluation score group and 156 cases of the high Gleason evaluation score group. The low Gleason score group includes 14 cases with a 3+3 score and 196 cases with a 3+4 score, which are lower than the first grade 3 and the second grade 4. The high Gleason score group consisted of 116 cases with a 4+3 score, 16 cases with a 4+4 score, 21 cases with a 4+5 score, and 3 cases with a 5+4 score, which were higher than 1st grade 4 and 2nd grade 3. includes

Referring to FIG. 8, A is the result of learning the 68 feature values shown in FIG. 6 of the T2-weighted MR image (T2wMR) with the classifier, and B is the classifier with the 68 feature values shown in FIG. 6 of the ADC map. C is the result of learning 136 feature values with the classifier as the combined value of A and B feature values (Concatenated features), D is the average value of A feature value and B feature value (Averaged features, 68 pieces) is the result of learning with the classifier, E is the result of learning the value obtained by multiplying the A feature value and the B feature value (Multiplied features, 68) with the classifier, and F is the correlation information (feature value based on the combined feature matrix, 14 features). ) is the result of learning with the classifier, and G is the result of learning the concatenated joint feature matrix features (150) of the present invention with the classifier. As shown in FIG. 8, 6 evaluation criteria of the 7 methods were determined in the classifier learning result. The six evaluation criteria are accuracy (ac), sensitivity (sen), specificity (spe), positive predictive value (PPV), negative predictive value (NPV), and AUC (classifier operating characteristic curve). Sensitivity and PPV are the ratio of true positives to actual Gleason rating score cases and predicted cases, respectively. Specificity and NPV are the ratio of true negatives to predicted cases versus actual Gleason score cases, correspondingly. Finally, AUC, like the actual Gleason score, represents the probability of a classifier accurately discriminating tumor malignancy. Referring to FIG. 8, the G method of the present invention showed the best performance in AUC, accuracy, sensitivity, and positive predictive value, and showed above-average performance even in negative predictive value, and when comprehensively considering the six evaluation criteria, the G method showed the best performance in predicting tumor malignancy. It was confirmed that excellent performance was exhibited.

A method of predicting target data using correlation information based on different medical images according to an embodiment of the present invention includes acquiring a plurality of medical images using different methods; extracting feature values for each medical image from the plurality of medical images; generating a joint feature matrix based on feature values obtained as a result of the extraction; calculating correlation information based on the generated combined feature matrix; and predicting target data using the calculated correlation information and the obtained feature values.

According to various embodiments, the target data is to be derived by repeatedly machine learning the feature value extraction, the combined feature matrix generation, the correlation information calculation, and the steps of using the calculated correlation information and the acquired feature values. It can be ground truth.

According to various embodiments, the target data may be tumor malignancy.

According to various embodiments, the target data may be prostate cancer.

According to various embodiments of the present disclosure, the plurality of medical images may be obtained from the same imaging device under different conditions or through different imaging devices.

According to various embodiments, the predicting of the target data may include generating a combined feature value by concatenating the computed correlation information and the acquired feature values.

According to various embodiments of the present disclosure, the predicting of the target data may include inputting the combined feature values to a pre-learned deep neural network and corresponding to the plurality of medical images using result values calculated through the deep neural network. Predicting the target data to be; may further include.

According to various embodiments, each of the plurality of medical images includes a first medical image and a second medical image obtained by different methods, and a first characteristic value and a first feature value are obtained from the first medical image and the second medical image, respectively. extracting a second feature value; generating the combined feature matrix based on the first feature value and the second feature value; and calculating the correlation information based on the combined feature matrix.

According to various embodiments of the present disclosure, obtaining a third medical image using a method different from that of the first medical image and the second medical image; extracting a third feature value from the third medical image; generating the joint feature matrix based on the first feature value, the second feature value, and the third feature value; and calculating the correlation information based on the combined feature matrix.

According to various embodiments, the correlation information may be calculated based on a gray level co-occurrence matrix (GLCM) considering brightness values and position information of pixels.

According to various embodiments, the feature values may include at least one of a histogram feature value, a texture feature value, and a shape feature value.

According to various embodiments, the method may further include pre-processing the plurality of medical images, and the pre-processing may include performing intensity normalization on the plurality of medical images; and cropping the plurality of medical images to divide a region of interest in the plurality of medical images.

According to various embodiments, a tumor malignancy prediction program using correlation information based on different medical images may be combined with a hardware computer and stored in a medium to execute the method of any one of claims 1 to 9. there is.

Steps of a method or algorithm described in connection with an embodiment of the present invention may be implemented directly in hardware, implemented in a software module executed by hardware, or implemented by a combination thereof. A software module may include random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any form of computer readable recording medium well known in the art to which the present invention pertains.

Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains can be implemented in other specific forms without changing the technical spirit or essential features of the present invention. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

100: target data prediction device
110: image processing unit
120: image feature value extraction unit
130: correlation information generation unit
140: deep learning unit
150: target data determination unit
160: database

Claims (11)

A method for predicting target data using correlation information based on different medical images,
acquiring a plurality of medical images using different methods;
pre-processing the plurality of medical images to improve prediction accuracy;
extracting feature values for each medical image from the plurality of medical images;
generating a joint feature matrix by combining feature values obtained as a result of the extraction in a matrix form;
calculating correlation information based on the generated combined feature matrix; and
generating a combined feature value by concatenating the computed correlation information and the acquired feature values; and
Inputting the combined feature values into a pre-learned deep neural network and predicting target data corresponding to the plurality of medical images using result values calculated through the deep neural network; Target data prediction method using correlation information based on different medical images.
According to claim 1,
The target data is the ground truth to be derived through repeated machine learning of the feature value extraction, the combined feature matrix generation, the correlation information calculation, and the steps of using the calculated correlation information and the acquired feature values. A method of predicting target data using correlation information based on different medical images.
According to claim 2,
The target data prediction method using correlation information based on different medical images, characterized in that the target data is tumor malignancy.
According to claim 3,
The target data prediction method using correlation information based on different medical images, characterized in that the target data is prostate cancer.
According to claim 1,
The method of predicting target data using correlation information based on different medical images, characterized in that the plurality of medical images are acquired under different conditions from the same imaging device or through different imaging devices.
According to claim 1,
The correlation information is target data using correlation information based on different medical images, characterized in that it is calculated based on a gray level co-occurrence matrix (GLCM) considering brightness values and location information of pixels. prediction method.
According to claim 1,
The method of predicting target data using correlation information based on different medical images, characterized in that the feature values include at least one of a histogram feature value, a texture feature value, and a shape feature value.
An apparatus for predicting target data using correlation information based on different medical images,
an image processing unit that acquires a plurality of medical images by using different methods and pre-processes the plurality of medical images to improve prediction accuracy;
an image feature value extraction unit extracting feature values for each medical image from the plurality of medical images and generating a joint feature matrix by combining the feature values acquired as a result of the extraction in a matrix form;
a correlation information generating unit calculating correlation information based on the generated combined feature matrix;
a deep learning unit inputting a concatenated feature value obtained by concatenating the computed correlation information and the acquired feature values to a pre-learned deep neural network, and calculating a resultant value through the deep neural network; and
A target data prediction device using correlation information based on different medical images, comprising: a target data determination unit that predicts target data corresponding to the plurality of medical images using result values calculated through the deep neural network.
According to claim 8,
The target data is the ground truth to be derived through repeated machine learning of the feature value extraction, the combined feature matrix generation, the correlation information calculation, and the steps of using the calculated correlation information and the obtained feature values. Target data prediction device using correlation information based on different medical images.
According to claim 8,
The correlation information is target data using correlation information based on different medical images, characterized in that it is calculated based on a gray level co-occurrence matrix (GLCM) considering brightness values and location information of pixels. prediction device.
A computer program for predicting target data using correlation information based on different medical images stored in a medium in order to execute the method of any one of claims 1 to 7, coupled to a computer that is hardware.

Images