KR101443187B1 - medical image retrieval method based on image clustering - Google Patents

Abstract

The present invention relates to an image clustering-based medical image retrieval method, which comprises: (a) dividing each input medical image into sub regions and extracting an OCS-LBP (Oriented Center Symmetric Local Binary Patterns) step; (b) clustering the feature values into a code book, and converting each of the images divided in the patch unit into a Bag-of-Feature (BoF) feature through the code book; (c) generating the BoF feature as a feature vector, and learning and classifying the feature vector through a random forest learning classifier; (d) repeating the steps (a) to (c) when the query image is input, and measuring similarity with the medical image through a small number of specific classes of the classified class; And (e) outputting a search output image according to the measured similarity.
The present invention proposes a new OCS-LBP feature suitable for medical images in order to efficiently search for medical images, applies BoF to effectively reduce feature dimensions, The present invention provides a medical image retrieval method that displays high-performance retrieval results by a similarity distance measurement through a random forest classifier instead of matching the entire images.

Description

[0001] The present invention relates to a medical image retrieval method based on image clustering,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a medical image retrieval method, and more particularly, to a medical image retrieval method based on image clustering which shows a high-performance retrieval result.

Recently, as the medical industry develops, there is an increasing need to manage various medical images obtained from medical related equipment. In addition, research and development on integrated medical information system have been actively carried out according to the necessity of fusion research of IT industry and medical industry. In the medical field, the PACS (Picture Archiving Communication System), which is a medical support system, and the Digital Imaging and Communications in Medicine (DICOM), a standard network protocol for medical images, are based on various IT technologies such as image processing, ) Were established.

However, since the PACS system currently used depends on the keyword previously input by the administrator for the medical image, there is a limit in that it can not retrieve various medical information included in the image. In addition, keyword-based medical image retrieval is a very inefficient method in terms of cost and time for inputting and managing keywords although the accuracy of the applied keywords is excellent. In order to overcome these limitations, a content - based image retrieval method that extracts feature values included in medical images and uses them for retrieval, and image classification methods that classify medical images by type have been studied for the past 10 years.

In the previous study, we set the initial importance area and weighted the areas according to predefined criteria. Then, we propose a method of retrieving medical images through comparison of characteristic similarities of important regions set in each important region and query regions.

In another study, the visual features of the image were extracted using the color structure descriptor and the texture descriptor defined in MPEG-7, and the images were classified using a multi-class SVM (Multi-class Support Vector Machine). We proposed a medical image retrieval method using an ensemble vector that combines membership scores from each classifier, by learning different multi - class SVMs for each feature descriptor.

Also, there is a research that proposed GIFT (GNU Image Finding Tool) system that extracts color and texture features from images and classifies images automatically through SVM classifier.

In another study, we segmented the X-ray image into regional patches and extracted features from each patch to create a bag-of-feature (BoF). In addition, we proposed an algorithm that divides and retrieves images by learning BoF feature and kernel based SVM. However, as the dimension of BoF increases, the classification performance using SVM improves, but the classification time increases in inverse proportion to the performance.

In addition, the medical images were divided into patches, and BoF was generated using the SIFT (Scale Invariant Feature Transform) feature extracted from each patch. We then proposed a method of searching images by applying different weights to the visual words corresponding to each bin of the BoF using a boosting algorithm.

Then, we classify the medical images into predefined categories through image classification, and propose an algorithm that assigns annotations to each image by body graph of each image. We also proposed a novel image retrieval method based on keyword by annotation.

Thus, in the conventional research, X-ray and CT image are generally composed of gray images, and the background and the foreground are clearly distinguished from each other. Therefore, the characteristic values that can be applied to the natural image are limited and the classification and the searching method different from the natural image are required.

SUMMARY OF THE INVENTION [0006] The present invention, which is intended to solve the problems described above, proposes a new feature suitable for a medical image to effectively search for a medical image, applies a method for effectively reducing the feature dimension, And to provide a new medical image retrieval method that displays high-performance retrieval results through a classifier that matches the entire image with respect to the query image.

According to an aspect of the present invention, there is provided a method of analyzing a medical image, the method comprising: (a) dividing each input medical image into sub regions and extracting an OCS-LBP (Oriented Center Symmetric Local Binary Patterns) (b) clustering the feature values into a code book, and converting each of the images divided in the patch unit into a Bag-of-Feature (BoF) feature through the code book; (c) generating the BoF feature as a feature vector, and learning and classifying the feature vector through a random forest learning classifier; (d) repeating the steps (a) to (c) when the query image is input, and measuring similarity with the medical image through a small number of specific classes of the classified class; And (e) outputting a search output image according to the measured similarity.

The step (a) may include dividing each medical image into 2 × 2 sub-areas and dividing each of the sub-areas into 4 × 4 patches. Extracting the OCS-LBP pattern for each of the divided patches; And generating a histogram for each of the sub regions to extract a feature value.

The step (b) may further include generating a code book by applying K average clustering to the feature values of the respective input images; And generating a Boolean (Bag-of-Feature) histogram through the code book.

In addition, the step (c) includes the steps of: (d-1) setting the number T of decision trees constituting the random forest; (d-2) selecting m random variables from the feature vector set, and dividing the learning data using the determination function of each node and the random variable; (d-3) growing the crystal tree to a maximum depth; And repeating the steps (d-2) and (d-4) if any one of the trees is completed and the number of the completed trees is less than the T.

The step (d) may further include: selecting a K-neighbor neighborhood (K-NN) class among the classes of the query image by the random forest classifier; Preferably, the method further comprises measuring a similarity distance only for an image labeled with the K classes of the input image. Preferably, K is 2, and the similarity distance is a BHattacharyya distance desirable.

The present invention proposes a new OCS-LBP feature suitable for medical images to effectively search for medical images, applies BoF not only to effectively reduce feature dimensions but also to improve query speed and performance The present invention provides a medical image retrieval method that displays high-performance retrieval results by a similarity distance measurement through a random forest classifier instead of matching the entire images.

In addition, the present invention proves that among the various similarity distance measurement algorithms, the best method of measuring the distance of the Bacharachia is experimentally proved, and when the proposed random forest classifier is combined with the Bacharya distance measurement method, Which is about 28%.

As shown in the experimental results, the features proposed in the present invention show improved search results compared to the existing features, and it has been confirmed that the proposed system can effectively improve the search time as well as the search performance.

FIG. 1 is a flowchart illustrating a method of searching for a medical image based on image clustering according to an embodiment of the present invention,
FIG. 2 is a schematic diagram of an algorithm sequence of a medical image retrieval method based on image clustering according to an embodiment of the present invention,
FIG. 3 is a schematic diagram of a sub region segmentation method in an image clustering based medical image search method according to an embodiment of the present invention,
4 is a schematic diagram of a method of generating an OCS-LBP histogram in a medical image retrieval method based on image clustering in an embodiment of the present invention,
FIG. 5 shows a BoF histogram of an input image generated using a codebook in a medical image searching method based on image clustering according to an embodiment of the present invention,
FIG. 6 is a schematic diagram illustrating a classification process of a database image in a medical image search method based on image clustering according to an exemplary embodiment of the present invention,
FIG. 7 is a graph comparing image retrieval performance of each distance using the image clustering-based medical image retrieval method according to the embodiment of the present invention,
FIG. 8 is a graph comparing image retrieval performance of each feature applying the image clustering-based medical image retrieval method according to the embodiment of the present invention,
FIG. 9 is a graph illustrating a comparison between a random forest learning classifier (RF) applying image clustering based medical image search method according to an embodiment of the present invention and image search performance combining each feature,
FIG. 10 is a graph showing image retrieval performance and search time according to a change of K value in a K-proximity neighbor class applying the image clustering-based medical image retrieval method according to an embodiment of the present invention,
11 is a graph illustrating a comparison between a method of applying the image clustering-based medical image search method and a search performance using MSVM according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention, and how to accomplish it, will be described with reference to the embodiments described in detail below with reference to the accompanying drawings. However, the present invention is not limited to the embodiments described herein but may be embodied in other forms. The embodiments are provided so that those skilled in the art can easily carry out the technical idea of the present invention to those skilled in the art.

In the drawings, embodiments of the present invention are not limited to the specific forms shown and are exaggerated for clarity. Also, the same reference numerals denote the same components throughout the specification.

The expression "and / or" is used herein to mean including at least one of the elements listed before and after. Also, singular forms include plural forms unless the context clearly dictates otherwise. Also, components, steps, operations and elements referred to in the specification as " comprises "or" comprising " refer to the presence or addition of one or more other components, steps, operations, elements, and / or devices.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a flowchart illustrating an image clustering-based medical image retrieval method according to an embodiment of the present invention. FIG. 2 is a schematic diagram of an algorithm sequence of an image clustering-based medical image retrieval method according to an embodiment of the present invention. As shown in FIG. 1, the medical image searching method according to the embodiment of the present invention includes the steps of (a) dividing each input medical image into sub regions, and performing OCS-LBP (Oriented Center Symmetric Local Binary Patterns) Extracting a value (S100); (b) clustering the feature values into a code book, and converting each of the images divided by the patch unit into a Bag-of-Feature (BoF) feature through the code book (S200 ); (c) generating the BoF feature as a feature vector, and learning and classifying the feature vector through a random forest learning classifier (S300); (d) repeating the steps (a) to (c) when the query image is input (S410), and measuring the degree of similarity with the medical image through a small number of specific classes among the classified classes (S430); And (e) outputting a search output image according to the measured degree of similarity (S500).

As described above, according to the present invention, since the characteristic values of the medical image such as the X-ray and the CT image are reflected, the OCS-LBP (Oriented Center Symmetric Local Binary Patterns) feature (S100).

2, the feature values extracted from the learning image are grouped into a code book in the step of generating a bag-of-feature (BoF), and each of the images is converted into a meaningful new dimension The BoF feature is transformed into a BoF (Bag-of-Feature) feature (S200). The extracted BoF feature is applied to the random forest classifier stage of FIG. 2 and classified into N classes having similar characteristics by the learned classifier (S300) , The same OCS-LBP feature is extracted and the BoF feature is extracted through the code book (S410)

Then, in general, the similarity measure uses a method in which the query image retrieves all the images in the database, but in the present invention, the BoF feature extracted from the query image is applied to the learned random forest to obtain the most similar K-nearest neighbor neighbor, and KNN) classes, and searches only the images included in the selected classes through the BoF similarity measurement with the query image (S430).

Hereinafter, a flow of a medical image retrieval method based on image clustering according to an embodiment of the present invention will be described in detail.

First, in step (a) (S100), the step of extracting feature values using OCS-LBP extracts important feature points of the image, which is the most used SIFT feature in general image search and object recognition, Information and size are used as feature information. However, unlike a natural image, medical images often have no significant feature points in a medically important area. Therefore, since there is a limit to search for similar images using only SIFT features, the present invention does not extract feature points from an image, Similar to the image search method, a method of dividing an image into sub-regions and extracting features in patch units is used.

In the embodiment of the present invention, similar to the existing methods, features are extracted on a patch-by-patch basis, but features extracted from each patch unit are combined into one sub-region unit by placing sub-regions, .

3 is a schematic diagram of a sub region segmentation method in a medical image search method based on image clustering according to an embodiment of the present invention. As shown in FIG. 3, an image is divided into 2 × 2 sub-images, and each sub-image is further divided into 4 × 4 to construct each patch, and features extracted from each patch are combined and used as one sub-region feature .

Local Binary Patterns (LBP), which is a regional binary pattern technique, is mainly used for analyzing and expressing the texture component of image in image search. LBP is a kind of operator representing the relationship of neighboring pixels with respect to the center pixel, and expresses the center pixel as a brightness value (P, R) obtained by dividing a circle having a radius of R into P equal coordinates.

However, this method is not suitable for real-time search because feature vectors are combined into feature (256) dimensions and feature extracted for each patch, resulting in 4096 (4 × 4 × 256) dimensions of one sub-region. In order to solve this problem, the conventional research used CS-LBP (Center-symmetric LBP), which is a texture feature that has similar performance to LBP and has fewer feature dimensions.

CS-LBP is a kind of operator representing the relationship of neighboring pixels of the center pixel. The difference between the center pixel and the neighboring four pixels having a distance of 1 is calculated, and the result is aligned counterclockwise to obtain the CS-LBP value for the center pixel. LBP requires a total of 256 dimensions, but CS-LBP is suitable for real-time processing because it is reduced to 256 (4 × 4 × 16) dimensions when total (16)

We propose an Oriented CS-LBP that improves CS-LBP so that feature dimension can be reduced and the gradient direction information of image can be extracted effectively.

4 is a schematic diagram of an OCS-LBP histogram generation method in a medical image retrieval method based on image clustering in an embodiment of the present invention. As shown in FIG. 4, the OCS-LBP compares the pixel values of the symmetric regions and finds a direction having a large value when the difference of the values is equal to or greater than the threshold value T by Equation (1) To accumulate the difference values.

In Equation (2), n _i and n _{i + (N / 2)} denote the values of pixels symmetrical to each other in all the pixels N included in the R region.

 That is, in the embodiment of the present invention, in order to develop a search algorithm robust against a local deformation of an image, the image is divided into 2 × 2 regions and divided into 4 × 4 patches for each sub region, and eight OCS-LBP Extract the pattern. The histograms of 128 (4 × 4 × 8) dimensions are generated for each sub-region through the combining process of the extracted patches of each sub-region, and thus the total feature dimension extracted from one image is 2 × 2 × 128 = 512 Feature is generated. However, if the dimension of the feature vector is too large, real-time image retrieval is impossible. Therefore, the BoF histogram algorithm is applied to transform the feature dimension into a meaningful reduced dimension.

That is, in step (b) S200, the step of transforming into the feature of Bag-of-Feature (BF) is performed by first performing K-mean clustering on the features of 512 dimensions extracted from each learning image, And collects the center vectors of each cluster to generate a code book. In the code book, the center vector of each cluster is called a code word. Then, the same OCS-LBP feature is extracted from the input image and applied to a code book so that the distance is voted on the most similar codeword, thereby generating a BoF histogram as shown in FIG. In the embodiment of the present invention, the code book has the best performance when the codebook size is 200. FIG. 5 is a BoF histogram of an input image generated using a codebook in an image clustering-based medical image search method according to an embodiment of the present invention.

(c) Step (S300) is a learning and classification step by a random forest learning classifier. The random forest classifier is a classifier using a decision tree ensemble. Since the classifier is based on a combination of binary decision trees, It has a speed of training and is known as an excellent classifier for executing large amounts of data.

In the embodiment of the present invention, the BoF histogram is extracted from the learning data and the random forest classifier is learned using the algorithm of Table 1. The CT image used in the embodiment of the present invention is a tomographic image of each body part, and is classified into 14 classes according to the part. Therefore, we learned to classify 14 random forest classifiers into 4 classes and use 499 image data sampled at each site for learning. The process of learning random forest is shown in [Table 1].

1. Set the number T of decision trees to construct a random forest

2. Select m random variables from the input BoF histogram feature vector set.
The decision function f (v _i ) of each node is the i
By using variables, you can repeatedly write learning data to the left and right subsets
Sent by split.

3. Each RF tree grows according to the following steps:
a. N new bootstrap samples are selected from the learning set S _n , and n
Use the bootstrap sample to automatically grow the tree.
b. At each internal node, each node randomly selects p variables,
Determine the best decision function using only variables.
c. Grow the tree up to the highest depth of the tree.

4. If one tree is completed, if it is smaller than the number of decision tree T, go to step 2 again

As described above, in the embodiment of the present invention, a 1.0% learning data set is used for each subtree in order to learn a random forest for classification, and a random forest classifier is constructed by generating a total of 40 decision trees.

Here, after the random forest classifier is learned, all the database images are applied to the random forest and the class index is given to each image by off-line.

을 갖게 되고, 총 T개의 트리 확률 값을 산술평균하여 최종적으로 각 클래스 에 대한 확률 값을 [수학식 3]을 통해 얻을 수 있다.
6 is a schematic diagram illustrating a classification process of a database image in a medical image search method based on image clustering according to an embodiment of the present invention. As shown in FIG. 6, the input image is transformed by random forest classifiers into probability values for class c _i for each tree t

, And the total T tree probability values are arithmetically averaged to finally obtain a probability value for each class through Equation (3).

은 전체 트리 집합을 의미한다.In Equation (3)

Refers to the entire tree set.

After classification of the database images, each i-th image consists of a 200-dimensional BoF histogram (P ⁱ ) and a set of class labels.

(d) repeating the steps (a) through (c) of the inputted query image in step (S410), measuring the degree of similarity between the query image and the input image (S430) The OCS-LBP feature of 512 dimensions is extracted in the same way and the 200-dimensional BoF histogram is applied to the code book.

Then, the BoF histogram of the query image is input to the random forest classifier and the classifier selects the most similar K-neighbor class (K-NN) class among the 14 classes. The reason why the K classes are selected without using one of the most similar classes in the embodiment of the present invention is that since the database images having similar shapes can be classified into different classes in the learning process, To prevent. In the present invention, the best performance was obtained when K was 2.

When a query image is input, not all database images are searched, but the same method as shown in Fig. 6 is applied to a random forest, and two closest classes are selected. Thereafter, the similarity distance measurement is performed only on the images labeled with the two closest classes.

In the embodiment of the present invention, the Bhattacharyya distance [13], which is known to be superior to the Euclidean distance measurement method traditionally used for similarity measurement, is used. The method of measuring the distance is applied as shown in [Equation 4] according to the characteristics of the present study.

을 m차원의 질의 BoF 히스토그램이라고 하고, BoF(t)를 k번째 클래스에 속하는 데이터베이스 영상의 BoF 히스토그램이라고 한다면, 두 특징간의 거리는 각 히스토그램의 관계와 k번째 클래스에 대한 분류 확률

에 의해 결정되며 결과 값이 작을수록 유사성이 높은 영상으로 검색된다.if

Is the BoF histogram of the m-dimensional query, and BoF (t) is the BoF histogram of the database image belonging to the kth class, the distance between the two features is the relationship of each histogram and the classification probability

And the smaller the result, the more similar images are retrieved.

Experiment and performance evaluation

In the present invention, the medical database used for the performance evaluation is a CT image, which is composed of 14 classes for each body part and includes a total of 1080 images.

In the first experiment, Eucldiean street, Canberra street, Manhattan street, and Bhattacharyya street used in the present invention, which are three traditional similarity distance measurement methods generally used in image search, , And proved the superiority of Batarya distance.

For performance measurement, OCS-LBP features were extracted for query images and all database images without using a random forest, and similarity was measured by generating a BoF histogram for each feature. For the performance measurement, the search results are sorted by the top K, and the precision of each top K is compared.

FIG. 7 is a graph comparing the image retrieval performance of each distance using the image clustering-based medical image retrieval method according to the embodiment of the present invention. As shown in FIG. 7, the accuracy of the Bacharya distance measure showed the best performance with an average of 50%, and Manhattan distance and Euclidian distance showed similar results with 47% and 46%. The Canberra street had the lowest performance at 43%.

In the second experiment, in order to compare the performance of the OCS-LBP feature proposed in the present invention, the Bartaglia distance is used as a basic distance measurement method, and the OCS-LBP proposed in the present invention, the CS -LBP and SIFT. FIG. 8 is a graph comparing image retrieval performance of each feature applying the image clustering-based medical image retrieval method according to the embodiment of the present invention.

As shown in FIG. 8, it can be seen that the OCS-LBP feature proposed by the present invention improves the performance from a minimum of 9% to a maximum of 32%, compared with the conventional features. This shows that the OCS-LBP feature quantifies the gradient direction and size of the image and reflects the characteristics of the medical image.

In the third experiment, the performance of image retrieval was measured by applying random forest to each feature. The random forest is pre-learned for each feature. When the query image is input, the query image is classified into a specific class as described above, and Equation (4) is applied to the upper two classes, As shown in Fig. As shown in Fig. 8, the OCS-LBP feature shows the best performance as in Fig. FIG. 9 is a graph illustrating a comparison between a random forest learning classifier (RF) applying image clustering based medical image search method according to an embodiment of the present invention and image search performance combining each feature.

As shown in FIG. 9, the proposed algorithm (RF + OCS-LBP) shows an average precision of 81%, which is 28% higher than the precision before applying the random forest.

In the fourth experiment, an experiment was conducted to determine the number of K in the algorithm that selects the most similar K-nearest neighbor class among the 14 classes for the query image. 10 is a graph illustrating image retrieval performance and search time according to a change of K value in a K-nearest neighbor class applying the image clustering-based medical image retrieval method according to an embodiment of the present invention. FIG. 10A shows the search precision, and FIG. 10B shows the search speed in seconds.

As shown in Fig. 10, when K = 2, it is found that the most efficient search performance and speed are obtained. The value of K affects the search time. The smaller the value of K, the better the search speed. However, the performance decreases and the performance increases proportionally with increasing K, but converges to the same value from K = 2. Therefore, in the present invention, K = 2 is set.

As a fifth experiment, we compared the performance of MSVM, which is most commonly used in conventional image retrieval. Experiments were performed by classifying the database images into classes according to the previously learned MSVM, and by performing the same process on the query images, the distance of the Bata- karari distance was obtained only for the K-neighbor class. 11 is a graph illustrating a comparison between a method of applying the image clustering-based medical image search method and a search performance using MSVM according to an embodiment of the present invention.

As shown in FIG. 11, it can be seen that the search rate decreases as the number of Top K increases in the MSVM. In the case of MSVM, the overall performance is about 72%, but it is about 9% lower than the proposed method.

In addition, the application of the random forest classifier in the medical image retrieval is not only an improvement in retrieval performance but also a means for shortening the retrieval time. Sixth, the average retrieval time before and after applying the random forest was measured.

As shown in [Table 2], it was found that the search time was improved by about 0.26 seconds when the random forest classifier was used. Therefore, it can be seen that classifying database images by class and performing search on query images increases efficiency in search speed as well as search performance.

FIG. 12 shows a system interface and a search result screen to which a medical image search method based on image clustering is applied according to an embodiment of the present invention.

As described above, the present invention proposes a new OCS-LBP feature suitable for medical images and efficiently applies a BoF to effectively reduce the feature dimension in order to effectively retrieve medical images. In order to improve the search speed and performance, instead of matching the entire image with respect to the query image, a search method that displays the top K search results by the Bata-Karya distance measurement only for the K-proximity neighbor class separated by the random forest classifier .

Experimental results show that the Bartagarya distance measurement method is the best among the various similarity distance measurement algorithms. Likewise, when the proposed random forest classifier is combined with the Bartaglia distance measurement method, the performance is increased to about 28% . Also, as shown in the experimental results, the proposed features showed improved search results compared to the existing features, and the proposed system could effectively improve the search time as well as the search performance.

While the invention has been shown and described with respect to the specific embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined by the appended claims. Anyone with it will know easily.

Claims (7)

(a) dividing each input medical image into sub-regions and extracting an OCS-LBP (Oriented Center Symmetric Local Binary Patterns) feature value on a patch-by-patch basis;
(b) generating a code book by applying K average clustering to the feature values of the respective input images, generating a BoF (Bag-of Feature) histogram through the code book, Converting images to BoF features;
(c) generating the BoF feature as a feature vector, and learning and classifying the feature vector through a random forest learning classifier;
(d) repeating the steps (a) to (c) when the query image is input, and measuring similarity with the medical image through a small number of specific classes of the classified class; And
(e) outputting a search output image according to the measured degree of similarity.
The method according to claim 1,
The step (a)
Dividing each medical image into 2x2 sub-regions, and dividing each medical image into 4x4 patches for each sub-region;
Extracting the OCS-LBP pattern for each of the divided patches;
And generating a histogram for each of the sub regions to extract a feature value.
delete The method according to claim 1,
The step (c)
(d-1) setting the number T of decision trees to constitute a random forest;
(d-2) selecting m random variables from the feature vector set, and dividing the learning data using the determination function of each node and the random variable;
(d-3) growing the crystal tree to a maximum depth; And
(d-4) repeating the step (d-2) and subsequent steps if any one of the trees is completed and the number of completed trees is less than the T Search method.
The method according to claim 1,
The step (d)
Selecting a K-neighbor neighborhood (K-NN) class among classes of the query image by the random forest classifier;
And measuring the similarity distance only for the images labeled with the K classes of the input image.
6. The method of claim 5,
Wherein K is 2, and the image clustering-based medical image retrieval method.
6. The method of claim 5,
Wherein the similarity distance uses a Bhattacharyya distance.

Images