KR101093107B1 - Image information classification method and apparatus - Google Patents

Abstract

The present invention relates to an image information classification method and apparatus for classifying image information according to color features and boundary line features. The image information classification method according to the present invention comprises the steps of: receiving image information and detecting a first feature vector according to a color structure descriptor; Receiving the image information and detecting a second feature vector according to a boundary histogram descriptor; Generating a plurality of membership scores indicating how close the first and second feature vectors are to respective class classifiers corresponding to a plurality of classes, respectively, wherein the membership scores are classification information of the image information. Becomes

Image information, classification, color structure descriptor, boundary histogram descriptor

Description

Image information classification method and apparatus

The present invention relates to image information processing technology, and more particularly, to an image information classification method and apparatus for classifying image information according to color features and boundary line features.

Medical imaging plays a central role in patient diagnosis, medical therapy, surgical planning, medical certification and training, and as the medical images are digitized recently, researches on technology for classifying and retrieving medical images are increasing.

Reflecting these demands, PACS (Picture Archiving Communication System), a medical support system based on various technologies such as image processing, communication, and database [Ko, B., Seo, M. S. Nam. J-Y. "Frip: Microscopic Cell Nuclei Segmentation Based on Adaptive Attention Window." Journal of Digital Imaging, Published online, 1-16, June 2008.] and Digital Image and Communication in Medicine (DICOM), the standard network protocol for medical imaging, have been established.

However, the retrieval was very cumbersome and inefficient because a large number of medical images had to be given texts and classified according to their types.

Moreover, classifying medical images by people reflects the subjective point of view of the worker, which may cause problems, and has a disadvantage in that it takes time and cost. To solve these problems, various types of classification and retrieval methods have been proposed.

For example, Liu [Liu Y., Dellaert F. "Classification Driven Medical Image Retrieval" Proceeding of the Int. Workshop on Image Understanding 1-7, 1998] classified images using image index feature selection rules and proposed a classification search-oriented image retrieval framework using image semantic based similarity measurement.

Mojsilovic, Mojsilovc A. and Gomes J. "Semantic based categorization, browsing and retrieval in medical image databases." Proceedings of the Int. Conf. on Image Processing 3, 145-148, 2002.] proposed a new automatic classification method using various features of medical images. This method classifies an image using a visual feature set based on the meaning of features acquired from the image.

Greenspan [Greenspan H. "Medical Image Categorization and Retrieval for PACS Using the GMM-KL Framework." IEEE Transactions on Information Technology in Bio Medicine 11, 190-202, 2007.) describes the KL (Kullback-Leibler) distance and Gaussian mixed model. We propose a stochastic image representation method using (Gaussian Mixture Modeling).

The GMM-KL framework was used to categorize image measurements and X-ray images by body region. Bhattacharya, Bhattacharya P. and Rahman M. M. "Image Representation and Retrieval Using Support Vector Machine and Fuzzy C-means Clustering Based Semantical Spaces." Proceedings of the, Int. Conf. on Pattern Recognition 2, 1162-1168, 2006.] proposed a learning-based framework for medical image retrieval with a linear combination of teacher learning (probabilistic multiclass SVM) and fuzzy c-mean clustering. The similarity measure was obtained by comparing the query image with the membership score obtained from the learning algorithm.

Mueen [Mueen A. Zainuddin R. Baba M. S. "Automatic Multilevel Medical Image Annotation and Retrieval." Journal of Digital Imaging, Published online, 1-6, Sept. 2007.] proposed a method for inserting and retrieving multi-layered automatic medical image annotations using keywords based on concept or class hierarchy. In addition, multi-layer feature extraction is used for automatic feature extraction, which provides global and local level features with pixel intensity. Among them, X-ray image can be executed only by keyword (class name).

As described above, content-based image retrieval is a very active area in the field of image processing and computer vision. Among these, medical images can be judged differently according to the subjective viewpoint of the operator, and have a characteristic that is composed of a region of interest containing important contents and a meaningless monochrome background. Therefore, a classification and retrieval method different from general natural images is required. .

The present invention extracts feature vectors from image information by using a Harris corner detection-based color structure descriptor (CSD) and a boundary histogram descriptor (EHD) for texture features, and extracts the extracted feature vectors corresponding to each of a plurality of image types. It is an object of the present invention to provide a method and apparatus for classifying image information for generating a membership score for marking an image type by applying to a multi-class SVM, respectively.

It is another object of the present invention to provide a method and apparatus for classifying video information by combining membership scores detected from video information into one ensemble vector to improve the accuracy of the search.

It is still another object of the present invention to provide a method and apparatus for classifying video information by adding correlated categories vector (CCV) based similarity information to the ensemble vector so that a search for similar video information can be performed accurately and quickly.

According to an aspect of the present invention, there is provided a method of classifying image information, the method comprising: detecting first feature vectors according to color structure descriptors by receiving image information; Receiving the image information and detecting a second feature vector according to a boundary histogram descriptor; Generating a plurality of membership scores indicating how close the first and second feature vectors are to respective class classifiers corresponding to a plurality of classes, respectively, wherein the membership scores are classification information of the image information. Characterized by being.

The present invention described above has the effect of enabling effective classification and retrieval of medical image information, particularly X-ray image information.

In addition, the present invention has an effect that can cause automatic annotation generation based on the image classification to be performed in the future, improve the classification performance for similar categories.

A configuration of an image information classification apparatus according to a preferred embodiment of the present invention will be described with reference to FIG.

The image information classification apparatus includes a feature vector detector 100 according to a color structure descriptor, a feature vector detector 102 according to a boundary histogram descriptor, a class classifier 104 using an SVM, a CCV generator 106 and a storage medium ( 108).

The feature vector detector 100 according to the color structure descriptor receives image information, detects the feature vector according to the color structure descriptor, and provides the detected feature vector to the class classification unit 104 using the SVM.

The feature vector detection unit 102 according to the boundary histogram descriptor receives image information and detects the feature vector according to the boundary histogram descriptor and provides the image to the class classification unit 104 using the SVM.

The class classifier 104 using the SVM obtains a plurality of class classifiers corresponding to a plurality of image types (hereinafter, referred to as classes) through learning and detects feature vectors detected according to the color structure descriptor and the boundary histogram descriptor. Are applied to each of the plurality of class classifiers to generate a plurality of membership scores indicating how close the image information is to each class, and to a class classifier that has generated the largest membership score among the plurality of membership scores. The video information is classified into corresponding classes, and the plurality of membership scores are combined into an ensemble vector to increase the accuracy of the search.

The CCV generator 106 compares an ensemble vector detected from the image information with an ensemble vector of image information registered in a database in advance, and when the membership scores of positions corresponding to each other in the two ensemble vectors are equal to or greater than a predetermined value. With respect to the membership score, it is determined that the two pieces of image information have similarities, and according to the determination result, a CCV having recorded similarity determination information indicating a value of 0 or 1 is generated.

The storage medium 108 performs the process of the feature vector detector 100 according to the color structure descriptor, the feature vector detector 102 according to the boundary histogram descriptor, the class classifier 104 using the SVM, and the CCV generator 106. In addition to providing a storage area for the image information, and stores the image information and the ensemble vector and CCV corresponding to the image information.

An operation of the apparatus for classifying image information according to an exemplary embodiment of the present invention will be described in more detail with reference to the accompanying drawings.

<Feature Vector Detection Process According to Color Structure Descriptor>

The feature vector detection unit 100 according to the color structure descriptor receives image information and detects a feature vector according to a color structure descriptor (CSD) for the image information.

In general, X-ray image information has a characteristic of having a dark background and a bright foreground including important contents. Accordingly, the feature vector detector 100 according to the color structure descriptor quantizes the image information in 128 levels of contrast, and divides the image information into N × N subblocks according to the MPEG-7 standard. Herein, the size of the subblock may be set to 8 × 8.

As shown in FIG. 2, the X-ray image information includes a useless background area. Accordingly, the feature vector detector 100 according to the color structure descriptor removes a background area having a contrast value less than or equal to a predetermined value.

Then, the feature vector detector 100 according to the color structure descriptor is Harris C. and Stephens MJ “A combined corner and edge detector.” Proceeding of the remaining subblocks except the subblocks according to the background region. Harris corner points are detected according to the Alvey Vision Conference, 147-152, 1998.), and only sub blocks including one or more Harris corner points are selected.

Thereafter, the feature vector detector 100 according to the color structure descriptor extracts 128 bin color structure histograms for the selected subblocks, resulting in 128 quantized black and white color spaces. Thus, the color structure histogram is a one-dimensional array of values quantized in m bits.

M in Equation 1 means one of the sets {256, 128, 64, 32}, and s is the scale of the combined structural element (sub block). In each subblock, the color structure histogram is accumulated on the principle of expressing the color in the subblock.

For example, eight intensity levels and eight color structure histograms are made of 8x8 subblocks, and the number is increased by one if a large number of appropriate bins are identical to the subblock position in a pre-separated color structure histogram that matches the image information. . Accordingly, the feature vectors of the image information coincide with the distribution of a large number of subblocks and each recorded color structure histogram.

The extracted 128 color structure descriptor bin histogram values are normalized to a range of 0 to 1 for SVM learning. Here, the color structure descriptor bin histogram value is a feature vector according to the color structure descriptor.

<Feature Vector Detection Process According to Boundary Histogram Descriptor>

The feature vector detector 102 according to the edge histogram descriptor is used to detect a boundary feature histogram descriptor (EHD) (Won CS and Park DK "Efficient Use of MPEG-7 Edge Histogram Descriptor", ETRI Journal 24, 23 -30, 2002.) detects a border histogram descriptor from the input image information.

According to the boundary histogram descriptor, the image information is divided into non-overlapping sub-images, and each sub-image is divided into non-overlapping image blocks as shown in FIG. The size of the image block is determined through Equation 2.

,

,

 In Equation 2, the desired num block is the total number of image blocks in the image information. In the present invention, the default value of the desired Num block was determined to be 1100 through experiments.

FIG. 2B illustrates five types of boundary filters applied to sub-images (vertical, horizontal, 45 ° diagonal, 135 ° diagonal, and no direction) to generate a border histogram. Using the feature values extracted using the five types of filters, the feature vector detector 102 according to the boundary line histogram descriptor detects a maximum boundary line value.

In general, in X-ray image information, which is a kind of medical image information, texture features, especially boundary lines, may be an important feature component. Therefore, the feature vector detector 102 according to the boundary histogram descriptor of the present invention additionally extracts the global boundary histogram and the semi global histogram as a feature value as well as the local histogram value. Here, the feature value is a feature vector according to the boundary histogram descriptor.

Class classification using SVM

The feature vectors according to the color structure descriptor and the border histogram descriptor are classified into predefined classes by the class classification unit 104 using the SVM.

는 공간

차원에서 존재하고, 출력값

는

로 표현되어 진다면, 수학식 3은 일반적인 이진 선형 분류 함수를 나타낸다.For example, given training data in advance

Space

Present in the dimension, output

Is

Equation 3 represents a general binary linear classification function.

,

인 두 개의 클래스가 있다고 할 때,

의 w와 b값들에 대해 구해진 최적의 값이 초평면이 된다. 상기 SVM의 주된 목표는 최적의 평면을 결정하는 매개변수 w와 b를 찾는 것이다. 이 값들은 아래의 제약식인 수학식 4와 목적식인 수학식 5를 통해 계산되어진다.In Equation 3, x denotes an input vector, w denotes a weight vector, and b denotes a bias. At this time,

,

Suppose you have two classes that are

The optimal value obtained for the w and b values of is the hyperplane. The main goal of the SVM is to find the parameters w and b that determine the optimal plane. These values are calculated by the following equation (4) and the target equation (5).

는 라그랑제 승수를 나타 낸다. In Equations 4 and 5, N is the total number of images,

Denotes the Lagrange multiplier.

The determined values are applied to Equation 6, which is an SVM classification function, to enable classification of input images.

는 커널함수 값,

는 각 커널 출력들의 가중치를 각각 나타내며

은 +1 클래스와 -1 클래스처럼 x의 클래스 멤버십 스코어를 결정한다. 초평면에서 분류 함수 f(x)는 영상이 속하는 하나의 클래스(+1)가 다른 클래스(-1)로부터 얼마나 다른지를 측정하는 서포트 벡터로 결정이 된다. In Equation 6, X represents an input image,

Is the kernel function value,

Denotes the weight of each kernel output, respectively

Determines the class membership score of x, like the +1 and -1 classes. In hyperplane, the classification function f (x) is determined as a support vector that measures how different one class (+1) to which an image belongs from another class (-1).

Since the X-ray images used in the present invention are classified into any 20 categories for each part of the body, the multi-class SVM is used, and the RBF (Radial Basis Function) is used as the kernel function.

The multi-class SVM uses Equation 7 below, and denotes values of the input image X processed in each class. Where the largest value is selected as the final X classification class.

In the SVM learning, as shown in FIG. 4, 100 images are trained for the remaining categories except for the small number of toes, thighs, and elbows, thereby generating a total of 1,754 training data for 20 categories.

In the present invention, since two feature vectors H-CSD and EHD are applied to the multi-class SVMs, 2n SVM classifiers are generated.

와

를 출력한다.FIG. 5 shows feature vectors F _c and F _E (F _c : H-CSD feature vector and F _E : EHD feature vector) generated from input image information, respectively, and are provided to 20 SVM classifiers. Each of them has 20 membership scores for each feature vector.

Wow

.

)로 합쳐진다. 여기서, 상기 앙상블 벡터의 각 값은 입력 영상이 20개의 클래스에 대해 어느 정도 근접한지를 나타내는 분류정보이다. The printed membership scores are not categorized into one category, and one ensemble vector (

Are combined. Here, each value of the ensemble vector is classification information indicating how close the input image is to 20 classes.

The class classification unit 104 using the SVM stores the ensemble vector in the storage medium 108 corresponding to the input image information.

<CCV generation process>

Classes of X-ray image information used in the experiment for the present invention can be classified as independent or related to each other. For example, class 1 (the chest) is generally independent of class 11 (the fingers). However, class 10 (hand) and class 11 (finger) are related classes [Yao J., Zhang Z., Antani S., Long R., and Thoma G. "Automatic medical image annotation and retrieval", Neurocomputing 71, 2012-2022, 2008.]. As such, the associated features between the classes also greatly affect the similarity features.

Accordingly, the present invention generates a CCV (Correlated Categories Vector) using the class association between the ensemble vectors.

As shown in FIG. 6, the CCV generator 106 extracts an ensemble vector for the query image information and indicates '1' or '0' indicating a similarity determination result for the i-th dimension in the CCV satisfying Equation 8. Record it.

는 앙상블 벡터의 i번째 차원의 멤버십 스코어이다. 여기서, θ는 0.5의 값으로 설정될 수 있다. Θ in Equation 8 is a determination value for determining the relevant category,

Is the membership score of the i-th dimension of the ensemble vector. Here, θ may be set to a value of 0.5.

After the above CCV generation, the similarity measure is performed only with the values of the i th ensemble vector when the corresponding i th dimension CCV code is '1'.

Using the ensemble vector and the CCV, the final distance is measured by Equation 9 below, and the most similar images are shown in ascending order of the final distance.

는 i번째 질의 앙상블 벡터의 멤버십 스코어이고,

는 i번째 목표(데이터베이스) 앙상블 벡터의 멤버십 스코어이다. M은 CCV에 포함된 ‘1’들의 수이고,

와

는 각각 H-CSD와 EHD를 위한 멤버십 스코어의 가중치이다.

와

의 값 결정을 위하여, 모든 카테고리에서 임의로 다섯개의 영상 선택을 기반으로 다른 값들을 이용하여 정확한 평균이 계산된다. In Equation 9

Is the membership score of the i th ensemble vector,

Is the membership score of the i th target (database) ensemble vector. M is the number of '1's included in the CCV,

Wow

Is the weight of the membership score for H-CSD and EHD, respectively.

Wow

To determine the value of, an accurate average is calculated using different values based on a randomly selected five image selection in all categories.

와

는 각각 0.3과 0.7을 주었을 때, 정확(precision) 정도는 상대적으로 높다. 또한 결과에서 EHD는 의료 영상 분류를 위하여 중요한 특징임을 확인하였다. 따라서 본 발명에서는

와

를 각각 0.3과 0.7로 설정하였다.Through experiment as shown in Figure 7

Wow

Given 0.3 and 0.7, respectively, the precision is relatively high. The results also confirm that EHD is an important feature for medical image classification. Therefore, in the present invention

Wow

Was set to 0.3 and 0.7, respectively.

Here, for experiments, IRMA (Image Retrieval in Medical Application) [Vapnik V., "The Nature of Statistical Learning Theory." Springer-Verlag, 1999., Deselaers T. "The CLEF 2005 Automatic Medical Image Annotation Task", International Journal of Computer Vision 74, 55-58, 2007.] used 1000 images (20 classes).

8 shows the name of each class and the number of images used in the experiment.

9 is a main screen of an interface of the MISS. As a comparative experiment for verifying the performance of the method proposed in the present invention (ensemble vector + CCV), a method using only an ensemble vector without CCV and an addition method are used. Addition method is a method that adds the membership score value through SVM of H-CSD and the membership score value of EHD, respectively.

1 is a block diagram of a medical image classification apparatus according to a preferred embodiment of the present invention.

2 is a diagram illustrating a feature vector detection process according to a color structure descriptor according to a preferred embodiment of the present invention.

FIG. 3 illustrates five boundary filters for definition and boundary extraction of sub-images and image blocks according to an exemplary embodiment of the present invention. FIG.

4 is a view showing the name of each class and the number of learned images according to the present invention.

5 is a diagram illustrating a learning process and an ensemble vector generation process for database images according to the present invention.

6 illustrates a similarity measurement process according to the present invention.

Figure 7 is a comparison of the performance between different weights in accordance with the present invention.

8 is a view showing the name of each class and the number of experimented images according to the present invention.

9 is a diagram illustrating a main screen of the MISS interface, which is a medical image retrieval system according to the present invention;

Claims (14)

In the video information classification method, Receiving first image information and detecting a first feature vector according to a color structure descriptor; Receiving the image information and detecting a second feature vector according to a boundary histogram descriptor; Generating a plurality of membership scores indicating how close the first and second feature vectors are to respective class classifiers respectively corresponding to the plurality of classes; Combining the plurality of membership scores to generate an ensemble vector; Storing the ensemble vector corresponding to the input image information; Generating a CCV (Correlated Categories Vector) indicating a similarity determination result for each of the plurality of membership scores composed of the ensemble vectors and each of the plurality of membership scores included in an ensemble vector of image information corresponding to a pre-stored related category, Calculating a final distance indicating similarity to a related category using the CCV and the ensemble vector; And the plurality of membership scores are classification information of the image information. delete delete The method of claim 1, The CCV is calculated according to Equation 10, The final distance is calculated according to Equation (11).

In Equation 10, θ is a determination value for determining a related category,

Is the membership score of the i-th dimension of the ensemble vector.

In Equation 11

Is the membership score of the ensemble vector according to the i-th input image information,

Is the membership score of the ensemble vector of the image information corresponding to the i-th related category, M is the number of '1's included in the CCV,

Wow

Are the weights of each membership score. The method of claim 1, The first feature vector divides an input image into a plurality of blocks, removes a block corresponding to a background of each block, and detects and generates a color structure descriptor for the remaining blocks. The method of claim 1, And the second feature vector divides the input image into a plurality of blocks and detects and generates a border histogram descriptor for each block. The method of claim 6, And the second feature vector is generated by detecting boundary histogram descriptors globally or inversely with respect to the input image information. In the video information classification apparatus, A feature vector detection unit according to a color structure descriptor that receives image information and detects a first feature vector according to the color structure descriptor; A feature vector detection unit according to an edge line histogram descriptor that receives the image information and detects a second feature vector according to an edge line histogram descriptor; Generating a plurality of membership scores indicating how close the first and second feature vectors are to respective class classifiers respectively corresponding to the plurality of classes, combining the plurality of membership scores to generate an ensemble vector, and A class classifier configured to store an ensemble vector in a storage medium corresponding to the input image information; And Generating a CCV (Correlated Categories Vector) indicating a similarity determination result for each of the plurality of membership scores composed of the ensemble vectors and each of the plurality of membership scores included in an ensemble vector of image information corresponding to a pre-stored related category, And a similarity information generation unit configured to calculate a final distance indicating similarity with respect to a related category using the CCV and the ensemble vector. And the plurality of membership scores are classification information of the image information. delete delete The method of claim 8, The CCV is calculated according to Equation 12, And the final distance is calculated according to equation (13).

Θ in Equation 12 is a determination value for determining the relevant category,

Is the membership score of the i-th dimension of the ensemble vector.

In Equation 13

Is the membership score of the ensemble vector according to the i-th input image information,

Is the membership score of the ensemble vector of the image information corresponding to the i-th related category, M is the number of '1's included in the CCV,

Wow

Are the weights of each membership score. The method of claim 8, The first feature vector divides an input image into a plurality of blocks, removes a block corresponding to a background among the blocks, and detects and generates a color structure descriptor for the remaining blocks. The method of claim 8, And the second feature vector divides the input image into a plurality of blocks and detects and generates a border histogram descriptor for each block. The method of claim 13, And the second feature vector detects and generates a border histogram descriptor globally or inversely with respect to the input image information.

Images