KR100626838B1 - High speed image compare, retrieval system based on near maximum flow matching and method thereof - Google Patents

Abstract

The image retrieval system and method disclosed herein divides a character image object into n sectors from a midpoint, and extracts a threshold value using a statistical method when extracting and comparing feature values by quantizing a color histogram of the divided image. . The optimum similarity is measured using the extracted predetermined threshold value. As a result, accurate and fast matching can be performed.

Description

HIGH SPEED IMAGE COMPARE, RETRIEVAL SYSTEM BASED ON NEAR MAXIMUM FLOW MATCHING AND METHOD THEREOF}

1 is an overall block diagram of an image retrieval system according to a preferred embodiment of the present invention;

FIG. 2 is a diagram for describing a quantization process of an image segmentation result obtained by the image segmenter illustrated in FIG. 1 and a histogram of each segmented region; FIG.

3 shows an example of a color histogram of a character image;

4 is a view for explaining a method of calculating a minimum cost in a weighted Bipartite Graph;

5 is a view for explaining a configuration process of the WBM;

FIG. 6 is a diagram for explaining a Hungarian method for finding an optimal solution using addition and subtraction at a minimum value of a matrix; FIG.

의 각 정점이 0, 1, 2, 3으로 설정되고, 우측의

정점이 A, B, C, D로 설정되었을 때, 헝가리안 트리로 모든 연결 가능한 경우를 보여주는 도면;7 shows the left side of the weighted Bipartite Graph.

Each vertex of is set to 0, 1, 2, 3, and

A diagram showing all possible connections to a Hungarian tree when the vertices are set to A, B, C, D;

8 (a) to 8 (c) show examples of the use of a Hungarian tree using breadth-first search;

9 (a) to 9 (c) show examples of the use of a Hungarian tree using depth first search;

10 is a distribution curve of the same sector image;

11 is a flowchart showing an NMFM matching method according to a preferred embodiment of the present invention, performed in the multiple feature matching section shown in FIG. 1;

12 shows an example of registration of an NMFM;

FIG. 13 is a view for explaining a method of using NMFM for similarity alignment in the multiple feature matching unit shown in FIG. 1; FIG.

14-16 are graphs of performance evaluation for the present invention;

17 is a performance evaluation graph according to the number of quantization levels of GHM, WBM, and NMFM; And

18 is a graph for evaluating search speed performance of WBM and NMFM for an image search system according to the present invention.

* Description of the symbols for the main parts of the drawings *

100: image retrieval system 110: preprocessing unit

120: image segmentation unit 130: quantization unit

140: feature extraction unit 170: multiple feature matching unit

190: database

The present invention relates to a data retrieval system and method, and more particularly, to a character image retrieval system and method using multiple feature information.

As the basis of digital data is expanded from text to multimedia information, the importance of animation data and video data is increasing. Character use is increasing day by day in creating animation data or in producing animation content and related knowledge-based content. In the production of new characters, the necessity of searching for the current status of existing characters is increasing. To date, many characters other than famous characters have not been made into a database, which has caused problems in information sharing and digital content creation. In addition, when a new character is produced, a search system is not constructed, and thus a similar character is produced and a intellectual license dispute arises. Recently, the virtual expression of oneself using an avatar in the internet service is increasing. In addition, many digital services industries, such as education and games, have begun using character video. Many digital services already recreate and use characters of the same or similar form because of the lack of a system for searching and supplying suitable character images. Therefore, if the character information is made into a database and a system is searched for the character, the digital content can be reduced and the advanced digital information can be constructed.

However, since the character image has various characteristics of changing its shape and color, it is very difficult to search by using the form information or one color information, and even if the search is performed by using the color and form information together, the search performance is high. There is a falling problem. In addition, the character image has a problem that it is difficult to distinguish it by the normal image search method because the image and the color histogram having the same color even if the character image is different, there is no pattern and has a monochrome form unlike the general natural image There is a problem that the search method using the pattern is difficult.

SUMMARY OF THE INVENTION The present invention provides an image retrieval system and method for efficiently retrieving a character image that is difficult to retrieve using one type of form information or color information using multi-feature information including at least two feature information. It is.

An object of the present invention is to provide an image retrieval system and method that can perform accurate and fast matching.

In order to achieve the above object, the image retrieval system according to the present invention includes a database in which a plurality of data to be used for retrieval are stored; A preprocessing unit dividing an object to be searched into a plurality of areas having multiple feature information, and obtaining feature values of the divided areas; And determining a threshold value by a statistical method, and comparing the similarity of the data within the range of the threshold value with feature values of the divided region to determine an optimal similarity level for the search target. Characterized in that it comprises a matching portion.

In a preferred embodiment, the multi-feature matching unit no longer performs matching or weighted bipartite matching (WBM) matching when there is no data belonging to the threshold range among the data stored in the database. It is characterized by searching for similar images.

In a preferred embodiment, the multi-feature matching unit searches for a similar image by performing Near Maximum Flow Matching (NMFM) matching when there is data belonging to the range of the threshold value among the data stored in the database. .

In order to achieve the above object, the image retrieval method according to the present invention comprises the steps of: (a) setting all connections that can be connected in the Bipartite Graph; (b) extracting the pair with the lowest cost; (c) calculating a difference between the weight of the pair for obtaining the minimum cost and a predetermined threshold obtained statistically; (d) performing weighted bipartite matching (WBM) matching to obtain an optimal flow and matched pair when the calculated weights of the entire pair are outside the range of the threshold value; And (e) obtaining an approximate maximum flow by performing Near Maximum Flow Matching (NMFM) matching on the pair belonging to the threshold range when the calculated weight of the entire pair does not deviate from the range of the threshold value. It is characterized by including.

In a preferred embodiment, the image retrieval method comprises: pre-processing a target to be searched before the step (a) into a plurality of areas having multiple feature information and obtaining feature values for the divided areas. It further comprises a step.

(Example)

The novel image retrieval system and method of the present invention divides a character image object into n sectors from a midpoint, and quantizes a color histogram of the divided image to extract and compare feature values, thereby using a statistical method to determine threshold values. Extract. The optimum similarity is measured using the extracted predetermined threshold value. As a result, accurate and fast matching can be performed.

In particular, the image retrieval system according to the present invention has been described for finding the same character by using the multi-characteristic information of the character image, but this is only an example of a retrieval system using the multi-feature information, a large amount of information is stored The present invention can be applied to any retrieval system for retrieving specific information from an established database. For example, if you want to search for students who meet certain criteria (especially multiple features that match multiple criteria) in a database where scores for multiple students are stored, applying the present invention can provide more accurate search in a shorter time. You can get the result.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is an overall block diagram of an image retrieval system 100 according to an exemplary embodiment of the present invention, and FIG. 2 is an image segmentation result obtained by the image divider 120 shown in FIG. 1 and a histogram of each divided region. A diagram for describing a quantization process of the quantization unit 130 with respect to FIG.

Referring to FIG. 1, the image retrieval system 100 according to the present invention is largely composed of a preprocessor 110, a multi-feature matching unit 170, and a database 190. Image search (or data search) using multiple feature information combined with at least two kinds of information is performed. Here, the database 190 may be provided on the local area together with the image retrieval system 100 or may be provided at a remote location and connected online. In addition, the database 190 may be provided by itself in the image retrieval system 100, or any database provided by another organization or system may be applied.

In general, when multiple feature information is used for retrieving image data such as a character, there is a problem that it is difficult to implement and computation time is long as compared with retrieval of a single piece of information consisting only of shape or color information. However, since the image retrieval system 100 according to the present invention uses a NMFM (Near Maximum Flow Matching) matching method for retrieval using multi-characteristic information, the image retrieval system 100 has a performance similar to that of a conventional single information retrieval method and at a very high speed. The match can be calculated. Looking at each configuration of the present invention as follows.

First, the configuration and operation of the preprocessor 110 included in the image retrieval system 100 will be described. The preprocessor 110 may include an image divider 120, a quantizer 130, and a feature extractor 140. It is composed of

1 and 2, the image divider 120 divides an object image into a plurality of regions and calculates a color histogram of the divided image. In order to compare the images while adapting the shape change of the character image with the color histogram, the comparison is most optimal when the weights are different for matching the partial image with the large change and the partial image with the small change. The division of the object image into n sector images through the image divider 120 is to view a maximum flow chart by separating an area that is not changed from an area of the image and an area that is not changed. In addition, segmenting an image into a plurality of regions (ie, sectors) to view local image information is to consider shape information of each segmented image. Therefore, when the object image is divided into a plurality of sectors, plural pieces of information having both shape information and color information are formed.

The quantization unit 130 quantizes the color histogram obtained from the n divided images acquired by the image divider 120 using a parzen window in the form of a triangle. If the histogram information of the sector image obtained by the image dividing unit 120 is used for matching, the data amount is large because it becomes 3 bytes of information per pixel. For example, when the RGB level is 256, a total of 768 bins are required in one sector image, and when a six-sector image is divided, a total of 4608 bins are required. Thus, quantization is needed to reduce the information. Since the character image is a form in which the histogram shows a peak in a specific region, the character image has a feature of not losing much information even when quantized. However, if a peak appears in a specific region, an error occurs if the region is a boundary of quantization. In order to secure such a problem, in the present invention, when the quantization is performed, the Parzen windows are overlapped and divided into triangles. This reduces the error that occurs at the boundary of quantization. Looking at the configuration and function of the quantization unit 130 in more detail as follows.

3 is a diagram illustrating an example of a color histogram of a character image. 2 and 3, in the character image, a characteristic value often appears as several peaks on one side. As shown in FIG. 2 and FIG. 3, the color information of the image is not uniformly distributed throughout, but is concentrated in a specific area.

For example, if the total 256-level color distribution is divided into n bins, the values at the boundaries of one bin will change significantly. If the color histogram is concentrated in the center of the boundary area, the error is high because the difference between one bin and the other bin is greatly contributed when the number of bins is reduced. To solve this problem, the Parzen window is used. The basic formula for the Parzen window is:

은 양자화 영역의 길이,

는 현재 영역에서 떨어진 크기, varphi 는 양자화 윈도우, 그리고,

은

의 부피(volume) 함수를 각각 나타낸다. [수학식1]은

윈도우를 사용하여 영역에 있는 값을 통합하여 특정 값으로 변환할 때 사용한다. 이때 어떤 함수를 사용하는가에 따라서 그 영역의 값은 다양하게 나타난다. 오차를 최소화하기 위해서 가장 이상적인 방법은 가우시안(Gaussian) 윈도우를 사용하는 것이다. 그러나 가우시안 윈도우를 이용하게 되면 연산량이 늘어나는 단점이 있다. 우리가 목적으로 하는 것은 캐릭터 영상에서 특징 값들이 몇 개의 피크로 나타나고, 그 피크가 경계선에 있을 때 오차를 최소화하는 것이다. 따라서, 연상량은 줄이고 오차를 최소화하기 위해서 본 발명에서는 가우시안 형태 대신 삼각형의 Parzen 윈도우를 사용하였으며, 이를 수학식으로 표시하면 다음과 같다.In [Equation 1],

Is the length of the quantization region,

Is the size away from the current region, varphi is the quantization window, and

silver

Represent each volume function of. Equation 1 is

It is used when window is used to integrate the values in the area and convert them to specific values. The value of the range varies depending on which function you use. The ideal way to minimize errors is to use a Gaussian window. However, using a Gaussian window has the disadvantage of increasing the amount of computation. Our goal is to minimize the error when the feature values appear in a few peaks in the character image and those peaks are at the boundary. Therefore, in order to reduce the association amount and minimize the error, the present invention uses a triangular Parzen window instead of a Gaussian form, which is expressed as follows.

은 영상의 원래 히스토그램 레벨이고,

는 양자화할 히스토그램 레벨이다. 그리고,

은 히스토그램의 값들이고,

는 결과로 반영되는 히스토그램의 값이다. Q는 양자화 레벨 값인데, 이 값은 칼라의 가중치 값을 이용하여 계산된다. 캐릭터 영상의 값들은 몇 개의 부분에 밀집해서 있는 형태이기 때문에 [수학식2]를 이용해도 가우시안 윈도우를 사용한 경우와 크게 차이가 나지 않게 된다. 이와 같이 양자화부(130)에 의해 각각의 분할 영역에 대한 색상 히스토그램이 양자화되고 나면, 특징 추출부(140)를 통해 양자화된 각각의 영역에 대한 특징 값이 추출된다.In Equation 2,

Is the original histogram level of the image,

Is the histogram level to quantize. And,

Is the value of the histogram,

Is the value of the resulting histogram. Q is a quantization level value, which is calculated using the weight value of the color. Since the values of the character images are concentrated in a few parts, the equations 2 do not differ significantly from the Gaussian window. As described above, after the color histogram of each divided region is quantized by the quantization unit 130, the feature value of each quantized region is extracted by the feature extractor 140.

Referring back to FIG. 1, after feature values for multiple information including shape information and color information are extracted by the preprocessing unit 110, the multiple feature matching unit 170 may perform weighted bipartite matching (WBM) or NMFM ( Near Maximum Flow Matching) is performed to compare the similarity with respect to the color and shape information of the image and obtain an optimal similarity.

Matching performed in the multi-feature matching unit 170 is a matching for a pluralized feature combining color and morphological features. The use of multiple feature information when comparing optimized feature values has the advantage of higher accuracy than single feature information and easy retrieval even when there is a change in a single feature of an image. However, the multi-feature information has a variety of attributes, which is more difficult to compare than the single feature comparison. Generally, a matching method combined with a search method may be used to compare the multi-feature information. However, since the matching of an image using the multi-feature information can be regarded as a comparison matching of binary information, in the present invention, the Bipartite Graph Comparing multiple feature information using the matching of.

In order to calculate similarity by comparing two pairs of multiple feature information using a bipartite graph, a max flow matching method may be used. When the comparison is performed by assigning different weights according to the attributes of the feature information, the maximum flow matching of the bipartite graph may be regarded as a weighted bipartite graph problem, that is, a weighted bipartite matching (WBM) problem.

If WBM is used for registration, it has the advantage of being able to compare the image by adapting to the change even if the shape change occurs. However, since WBM is of high complexity, it requires a large amount of computation and time in comparing large databases and real-time comparison. Therefore, the present invention proposes an NMFM that infers the approximate maximum flow value in order to overcome such disadvantages of WBM. This method calculates the approximate maximum flow by eliminating the areas where error is likely to occur and by inferring the flow value of the uppermost node pair where maximum matching can occur. It has a fast advantage, making it easy to use in a system that searches for images in real time.

First, a WBM matching method for performing maximum flow matching is as follows.

FIG. 4 is a diagram illustrating a method of calculating a minimum cost in a weighted bipartite graph, and a similarity comparison graph in which a minimum required cost is generated is illustrated.

의 각 정점이 우측의

정점과 정합을 하는 문제가 발생되는데, 이와 같은 정합의 문제에서 두 그래프의 유사도를 구하는 부분을 최대흐름문제(Maximum flow problem)로 변환 할 수 있다.

의 각 정점을 출발지라 하고

의 정점들을 도착지라 하면,

과

사이의 연결선은 두 정점을 잇는 경로로 간주할 수 있고, 출발지로부터 도착지로 화물을 수송했을 때 각 경로의 용량의 합을 최대화하는 문제를 최대 흐름 문제라 할 수 있다.Referring to FIG. 4, a weighted graph for each vertex connected to each vertex in the bipartite graph is referred to as a weighted bipartite graph. Weighted Bipartite Graph

Each vertex on the right

The problem of matching with vertices occurs, and the similarity of two graphs can be converted into the maximum flow problem.

Each vertex of is called the origin

The vertices of

and

The connecting line between the two peaks can be regarded as a path connecting two peaks, and the problem of maximizing the sum of the capacity of each path when the cargo is transported from the starting point to the destination is the maximum flow problem.

Weight When the value of the weight in the Bipartite Graph reaches the maximum, matching by the condition of the maximum flow is called matching, and matching by the condition of the maximum flow is called WBM. The maximum flow match is represented by the following equation.

이고, 각 정점에 연결된 에지의 가중치 합은

로 나타낸다. 그리고, [수학식3]의 G=(V, E)는 완전그래프(Complete Graph)에서 임의의 온전그래프(Perfect Graph)를 의미하며, 온전그래프의 가중치 합이 최대가 될 때, 그 때의 그래프가 최대 흐름 정합을 가지는 그래프가 된다. 이 같은 온전그래프를 찾아 정합을 하는 것이 바로 WBM 정합이다. WBM 정합을 이용하지 않고 순차적으로 연산을 수행한다면, 모든 온전 그래프의

의 값을 구하여야 하며, 이 경우 n!번의

을 구하기 때문에, 매우 큰 복잡도를 가지게 된다. 따라서, 모든 가능한 온전그래프의 가중치 합을 구하지 않고 최소의 연산으로 최대 흐름도를 찾는 알고리즘인 WBM을 이용한다.In Equation 3, one vertex matched in the graph G = (V, E) is E ₀ , and the weight value of E ₀ is

The sum of the weights of the edges connected to each vertex is

Represented by In addition, G = (V, E) in [Equation 3] means an arbitrary perfect graph in the complete graph, and when the sum of the weights of the whole graph becomes the maximum, the graph at that time Becomes the graph with the largest flow match. It is WBM registration that finds and matches such an intact graph. If you perform the operations sequentially without using WBM matching, all of the whole graph

Must be obtained, in which case n!

Since we find, we have a very large complexity. Therefore, we use WBM, which is an algorithm that finds the maximum flow chart with minimum computations without finding the sum of the weights of all possible intact graphs.

In WBM matching, the Hungarian method is used to obtain the minimum cost, which is used to find the matching. The basic concept of the Hungarian method starts from the premise that subtracting the minimum value from each edge does not affect the optimization flow. The Hungarian method uses the minimum cost value to get the maximum flow value.

The minimum required cost means the cost of moving from one vertex to another. In the G = (V, E) graph, when the maximum flow of any edge (a, b) is W _max and the weight of _{(a, b)} is W _{(a, b)} , the minimum flow cost W _min is Equation 4]

W _min = W _max -W _{(a, b)}

In Equation 4, finding the minimum required cost using W _min is equivalent to finding the matched pair of maximum flow. This method of finding the minimum cost is often used for similarity comparison.

과

의 각 정점이 고유의 값을 가지고 있고,

과

가 같은 속성을 가진 그룹이라고 할 때, 두 그룹의 유사도가 얼마나 비슷한지는 최소 소요 비용을 이용하여 결정된다. In WBM matching, you can get W _min from the similarity comparison even if you do not have W _max . For example, at G = (V, E)

and

Each vertex in has a unique value,

and

When is a group with the same property, how similarity between the two groups is determined by using minimum cost.

In FIG. 4, when one intact Bipartite Graph is set, a minimum flow is illustrated using a difference in inherent values of each vertex. For example, if one vertex has a value of seven and a comparison vertex has a value of five, then the two similarities will be two, which represents the minimum flow cost.

After getting all the values of all possible intact graphs and selecting the intact graph that constitutes the maximum flow chart, many operations are performed. However, assuming that a few vertices are connected and all the remaining vertices are connected to their maximum values, we can predict the maximum potential of the intact graph. If the predicted value is smaller than the predicted value of other ontographic matching pairs, the intact graph cannot have any further operation because it cannot have the maximum flow value. Using this method, the WBM algorithm finds the matching pair that is the maximum flow chart.

FIG. 5 is a view for explaining a configuration process of the WBM, and examples of four cases generated when the WBM is shown, respectively. FIG. 5A illustrates a case where a connectable path {(2, a), (3, c),} is set in a state where one node (1, d) is set, and (b) two nodes. The case where only {(1, d), (2, a)} is connected is shown. And, in (c), assuming that one node (1, c) is fully connected and one node (2, a) is connected, other connectable paths are shown, and (d) shows an intact mating state. Each is shown.

If one node is completely connected, (n-1)! Of whole graphs as shown in (a) of FIG. 5 are generated, and each cost value is predicted. In this case, when vertex 2 of group X is set to a, when the smallest minimum cost is obtained, (1, d) and (2, a) can be regarded as connected. In this case, the remaining vertices may be connected in various forms, and FIG. 5C shows a pair that can be connected after two vertices are connected. In a potential intact graph, there may be several forms such as (a), of which the elimination of the inexpensive intact graph is skipped to step (b), and in step (b) In step (c), we can create a matched pair with the least cost as we create the node by removing the intact graph, which cannot be costed in the same way.

Equation 5 below calculates the total flow value when an arbitrary pair of edges is set in the WBM.

는 새롭게 최대 정합의 후보로 설정된 에지의 흐름값을 의미하며,

는 정합되어 있지 않는 나머지 노드들 중에서 연결 가능한 최소값을 의미한다. [수학식5]에서 알 수 있듯이 WBM은 흐름값이 최소가 되는 모든 에지 쌍을 후보로 선정하면서 반복적으로 노드를 증가하면서 E 값을 구하고, 노드가 증가되기 이전의 E 값이 현재의 E 값보다 흐름 값이 크게 되면 상위의 정합 쌍은 연결 불가능 경로가 된다.In Equation 5, E denotes the total flow value of the currently set edge pair, and Exy becomes the flow value of the edge currently set to the maximum match.

Means the flow value of the edge newly set as the candidate of maximum matching,

Is the minimum connectable value among the remaining nodes that are not matched. As can be seen from Equation 5, WBM selects all edge pairs whose flow value is the minimum and obtains the E value by repeatedly increasing the node, and the E value before the node is increased is greater than the current E value. If the flow value becomes large, the upper matched pair becomes an unconnectable path.

A representative method for performing WBM on a Bipartite Graph is the Hungarian Method, which was devised by Konig. This method uses a Hungarian tree to apply the algorithm to the stems with the largest flow of nutrients in the roots, while the branches that carry the smallest nutrients no longer grow. This method adds or subtracts the minimum value by writing a matrix, and there is a method using a tree. The idea of using matrices is basically that even if you add or subtract the same value in each row or column, the optimal solution does not change. Thus, by subtracting or adding the appropriate values in each row or column, all weight values are greater than or equal to zero and replaced with many zeros. If we can only assign this to a weight of zero, this is the optimal solution.

FIG. 6 is a diagram illustrating a Hungarian method for obtaining an optimal solution using addition and subtraction at a minimum value of a matrix. Referring to FIG. 6, FIG. 6A illustrates an example of setting weights when respective nodes are connected. In (a), for each row, the minimum value among the costs in the row is selected to set a value subtracted from the cost of the row. If you select the minimum value of each row in (a) and subtract the value of the remaining rows by that value, it becomes the form of (b). In (b), for each column, the minimum value of the costs in the column is obtained, and the operation of setting the subtracted cost of each column is performed. If you select the minimum value of each column in (b) and subtract the value of the remaining columns with the value, it becomes the form of (c).

After performing steps (a) and (b), you get several zeros. At this time, it is determined whether the number of straight lines is the minimum among the methods of connecting rows and columns in a straight line so that all the values of 0 can be deleted. If the determined number of straight lines is less than the number of rows, the next step is performed. If the number of straight lines is the same as the number of rows, an optimal solution is created. For example, if you draw a line to erase all zeros in (c), this would be 1 row and 4 columns. At this time, since the drawn line is 2 and smaller than 4, which is the number of all lines, the next step is performed. That is, in (c), if the number of straight lines is smaller than the number of rows, first obtain the minimum cost among the costs that cannot be erased by the straight line, subtract this cost from the cost not erased by the straight line, and add it to the area crossed twice by the straight line. Will run again.

In FIG. 6C, the minimum value is 1 in 3 rows and 1 column among the portions remaining without drawing a line. In this case, 1 is subtracted from the rest of the undrawn line, and 1 is added to the value of 0 in the first row and the fourth column that overlaps twice, and the result is (d). Subsequently, if a straight line for erasing zero is drawn in the same manner as that performed in (c) in (d), three straight lines appear. In this case, if you obtain the minimum value of 2 in the same way, add 2 to the point where the two lines meet, and subtract 2 from the point where the straight line does not pass, you get the same result as (e). Since there are four straight lines, all methods are over. The optimal solution is to find the row or column in which 0 appears once in FIG.

Below is the Hungarian tree used to find the perfect match.

의 각 정점이 0, 1, 2, 3으로 설정되고, 우측의

정점이 A, B, C, D로 설정되었을 때, 헝가리안 트리로 모든 연결 가능한 경우를 보여주는 도면이다.7 shows the left side of the weighted Bipartite Graph.

Each vertex of is set to 0, 1, 2, 3, and

When vertices are set to A, B, C, and D, they show all possible connections to Hungarian trees.

Referring to FIG. 7, the first level of FIG. 7 is a cost when A is combined with 0, 1, 2, and 3, and the remaining values are set to minimum. The second level, except for the items set in A and the first level, assigns possible paths to B and sets the minimum in the rest. In the same way, the third level is the minimum cost of setting up the remaining two joins with two joins. The characteristic of each level is that lower levels cannot cost more than higher levels. Since the upper level selects the minimum cost without determining the pair in the rest other than the currently set pair, the lower level costs are equal to or greater than the upper level cost. If there is a higher level that has a higher cost than a lower level, the Hungarian tree no longer needs to grow, because the higher level can no longer have a lower level incurring smaller costs. Therefore, if you select one route and create a branch, and the branch is set smaller than the other route, you do not need to create the branch of that route. Using this method, we can quickly infer matching by not performing parent-level operations that can no longer grow.

When using a Hungarian tree, a breadth first search method that calculates routes of the same level first and a depth first search method of selecting one route and creating child nodes may be used. In case of using width priority search, the calculation is completed very fast when the value of child node is set to the minimum value. On the other hand, if the result is determined after many levels of operation, the depth-first search is faster.

Since the character search system uses the color histogram of the images to be compared, high similarity values appear. In this case, it is easy to use the depth-first search method because the width-first search sets up a parent node that cannot grow after many levels of movement. However, it is easy to use the breadth-first search method in the group where the features of the image are completely separated.

8 (a) to 8 (c) are diagrams showing an example of using a Hungarian tree using breadth-first search.

Referring to FIG. 8 (a), FIG. 8 (a) shows the first step of the breadth-first search. In FIG. 8 (a), (A, 0) is combined and the remaining values are all minimum. When the nodes are connected, the weight value is 101, and the remaining routes have values of 105, 99, and 108 type. . The first route in a Hungarian tree is usually the one with the least cost.

Among the three connection nodes, (A, 2), (B, 1) has the smallest cost when 103. Here, a root whose upper level node values are greater than 103 is set as a path that cannot grow because a smaller value can no longer come out. In FIG. 8 (a), since (A, 1) and (A, 3) have a value greater than 103, no further child nodes are created.

In breadth-first search, the child node is also created when the child node goes down one level. FIG. 8 (b) shows a form in which child nodes are formed in pairs (A, 0). Referring to FIG. 8 (b), the node having the smallest cost in the pair of (A, 0) and (A, 2) becomes (A, 2) (B, 1). Therefore, the lowest cost pair is selected first to create a child node.

8 (c) shows a form in which child nodes are made in (A, 2) (B, 1). Referring to FIG. 8 (c), (A, 2), (B, 1), (C, 0), and (D, 3) have a minimum value of 103, and all of the upper values having a value greater than 103 are represented. The nodes no longer need to grow, so the match ends.

Going forward, the depth-first search algorithm looks like this:

9 (a) to 9 (c) are diagrams showing an example of using a Hungarian tree using depth first search. The depth-first search algorithm creates a node with a minimum cost, makes the child node of that node the node with the minimum cost, and then goes to the parent node when the child node corresponding to the minimum cost has reached the end of the level. The next step is to create the lowest cost child node.

Referring to FIG. 9 (a), if (A, 2), (B, 1) is selected in which a child node is created at node (A, 2) corresponding to the minimum cost and the smallest cost is set in the child node, (( A, 1) and (A, 3) become nodes that cannot grow. At this time, after (A, 2) and (B, 1) are selected, (A, 2) and (B, 1) are created next, and child nodes are searched first by depth. In FIG. 9 (b), the next node is made from (A, 2) and (B, 1), and the values of (A, 2), (B, 1), (C, 0) and (D, 3) When set to the minimum value, all nodes except the root node of (A, 0) become nodes that can no longer grow. When the root node of the least cost is selected and the depth search is finished, the depth node is selected again by selecting the parent node that can grow. 9 (c) shows a form in which a child node is created from (A, 0) which is a root node. After creating the child nodes, when the minimum cost of each node is calculated, it becomes a node that can no longer grow because the minimum cost is greater than the value of the lowest child node of (A, 2), so that (A, 2), (B, 1), (C, 0), (D, 3) is the optimized result.

In the above, the depth-first search does not need a new connection to search a tree that cannot be grown, such as the breadth-first search, because the depth search verifies whether the node can grow in the depth search. Depth-first retrieval uses a recursion method that first creates a root, then selects the least cost root of that route, then creates a child node from that route, and each child node that creates a child node again. Before creating a child node, you need to check the need for the child node to be created. To do this, the module that creates the child node, calculates the minimum cost, and sets that node as the least expensive node takes precedence if its value is less than the current minimum cost and it is a lower node than the node with the current minimum cost.

WBM to which the breadth-first search algorithm and the depth-first algorithm described above have been applied has been used to infer the maximum flow and the maximum flow matched pair in the network field so far, and has not been widely used in the image search system. However, in the case of a search using multiple information including shape information and color information as in the present invention, the WBM can derive a very effective search result. In addition, with WBM, you can get information from two of the largest flow and maximum flow matched pairs simultaneously.

However, it is not necessary to obtain the maximum flow matched pair if only checking the similarity in image comparison, so if the computation part of this part can be eliminated, it is possible to perform a very fast search while having the same performance as WBM. Therefore, the multi-feature matching unit 170 uses NMFM matching, which checks only the similarity of images without obtaining the maximum flow matching pair, for multi-feature matching. This method calculates the approximate maximum flow by eliminating the areas where error is likely to occur and by inferring the flow value of the uppermost node pair where maximum matching can occur. It has a fast advantage, making it easy to use in a system that searches for images in real time.

The concept of NMFM matching is as follows.

In the NMFM matching used in the present invention, when extracting and comparing multiple features in an image object, a threshold value is extracted using a statistical method and similarity is measured using this value. When the statistical method is used, the distribution of distance difference between images is obtained by comparing experiments with a large number of image data, and then a specific distance difference σ is obtained when most images are composed of the same image, and when the comparison value with the query image is σ or less. The same image is processed. Most of the image comparisons used in NMFM matching use a statistical method, which uses σ as a specific threshold determined by statistics when a discrimination is required. Therefore, in the present invention, when comparing images using WBM, the σ value must be obtained through experiments in order to distinguish the same image. If there is no σ value, WBM simply sorts the images in the order of similarity. The purpose of the image search is not to sort the similarity of the images, but to search for the same image or an image having a similarity less than a specific value. Therefore, even when comparing images using WBM, σ values should be obtained through experiments.

Until now, the method of using sigma as a specific threshold has been used to determine whether the sigma value is the same image after the comparison result of the image, but the NMFM proposed in the present invention has the characteristic of using the sigma value as a prediction index. The threshold value σ is not a fixed value but a value that is continuously changed statistically according to the accumulation of data, and may be arbitrarily adjusted within a predetermined range by the user's adjustment depending on the configuration of the circuit.

The specific threshold σ is calculated in a statistical manner using the similarity distribution of the images. The similarity distribution equation for obtaining the threshold value σ is as shown in Equation 6 below.

Di = Ws -Wd

Here, Di means a distance difference between Ws, which is a query image, and Wd, which is a database image. N (Di) represents the distance difference of i value when all n images are compared, and the overall distribution is the sum of N (Di) values, which is S (D). .

10 is a distribution curve of the same sector image obtained from Equation (6). The threshold σ shown in FIG. 10 may vary from system to system when determining the standard deviation after the actual experiment. For example, in a system requiring 95% accuracy, the end point of the region where 95% of the same image is located is determined as σ. In NMFM, σ is used to set up a node that no longer grows in the Bipartite Graph.

In NMFM, N edges are set, the minimum cost of each edge is set in a matrix, and a pair having the minimum cost is obtained. The method of finding the minimum flow is shown in [Equation 7].

M (x, y) = min (sigma -W _{x'y '} )

In Equation 7, M (x, y) is a pair having a minimum flow, and W _{x'y '} is weight values connected to a specific sector. Once the minimum flow pair is determined, remove the pair and then repeat the next pair using Equation (7). As such, the NMFM obtains the next total flow when the pair with the least cost is found and sees this flow as the approximate maximum flow. In the NMFM algorithm, if the weights of the entire pairs are all greater than σ (i.e. no minimum flow pairs exist), WBM is used instead of NMFM to derive the optimized flow. The overall algorithm of NMFM is as follows.

FIG. 11 is a flowchart illustrating an NMFM matching method according to a preferred embodiment of the present invention, which is performed by the multiple feature matching unit 170 illustrated in FIG. 1, and FIG. 12 is a diagram illustrating an example of matching NMFM.

11 and 12, the NMFM matching method according to the present invention first establishes all connectable connections in the Bipartite Graph (step 7110), and extracts a pair that obtains the minimum cost (step 7120, FIG. 12A). ) Reference). In this case, the difference between the weight of the pair having the minimum cost and σ is calculated (step 7130). Here, σ is a variable calculated using a statistical method, and is a specific distance difference obtained from a case where most distances are composed of the same image after obtaining a distribution of distance differences between images through comparison experiments of many image data. Subsequently, it is determined whether the weights of all pairs are greater than σ (i.e., whether the distribution of the distance difference between images is out of the range of σ) (step 7150). As a result of the determination in step 7150, if the weights of all pairs are larger than σ (that is, the distribution of distance difference between images is out of the range of σ), WBM matching is performed to find an optimal flow and matched pair (step 7200). . As a result of the determination in step 7150, if the weights of all the pairs are not larger than σ (that is, the distribution of the distance difference between the images does not deviate from the range of σ), the connection pair is continuously obtained (step 7300, FIG. 12). (b)). At this time, the connection pair obtained in step 7300 is determined by Equation (7), sets a negligible edge coefficient within the error range, and sequentially ignores the pair over the corresponding coefficient. Then, the average flow value is calculated by dividing the flow value by the value of the connected edge to obtain an approximate maximum flow (step 7400, see FIG. 12C).

That is, if the image falls within the interval of sigma, NMFM matching is performed as in steps 7300 and 7400 of FIG. 12 to obtain an approximate maximum flow, and if the image is out of the range of sigma, or if there are no images in the sigma range. In this case, the image is processed as an unequal image. This feature allows the NMFM to save time in two cases: In the first case, when there is no image in the region of σ value, the calculation is performed as a completely different image without any further calculation, so that the calculation can be performed quickly without comparison. Second, in the case of calculating and aligning the distance difference in all images, the operation saves time by performing an operation using NMFM for the images in σ, and performs the operation using WBM for the remaining parts. Similar images can be sorted.

However, when the sector image is placed at the edge, the distance difference in a specific bin is very large in the image with a large change. The value of this distance difference appears as a large error value in the similarity of the image. The NMFM ignores sector images that show very large distance differences and maintains maximum flow with only the remaining sectors. [Equation 8] is to calculate the maximum flow.

는 무시하고자 하는 섹터의 개수를 의미한다.here,

Denotes the number of sectors to be ignored.

As can be seen from Fig. 11, the image retrieval method according to the present invention determines whether the weights of all pairs are greater than sigma, and if the weights of all pairs are greater than sigma (i.e., within the region of σ values). If there is no image belonging) WBM is performed to obtain an optimal flow. If there are images belonging to the region of σ, NMFM is performed to obtain an approximate maximum flow.

The big difference between these NMFMs and WBMs is whether or not to extract matched pairs. In addition to the maximum flow chart, WBM also finds matching pairs when the maximum flow chart is achieved. However, NMFM extracts the maximum flow diagram but does not find a matched pair. The purpose of image matching is not to find matching pairs, but to find the maximum flow chart. The computational complexity is much smaller than that of WBM because it eliminates unnecessary operations for finding matched pairs and predicts flow diagrams like WBM. Using the WBM algorithm for N vertices, the fastest way is to do n-1 traversals. If the flow chart is less than or equal to σ of n-1 traversals of the NMFM, the NMFM is always equal to or less than the operation of the WBM because no further traversal is necessary. Therefore, the operation of NMFM is faster than WBM. As will be explained in detail in the experimental results below, NMFM appears twice as fast as WBM in actual experiments.

Let's look at how the WBM and NMFM algorithms with these characteristics are applied to character image retrieval.

In order to use WBM to search for a character image, information to be compared should be set as an edge. The character search system divides the information of the image into n sectors. It is divided into n sectors, and color histogram information of the corresponding sector becomes a weight value of each sector. Each sector has a weight of R, G, and B, and one sector including these three values becomes the edge of the WBM. Therefore, the information to be compared to the edge of the WBM becomes three area values.

Color values contributing to one sector should be set with weights of R, G, and B. This is because if the weight is not set, there is a sector image having a large R value and a sector image having a large G value. If the total values of the sectors are the same, the color sizes are the same. [Equation 9] is a formula for obtaining the total value of each of R, G, and B. Before performing Equation 9, the color histogram of the sector image is obtained by the preprocessor 110 illustrated in FIG. This is followed by quantization.

는 섹터에 있는 R, G, B 색상의 크기 값이고,

는 각각의 빈에 해당하는 번호이며,

는 해당 번호의 히스토그램값이다. 양자화된 히스토그램에서 각각의 히스토그램의 빈의 번호와 해당 빈의 히스토그램 수를 곱하게 되면 섹터 안에 있는

값을 얻게 된다. 이 값은 섹터 이미지에 있는 각 색상의 크기이다. [수학식10]은 이 크기의 비율을 값을 얻기 위한 수학식이다.In Equation 9,

Is the magnitude value of the R, G, and B colors in the sector,

Is the number corresponding to each bin,

Is the histogram of the number. In a quantized histogram, multiply the number of bins in each histogram by the number of histograms in that bin.

You get a value. This value is the size of each color in the sector image. [Equation 10] is an equation for obtaining a value of this size ratio.

는 R, G, B의 각각의 가중치 값이고,

는 [수학식10]에서 얻어진 R, G, B의 수치이며,

는

의 값을 모두 더한 수치가 된다. 이 경우, [수학식10]에서 얻어진 각각의 가중치 값을 이용하여 거리차를 구한 다음 해당 거리차에 가중치 값을 곱해야만 기여되는 색상의 거리차를 얻을 수가 있다.here,

Is a weight value of each of R, G, and B,

Is a numerical value of R, G, B obtained from [Equation 10],

Is

Is the sum of all the values of. In this case, the distance difference can be obtained by calculating the distance difference using each weight value obtained in [Equation 10] and then multiplying the distance difference by the weight value.

If you want to find the minimum cost from one edge to another using WBM, you need to find the distance difference of the histogram. Although multiple values can be configured on one edge in WBM, the cost of the path should be treated as one value when the path is formed. The color distance difference in the image is represented by three distance differences of R, G, and B, and this distance difference must be converted into one value. The method of calculating the distance difference using the histogram of each sector is as follows.

As shown in [Equation 11], when the distance difference is obtained by using the histogram value, the distance difference of each of R, G, and B comes out. In order to use WBM, one weight value must be derived with each of these values.

Equation 12 is a formula for deriving one weight value.

In Equation 12, the W value becomes a weight of each sector image and can be matched using the weight.

When using WBM to compare images, depth-first search is more advantageous than width-first search. The reason is to use the distance difference when calculating the minimum cost from one node. When using the distance difference in the image, the distance difference of the histogram of the three areas of R, G, and B is multiplied by each weight value and the minimum value. This is because the information used to calculate the distance difference is not single information. Therefore, the part where the distance difference between one sector is minimized can be viewed as the same image. When selecting a value of a certain sector and looking at the minimum value in the remaining part, the result is similar to the minimum cost flow. WBM is also one of the more complex using trees. By using the features in the image and using the part where the depth-first search algorithm is faster than the width-first search algorithm, a faster algorithm can be developed and used for real-time video search.

When using WBM, it is divided into two types: the purpose of inferring the maximum flow value and the connection of the edges optimized for the maximum flow. It is an object of the present invention to infer the value of the maximum flow. Even if you derive the approximate maximum flow, the distance difference of the image can be obtained, so it is not necessary to derive the exact matching. NMFM does not calculate the maximum flow in a two-pronged graph, but rather the approximate maximum flow value. In order to make complete matching, the calculation is done using tree-type nodes, in which the parent node's flow is not larger than the child node's flow and the parent node's smaller than the child node's flow is removed from the maximum flow. have. This feature of the NMFM is used to predict the approximate flow in advance of the tree's parent and to obtain the maximum flow value of the predicted result. The WBM method derives the optimal pair, but because the optimal pair is meaningless in this system, a fast operating system is needed. In character image, WBM and its performance appeared in the same form. In addition to the database image, character search is used in various areas. When searching for a character in real time in a video, you must use a very fast matching method. The NMFM proposed in the present invention reduces the calculation amount of about 1/2 of the WBM because the NMFM makes a decision in the middle of the search without searching all the search steps of the WBM. There are many object images with similarity in the image database, but not in the image. Therefore, it is easy to search using NMFM to search for real-time data.

The search function of the image retrieval system can be divided into two operations, one for sorting according to the similarity and one for not sorting. When the images are sorted according to the similarity, all distance differences of the images to be compared should be set. In the case of searching only similar images without the need for sorting, NMFM is used and it is checked whether there are sectors in the area of? Value. Then, if there are no sectors in the σ area, they are regarded as different images. However, in the case of similarity alignment, the distance difference value should be set even when the images are not the same.

FIG. 13 is a diagram for describing a method of using NMFM for similarity alignment in the multiple feature matching unit 170 shown in FIG. 1.

Referring to FIG. 13, when the multi-feature matching unit 170 compares sectors of an image and there is at least one sector in the sigma value region in the matrix of comparison results, the image uses the NMFM method as it is. . The matrix data at the top of FIG. 13 is in the form of data in the sigma region. In this case, the distance difference (W = 1.64) comes out by using the value of NMFM. The matrix data in the lower part of FIG. 13 is a case where no value exists in the sigma region. In this case, try the first step into WBM to find a possible minimum cost value, and at the same time, if the NMFM does not think about σ, connect the pairs in order of the minimum cost, and find the difference between them, if they are within the mean standard deviation. Consider the value as the minimum cost. If not, use WBM to get the minimum cost.

Hereinafter, the experimental results and the performance thereof for the image retrieval system 100 according to the present invention will be described. Before describing the evaluation scale for evaluating the performance of the image retrieval system 100 as follows.

In content-based retrieval systems, evaluation scales are important indicators of the accuracy of the system. In the present invention, in the character search, evaluation scales are used as BEP (Bulls Eye Performance), NMRR (Normalized Modified Retrieval Rank), and NDS (Normalized Distance Sum).

First, the BEP is an evaluation measure of the shape and motion descriptor of MPEG-7, and means a percentage value in which an image is contained within twice the total number of images including the search image in the search image. Equation 13 below calculates the coefficient of BEP.

In Equation (13), NR (q) is the number of the same video image within 2 multiples, and NG (q) is the total multiple number of multiple images. For example, when 10 identical images exist, all images must exist within 20 images to approximate 1 to the BEP evaluation scale.

Subsequently, NMRR is a measure of the color and texture descriptors of MPEG-7, by B. Kroepelien, A. Vailaya and A. K. Jain, 1996, Proc. of IEEE Int. Conf. on Pattern Recognition, vol. 3, pp. The papers in 370-374, "Image database: A case study in Norwegian silver authentication," and the like describe the concept of NMRR. NMRR is a method of calculating how the same video as the queried image is sequentially received when the NR (q) is the same as the image queried in the entire image NG (q). Since BEP is a commonly used measure of video, no ordered values are recorded, but since NMRR is a part of searching for patterns and shapes of still images, the evaluation scale is set based on how many images are in front of the sequence of images found. . The evaluation scale of NMRR is shown in [Equation 14].

In Equation 14, GTM means max {MG (q)} value in all query images in the database image.

In 2000, the NDS was published by Seo Chang-deok and Kim Ho-ryul in the Journal of the Korean Society of Broadcast Engineers, Vol. 5, No. 1, pp. The evaluation scale proposed in the paper in 68-81, "Meaning-based Similar Trademark Search and New Search Efficiency Evaluation Scale in Image Databases," which shows that the closer to 0, the better the performance, and all the evaluation conditions are considered. There is an advantage.

Experiments with the present invention were performed using 500 images consisting of 39 groups of 39 images and 2500 images not grouped. First, we compared and searched the images of other groups using only the grouped images, and then examined how many identical images were found by randomly comparing 2500 ungrouped images with the grouped images.

Grouped images are divided into three parts. It is a video system with a similar color with a single color system, a video object with various colors combined, and a video object with a new form. In the experiment, three groups of 39 groups were placed in each of 13 groups. This is to evaluate the performance of the proposed system and the existing system more accurately by using image objects having three characteristics in the character retrieval system. In addition, since the experiment may be impossible to obtain accurate information when the experiment is performed only on the grouped images, 2000 general images that are not grouped are mixed.

In the experiment, the computer system used Pentium III 800 Mz and protected mode 32 bit Windows XP. The language used C / C ++ and Visual C ++ and MFC to infer the GUI results. All the search algorithms required for the experiment were fabricated by hand and operated at the standard application level without using special hardware accelerators for accurate performance evaluation.

Performance evaluation results for the present invention are shown in [Table 1] and 14 to 16.

This experiment is the result of Swain's color indexing method and the proposed algorithm, WBM and NMFM and Gevers's combination and color combination test results. [Table 1] shows four algorithms (ie, global color index, WBM, NMFM ( Swain's color index) and PicToSeek (Gevers' proposed form and color information combination) are evaluated, and FIGS. 14 to 16 are performance evaluation graphs for the four methods shown in Table 1, respectively. The smaller the numbers, the better the NMRR and NDS. The closer to 1, the better the BEP.

As can be seen from Table 1 and FIGS. 14 to 16, WBM and NMFM yielded good performances, respectively, with similar results, and PicToSeek followed Swain's method with the lowest performance. In terms of overall color, Swain's method has shown poor performance in areas with many color variations. Gevers's method has better performance than Swain's method, but since the shape information is used as geometric information, when the shape is changed in the character image, the distance difference is large compared to the geometric information. The performance is lower.

17 is a performance evaluation graph according to the number of quantization levels of GHM, WBM, and NMFM.

Referring to FIG. 17, the performance of the search is guaranteed when the number of bins exceeds the threshold level when quantizing the color histogram, and the optimized performance is maintained when all search methods using the color are at least 8 bins. In the case of GHM, the results show that the number of bins is at least slightly better than WBM and NMFM. The reason is that GHM effectively adapts to the image of characters that are monochromatic when creating color histogram without sector division. to be.

Search speed performance evaluation of WBM and NMFM is shown in Table 2 below and FIG. 18.

18 is a graph illustrating a search speed performance evaluation of WBM and NMFM for the image search system 100 according to the present invention.

As can be seen from Table 2 and FIG. 18, it can be seen that according to the present invention, NMFM shows an image search speed about 2 times faster than that of WBM. The reason for the similar speed when comparing 100 images is that there is a minimum that the system can measure. In the case of a small number of images, the speed measurement is similar, but when comparing a large number of images, the speed difference is remarkable. If you want to search for the character image in the video, you have to search for 30 frames per second. At this time, as the pre-processing process involves dividing and extracting the object to be searched from the image, and afterwards, the search process is required. In this case, NMFM is faster than WBM, so the application performance is excellent.

In the above, the configuration and operation of the circuit according to the present invention are shown in accordance with the above description and drawings, but this is merely described, for example, and various changes and modifications are possible without departing from the spirit of the present invention. .

In particular, the image retrieval system 100 according to the present invention has been described for finding the same character by using the multi-characteristic information of the character image, but this is only an example of a retrieval system using the multi-feature information, a large amount of The present invention can be applied to any retrieval system for retrieving specific information from a database in which information is stored. For example, if you want to search for students who meet certain criteria (especially multiple features that match multiple criteria) in a database where scores for multiple students are stored, applying the present invention can provide more accurate search in a shorter time. You can get the result. In this case, the application example divides the object to be searched into n sectors (eg, grades for each subject), and measures the optimal similarity using the threshold value σ obtained using a statistical method. . In this case, since the information used for the search is the multi-characteristic information, the desired data can be searched more diversely than when searching for the single-characteristic information. Since the present invention uses the NMFM method to be suitable for the search of the multi-characteristic information. The result is faster and more accurate search results.

The invention can also be embodied as computer readable code on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like, and may also be implemented in the form of a carrier wave (for example, transmission over the Internet). Include. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

According to the present invention as described above, it is possible to effectively search for a character image that is difficult to search using one type of information or color information by using the multi-feature information combined with at least two feature information, and search for the character image. Speed and accuracy are improved.

Claims (12)

A database in which a plurality of data are stored; A data dividing unit dividing each of the plurality of data into a plurality of regions; A quantizer configured to quantize a color histogram of the divided regions using a Parzen window configured to partially overlap the adjacent windows and a weight value of the color; A feature extractor for extracting feature values for the regions from the quantization result; And The threshold value is determined from a distribution of the distance difference between the query data and the similar data, and the similarity between the query data and the plurality of data is determined by detecting whether the feature value is within a range of the threshold value among the plurality of data. Including a multi-feature matching unit, The multi-feature matching unit, Establish all connectable connections in the Hungarian tree for the plurality of data; Extract the pair with the lowest cost in the connection; Calculate a difference between the threshold given to the weighted pair and the threshold; And if the calculation result does not deviate from a predetermined range, an approximate maximum flow is obtained by iteratively obtaining a pair having the minimum cost among the connections except the pair having the minimum cost. The method of claim 1, Each of the divided regions includes both color and morphological features of the data. The method of claim 1, The pazen window is

Consists of,

Is the original histogram level,

Is the histogram level to be quantized,

Are values of the histogram corresponding to the divided region,

Is a histogram value corresponding to the quantization result for the divided region, and Q is calculated using a weight value of color as a quantization level value. The method of claim 1, The multi-feature matching unit does not perform matching any more when data belonging to the range of the threshold value does not exist among the plurality of data stored in the database. The method of claim 1, The multi-feature matching unit searches for a similar image by performing weighted bipartite matching (WBM) when there is no data belonging to the range of the threshold value among the plurality of data stored in the database. Video retrieval system. The method of claim 1, The multi-feature matching unit searches for similar images by performing Near Maximum Flow Matching (NMFM) when there is data belonging to the range of the threshold value among the plurality of data stored in the database. Video retrieval system. Dividing each of the plurality of data stored in the database into a plurality of regions; Quantizing a color histogram of the divided regions using a Parzen window configured to partially overlap adjacent windows and a weight value of the color; Extracting feature values for the regions from the quantization result; And The threshold value is determined from a distribution of the distance difference between the query data and the similar data, and the similarity between the query data and the plurality of data is determined by detecting whether the feature value is within a range of the threshold value among the plurality of data. Including the steps of: Determining the similarity, Establishing all connectable connections in the Hungarian tree for the plurality of data; Extracting the pair with the lowest cost in the connection; Calculating a difference between a weight given to the pair of minimum cost and the threshold; And if the calculation result does not deviate from a predetermined range, obtaining an approximate maximum flow by iteratively obtaining a pair having the minimum cost among the connections except the pair having the minimum cost. . The method of claim 7, wherein Each of the divided regions includes both color and morphological features of the data. The method of claim 7, wherein The pazen window is

Consists of,

Is the original histogram level,

Is the histogram level to be quantized,

Are values of the histogram corresponding to the divided region,

Is a histogram value corresponding to the quantization result for the divided region, and Q is calculated using a weight value of color as a quantization level value. The method of claim 7, wherein The determining of the similarity may further include performing no matching when the calculation result is out of a predetermined range. The method of claim 7, wherein The determining of the similarity may further include performing weighted bipartite matching (WBM) to determine the approximate maximum flow and data pair when the calculation result is out of a predetermined range. How to search video. A computer-readable recording medium having recorded thereon a program for executing the image retrieval method according to any one of claims 7 to 11.

Images