KR101435010B1 - Method for learning of sequential binary code using features and apparatus for the same - Google Patents

Abstract

A sequential binary code learning method and apparatus using a plurality of features capable of providing high accuracy in retrieving an image composed of a plurality of features is disclosed. A step of extracting features from the acquired query image, a step of learning a multiple view hashing algorithm using the extracted features, and a step of acquiring a binary code corresponding to the binary code generated through the learned multi- And retrieving the image having the code. Thus, it is possible to provide a high degree of accuracy in retrieving an image composed of a plurality of features, and may be successfully applied to a content-based image retrieval system.

Description

TECHNICAL FIELD The present invention relates to a sequential binary code learning method and a learning apparatus using a plurality of features,

The present invention relates to a data-based hashing technique, and more particularly, to a sequential binary code learning method and apparatus using a plurality of features capable of generating binary codes with high accuracy using a plurality of features of an image .

The nearest-neighbor search problem is a problem of finding the most similar data to the query data in a given data.

Here, the nearest neighbors are continuously studied in the areas of computer vision, information retrieval, and machine learning.

Also, an important application in recent point-to-point search problems is content-based image retrieval (CBIR).

Here, CBIR is a technique for retrieving the image most similar to the query image in the database. In order to successfully execute the CBIR, it is necessary to search the image similar to the query image in a short time.

The easiest way to solve the nearest neighbor search problem is to find the most similar data by calculating the similarity between the query data and all the data in the database. In this simple method, the time complexity is proportional to the number of data in the database.

In applications such as CBIR, since the number of images in the database is very large, more than one million, the nearest neighbor search problem can not be solved by the simple method described above.

Therefore, an algorithm that utilizes a tree structure has been proposed to solve the nearest point search problem with a small time complexity.

However, the algorithm using the tree structure can obtain the closest neighbors with a lower dimension than the linear time complexity when the data is expressed in low dimensions. However, when the data is expressed in high dimensional order, .

Therefore, it is difficult to find the nearest neighbors to be lower than the linear time complexity in the case of high-dimensional data. Therefore, researches to retrieve approximate nearest neighbor (ANN) quickly without retrieving query data and accurate shortest neighbor data .

Hashing is an algorithm for designing a hash function that converts high-dimensional data into binary code as a representative method for solving ANN. Here, the hash function is designed so that the degree of similarity between codes can reflect the similarity of data.

In the early days of hashing, a hash function was randomly generated without regard to the distribution of data, and the theoretical basis for the similar data to have the same binary code was proposed. However, since the hash function that does not consider the data distribution can not reflect the similarity of the data, a hashing algorithm that maximizes the similarity of the data has recently been proposed.

The data-based hashing algorithm aims to find binary code with a small code length while preserving the similarity of the data.

Data-based hashing algorithms can be broadly divided into spectral hashing, iterative quantization, anchor graph hashing, semi-supervised hashing, and semantic hashing and supervised hashing with kernel.

Here, most data-based hashing algorithms assume that an image consists of only a single feature, but since an image can be represented by several heterogeneous features, it is necessary to study a hashing algorithm using various features. A data - based hashing algorithm using multiple features is named multi - view hashing.

A typical multi-view hashing algorithm is an extension of the spectral hashing method and aims to find an integrated binary code that preserves the integrated similarity.

However, the conventional multi-view hashing algorithm has a problem that the performance is degraded as the length of the binary code increases and the integrated similarity matrix is simply obtained.

SUMMARY OF THE INVENTION It is an object of the present invention to solve the above problems and provide a sequential binary code learning method using a plurality of features capable of providing high accuracy in searching an image composed of a plurality of features.

It is another object of the present invention to provide a sequential binary code learning apparatus using a plurality of features for performing a sequential binary code learning method using the plurality of features.

According to another aspect of the present invention, there is provided a sequential binary code learning method using a plurality of features according to the present invention, the method comprising: acquiring a query image; Extracting features from a query image, learning a multi-view hashing algorithm using the extracted features, and retrieving an image having a binary code similar to the binary code generated through the learned multi-view algorithm .

Here, learning the multiple view hashing algorithm using the extracted features may include obtaining a distance matrix from the extracted features, and generating an alpha- integrated similarity matrix using the alpha- And sequentially learning binary codes using the generated alpha- integrated similar matrices.

Here, the step of obtaining a distance matrix from the extracted features and generating an alpha- integrated similar matrix using the alpha- conjugation method on the obtained distance matrix may include: dividing the extracted feature into two subsets through principal component analysis Dividing the extracted features into a predetermined number of data by using the two subsets, and calculating a distance matrix corresponding to a view of each data divided into the predetermined number, Generating a kth nearest neighbor graph and combining the k nearest neighbor graphs generated in the two subset with a k nearest neighbor graph to generate a k nearest neighbor graph.

Here, the step of sequentially learning the binary codes using the generated alpha- integrated similar matrices may sequentially learn the binary codes by approximating non-diagonal elements.

According to another aspect of the present invention, there is provided a sequential binary code learning apparatus using a plurality of features according to an exemplary embodiment of the present invention. The sequential binary code learning apparatus includes a storage unit, an image acquisition unit that acquires a query image, Extracting features from the query image provided from the image acquiring unit, learning a multi-view hashing algorithm using the extracted features, and generating a binary code having a binary code similar to the binary code generated through the learned multi- And a processing unit for searching an image in the storage unit.

Here, the controller obtains a distance matrix from the extracted features, generates an alpha- integrated similarity matrix using the alpha- conjugation method on the obtained distance matrix, and then uses the generated alpha- Binary code can be learned sequentially.

Here, the controller may divide the extracted feature into two subsets through principal component analysis, divide the extracted features until a predetermined number of pieces of data are obtained using the two subsets, The k nearest neighbor graph is generated by performing alpha combining on the distance matrix corresponding to the view of each data divided by the number, and the k nearest neighbor graph generated from the two subset is combined into one k nearest neighbor graph Can be generated.

Here, the controller may sequentially learn binary codes by approximating non-diagonal elements.

According to the above-described sequential binary code learning method and apparatus using a plurality of features according to an embodiment of the present invention, a plurality of features are extracted from the acquired query image, and alpha- A similarity matrix is generated, and the binary code is sequentially learned using the generated alpha- integrated similarity matrix.

Thus, it is possible to provide a high degree of accuracy in retrieving an image composed of a plurality of features, and may be successfully applied to a content-based image retrieval system.

1 is a conceptual diagram illustrating a sequential binary code learning method using a plurality of features according to an embodiment of the present invention.
FIG. 2 is a flowchart illustrating a sequential binary code learning process using a plurality of features according to an embodiment of the present invention.
FIG. 3 is a flowchart illustrating a step of learning a multiple view hashing algorithm in FIG.
FIG. 4 is a graph illustrating the accuracy of an alpha-integrated similarity matrix according to alpha value change according to an embodiment of the present invention.
Figure 5 illustrates an algorithm for quickly obtaining an alpha integration similarity matrix according to an embodiment of the present invention.
6 is a conceptual diagram for explaining an algorithm for sequentially learning binary codes according to an embodiment of the present invention.
FIG. 7 is a block diagram illustrating a configuration of a sequential binary code learning apparatus using a plurality of features according to an embodiment of the present invention.
FIG. 8 shows an image search experiment using a sequential binary code learning method using a plurality of features according to an exemplary embodiment of the present invention, and an image search experiment result using a conventional method.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail.

It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In the present application, the terms "comprises" or "having" and the like are used to specify that there is a feature, a number, a step, an operation, an element, a component or a combination thereof described in the specification, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the relevant art and are to be interpreted in an ideal or overly formal sense unless explicitly defined in the present application Do not.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In order to facilitate the understanding of the present invention, the same reference numerals are used for the same constituent elements in the drawings and redundant explanations for the same constituent elements are omitted.

1 is a conceptual diagram illustrating a sequential binary code learning method using a plurality of features according to an embodiment of the present invention.

Referring to FIG. 1, a sequential binary code learning apparatus 100 (hereinafter, referred to as a sequential binary code learning apparatus) extracts various image features such as GIST and HOG from a query image, The integrated binary code (0101001) is generated through a multi-view hash function using various image features.

Then, the sequential binary code learning apparatus 100 compares the generated integrated binary code 0101001 with binary code corresponding to an image stored in the storage unit, and searches for the most corresponding image.

Hereinafter, a sequential binary code learning process using a plurality of features performed by the sequential binary code learning apparatus 100 according to an embodiment of the present invention will be described.

FIG. 2 is a flowchart illustrating a sequential binary code learning process using a plurality of features according to an embodiment of the present invention.

Referring to FIG. 2, the sequential binary code learning apparatus 100 determines whether a query image is acquired (S210), and extracts a plurality of features from the acquired query image if it is determined that the query image is acquired (S220) .

Thereafter, the sequential binary code learning apparatus 100 learns a multiple view hashing algorithm using a plurality of features extracted in step 220 (S230).

In step S240, the sequential binary code learning apparatus 100 retrieves the image corresponding to the query image from the pre-stored image using the learned multi-view hashing algorithm in operation 230.

Hereinafter, a process of learning a multiple view hashing algorithm using the extracted features of step 230 will be described in detail with reference to FIG.

FIG. 3 is a flowchart illustrating a step of learning a multiple view hashing algorithm in FIG. 2, FIG. 4 is a graph illustrating an accuracy of an alpha integrated similarity matrix according to an alpha value change according to an embodiment of the present invention, And shows an algorithm for quickly obtaining an alpha integration similar matrix according to an embodiment of the present invention. 6 is a conceptual diagram illustrating an algorithm for sequentially learning binary codes according to an embodiment of the present invention.

Since the multi view hashing algorithm according to an embodiment of the present invention is an extension of the spectral hashing method, a description of spectral hashing will be briefly described first.

Spectral hashing is intended to find a compact binary code that can preserve the similarity of data well, and the objective expression is expressed by Equation 1 below.

는 i 번째 데이터와 j 번째 데이터의 유사도를 표현한 유사행렬,

는 i번째 데이터에 해당하는 바이너리 코드,

는 데이터의 바이너리 코드를 모두 모아둔 바이너리 코드 행렬, M과 N은 바이너리 코드 길이와 데이터 개수를 의미한다.In mathematics room 1

A similarity matrix expressing the degree of similarity between the i-th data and the j-th data,

Is a binary code corresponding to the i-th data,

Denotes a binary code matrix in which binary codes of data are all collected, and M and N denote a binary code length and the number of data.

Equation (1) is a representative NP problem and can not find an exact answer within a polynomial time. Therefore, spectral hashing uses the Laplace-Beltrami operator to obtain the binary code.

There are two main methods of existing multiple view hashing algorithm based on spectral hashing. The first method is composite hashing with multiple information sources (CHMIS) and the second method is multi-view hashing for cross-view similarity (MVH-CS).

To briefly describe the first method, the binary code for a particular view is defined as: < EMI ID = 2.0 >

In Equation (2), sgn is a sign function. CHMIS can be obtained by summing the degree of similarity obtained from a specific view. By applying the integrated similarity thus obtained to spectral hashing, an integrated binary code can be obtained by the following equation (3).

Since Equation (3) is the same problem as spectral hashing, an exact solution can not be found, so an approximate solution should be sought. To obtain an approximate solution, the following Equation (4) is used.

는 특정 뷰 k에 대한 투영 행렬이다. 수학식 4를 해결하면 통합된 바이너리 코드는 다음의 수학식 5를 통해 얻을 수 있다.In Equation 4,

Is the projection matrix for a particular view k. By solving Equation (4), the integrated binary code can be obtained by the following Equation (5).

각 뷰에 대한 코드를 합쳐서 구한다.Here, MVH-CS is an algorithm for obtaining similar binary codes for different views in the case of similar data. After generating the binary code for each view, the integrated binary code looks like this

The code for each view is combined.

However, the multi-view hashing algorithm described above has a disadvantage in that performance is degraded as the length of the binary code increases, and an integrated similarity matrix is simply obtained.

Referring to FIG. 3, the sequential binary code learning apparatus 100 acquires a distance matrix from a plurality of features extracted through step 220 of FIG. 2 (S231).

Then, the sequential binary code learning apparatus 100 generates an alpha integrated similarity matrix using the alpha-combining method on the distance matrix obtained in step 231 (S233).

Here, the alpha combining method for generating the alpha integrated similar matrices will be described in detail.

Alpha combining method is a generalized combining method which can express widely used arithmetic mean, geometric mean, harmonic mean, etc. according to the change of alpha value.

이 주어진 경우 통합된 거리 행렬은 다음의 수학식 6을 통해 구할 수 있다.K distance matrix

The integrated distance matrix can be obtained by the following equation (6).

는 k번째 뷰에 대한 scailing 요소이며

는 양수 a와 b에 대한 alpha-divergence이고 다음의 수학식 7로 정의된다.In Equation (6)

Is the scailing element for the kth view

Is the alpha-divergence for the positive numbers a and b and is defined by the following equation (7).

The solution of Equation (6) is expressed by Equation (8) and is represented by arithmetic mean (alpha = -1), geometric mean (alpha = 1), and harmonic mean (alpha = 3) according to alpha value.

The solution of Equation (6) is expressed by Equation (16) and expressed as an arithmetic mean (alpha = -1), a geometric mean (alpha = 1), and a harmonic mean (alpha = 3) according to an alpha value. The Alpha-combining method is a method of giving more weight to a smaller value (distance) as the alpha value increases.

Referring to FIG. 4, each graph represents a difference in performance for a multiple view hashing algorithm according to an embodiment of the present invention according to a change in alpha value.

Caltech-256, CIFAR-20, and NUS-WIDE datasets were used.

4, there is a performance difference according to alpha value, and when alpha is 1 or more, good performance is provided. This can be explained by an experimental basis that it is reasonable to give a little more weight to a small distance.

Referring to FIG. 5, FIG. 5 illustrates a fast alpha combining method. Assuming that the number of data is N, simply performing alpha combining requires a time complexity of 0 (N ^ 2) I can not.

To solve this problem, we solved the alpha coupling quickly by using the divide-and-conquer technique.

This method is an algorithm developed by modifying the existing method of fast kNN (J. Chen, JMLR-09). First, in the divide process, the data is divided into two subsets through principal component analysis using Equation (9).

는 데이터 행렬의 주성분 요소이며

은 두 개의 부분 집합이 서로 얼마나 많은 원소들을 공유하는지 결정하는 상수이다. 또한, 수학식 9를 이용하여 데이터가 일정 개수 이하일 때가지 지속적으로 나눈다.In Equation (9)

Is the principal component of the data matrix

Is a constant that determines how many elements the two subsets share with each other. Further, using Equation (9), the data is continuously divided until the number of data is equal to or smaller than a certain number.

When the number of data becomes very small, alpha combination is performed on the distance matrix corresponding to each view, and then k nearest neighbor graph is generated. The k closest graphs generated from the two subsets are combined into one k closest graph and the k closest graph for all data can be obtained by repeating this process.

Referring back to FIG. 3, the sequential binary code learning apparatus 100 sequentially learns binary codes using the alpha integrated similarity matrix generating step 233 (S235).

Here, the process of sequentially learning the binary codes will be described in detail.

Sequential learning of binary codes, that is, a sequential binary code learning algorithm, is the same as the existing multi view hashing method, in which the spectral resolution method is extended to multiple view data.

를 잘 보존하는 통합된 바이너리 코드를 찾는 문제로 본 발명에 따른 일 실시예에서도 수학식 10을 해결하기 위해 순차적 해쉬 함수 학습 알고리즘을 개발하였다.The following equation (10)

In order to solve Equation (10), a sequential hash function learning algorithm has been developed in an embodiment according to the present invention.

은 통합된 유사행렬,

는 j번째 데이터에 해당하는 통합된 바이너리 코드이다. 통합된 바이너리 코드 간의 거리는 수학식 11로 정의된다.In Equation (10)

Is an integrated similarity matrix,

Is an integrated binary code corresponding to the jth data. The distance between the integrated binary codes is defined by Equation (11).

는 k번째 뷰에 해당하는 i번째 데이터의 바이너리 코드이며 수학식 12로 정의된 해쉬 함수를 통해 구할 수 있다.In Equation (11)

Is the binary code of the i-th data corresponding to the k-th view and can be obtained through a hash function defined by Equation (12).

Using Equation (12), the objective equations of 10 solved by the multi-view hashing algorithm can be summarized as Equation (13).

로 근사할 수 있다. 상기 과정에서 근사한 통합된 바이너리 코드를 이용하면 수학식 10을 수학식 14로 표현할 수 있다.The integrated binary code through equation (13)

. Equation (10) can be expressed by Equation (14) using an approximate integrated binary code in the above procedure.

Equation (14) can be solved by the generalized eigenvalue problem, but the binary code obtained through this has a disadvantage that it does not reflect the similarity of data as the code length becomes longer. This is a problem that occurs when the non-correlation condition between binary codes is simply approximated as shown in Equation (10).

In order to solve this problem, an embodiment of the present invention has developed an algorithm for sequentially learning binary codes.

이며, 도 6에서 확인 할 수 있듯이 대각 요소는 어떤 바이너리 코드와도 관계없이 항상 같은 상수 값이다.The binary noncorrelation condition of the binary code is

. As shown in FIG. 6, the diagonal elements are always the same constant value regardless of any binary code.

Therefore, it is important to approximate only non-diagonal elements, and the non-diagonal elements must satisfy Equation (15).

Equation (15) can not be differentiated due to the sign function, so that Equation (16) is devised so that the sign function is removed and the hash function is sequentially obtained by adding the penalty term to the objective expression of Equation (14).

조건을 추가하면 다음의 수학식 17의 가장 큰 고유 벡터를 구하는 것으로 수학식 16을 해결할 수 있다.In Equation 16,

By adding the condition, equation (16) can be solved by finding the largest eigenvector of the following equation (17).

Thus, the sequential binary code learning algorithm consists of finding the largest eigenvector of equation (17) to learn the i-th hash function.

를 필요로 하므로 단계 233을 통해 생성된

를 이용한다.The sequential binary code learning algorithm described in the above process is an integrated similarity matrix

Quot ;, so that the < RTI ID = 0.0 >

.

Therefore, according to the sequential binary code learning method using a plurality of features according to an exemplary embodiment of the present invention, it is possible to provide a high accuracy in searching an image composed of a plurality of features, have.

FIG. 7 is a block diagram illustrating a configuration of a sequential binary code learning apparatus using a plurality of features according to an embodiment of the present invention.

Referring to FIG. 7, the sequential binary code learning apparatus 100 according to an embodiment of the present invention may include an image acquisition unit 110, a processing unit 120, and a storage unit 130.

First, the image obtaining unit 110 provides the obtained query image to the processing unit 120 when the query image is obtained.

The processing unit 120 extracts a plurality of features from the query image provided from the image obtaining unit 110. [ Here, a plurality of features may include various image features such as GIST and HOG.

In addition, the processing unit 120 learns a multiple view hashing algorithm using the extracted features.

Specifically, the processing unit 120 obtains a distance matrix from the extracted features and generates an alpha- integrated similarity matrix using the alpha- conjugation method on the obtained distance matrix. In addition, the processing unit 120 sequentially learns the binary codes using the generated alpha- integrated similarity matrix.

Here, the alpha combining method for generating the alpha integrated similar matrices and the process of sequentially learning the binary codes have been described above, so that redundant explanations are omitted for easy understanding of the invention.

The processing unit 120 generates a binary code of a query image using the learned multi-view hashing algorithm and searches for an image having a binary code corresponding to the binary code generated in the storage unit 130. [

The storage unit 130 may be constituted by a large-capacity non-volatile storage device (for example, a hard disk drive), and a plurality of images having a binary code may be stored.

Here, the storage unit 130 may further update the image every time the image is retrieved, under the control of the processing unit 120. [ Also, the storage unit 130 may store various information set by the user such as an operating system for the operation of the sequential binary code learning apparatus 100 and an application program necessary for executing various functions.

Therefore, according to the sequential binary code learning apparatus using a plurality of features according to an embodiment of the present invention, it is possible to provide a high accuracy in searching an image composed of a plurality of features, have.

FIG. 8 shows an image search experiment using a sequential binary code learning method using a plurality of features according to an embodiment of the present invention, and an image search experiment result using a conventional method.

Referring to FIG. 8, an algorithm according to an embodiment of the present invention (Sequential Updated for Multi-View Spectral Hashing (SU-MVSH) and CHMIS, MVH-CS, MVH- The comparison was carried out using Caltech-356, CIFAR-20 and NUS-WIDE data, and 1000 query images were randomly extracted from the data. Then, each of the 500 query images (100 for Caltech-256 ) Is retrieved from the storage unit (130).

The performance evaluation was performed by comparing the accuracy and calculating the accuracy of the image with the same label as the query image in the retrieved image.

As shown in FIG. 8, the algorithm (SU-MVSH) according to an embodiment of the present invention provides the highest performance.

It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined in the appended claims. It will be possible.

100: Sequential binary code learning apparatus 110: Image acquisition unit
120: processing unit 130:

Claims (8)

A method for performing in a sequential binary code learning apparatus,
Extracting features from the acquired query image;
Learning a multi view hashing algorithm using the extracted features; And
And searching for an image having a binary code corresponding to the binary code generated through the learned multi-view algorithm. The method according to claim 1,
The step of learning a multiple view hashing algorithm using the extracted feature may include:
Obtaining a distance matrix from the extracted feature, and generating an alpha- integrated similarity matrix using the alpha- conjugation method on the obtained distance matrix; And
And sequentially learning the binary codes using the generated alpha- integrated similar matrices. ≪ Desc / Clms Page number 19 > The method of claim 2,
The step of obtaining a distance matrix from the extracted feature and generating an alpha- integrated similarity matrix using the alpha-
Dividing the extracted feature into two subsets through principal component analysis;
Dividing the extracted features until a predetermined number of data is obtained using the two subsets;
Generating a k nearest neighbor graph by performing alpha combining on a distance matrix corresponding to a view of each of the data divided by the predetermined number; And
And combining the k nearest neighbor graphs generated in the two subsets into one k nearest neighbor graph to generate a k nearest neighbor graph. The method of claim 2,
The step of sequentially learning the binary codes using the generated alpha-
And sequentially approximating non-diagonal elements to learn binary codes sequentially. A storage unit;
An image obtaining unit obtaining a query image and providing a query image obtained; And
Extracts a feature from the query image provided from the image acquisition unit, learns a multi view hashing algorithm using the extracted feature, and then generates a view corresponding to the binary code generated through the learned multi view algorithm And a processing unit for searching the storage unit for an image having a binary code. The method of claim 5,
Wherein,
A distance matrix is obtained from the extracted features, an alpha- integrated similarity matrix is generated using the alpha- conjugation method on the obtained distance matrix, and then the binary code is sequentially learned using the generated alpha- Wherein the binary code is a binary code. The method of claim 6,
Wherein,
Dividing the extracted feature into two subsets through principal component analysis, dividing the extracted features until a predetermined number of pieces of data are obtained using the two subsets, The k nearest neighbor graph is generated by performing alpha combining on the distance matrix corresponding to the view of the data, and the k nearest neighbor graph generated from the two subset is combined into one k nearest neighbor graph to form a k nearest neighbor graph. And generating a plurality of binary code sequences based on the plurality of features. The method of claim 6,
Wherein,
And the binary code is sequentially learned by approximating the non-diagonal elements.

Images