WO2013053320A1

WO2013053320A1 - Image retrieval method and device

Info

Publication number: WO2013053320A1
Application number: PCT/CN2012/082746
Authority: WO
Inventors: 田卉
Original assignee: 中国移动通信集团公司
Priority date: 2011-10-13
Filing date: 2012-10-11
Publication date: 2013-04-18
Also published as: CN103049446B; CN103049446A

Abstract

An image retrieval method and device. The method includes: performing sub-image division on an image to be retrieved to obtain a plurality of sub-images; performing image feature extraction on each designated sub-image in the plurality of sub-images to obtain a feature vector of each designated image; for each image in an image library, determining the similarity between the image and the image to be retrieved based on the feature vector of each sub-image in each sub-image group to be matched of the image and the feature vector of each designated sub-image; and determining an image retrieval result corresponding to the image to be retrieved based on the similarity between each image in the image library and the image to be retrieved. The solution provided by the embodiments of the present invention is employed to improve the effectiveness and retrieval efficiency of the retrieval result when performing retrieval based on image content.

Description

Image retrieval method and device

The present application claims priority to Chinese Patent Application No. 201110309996.0, entitled "An Image Retrieval Method and Apparatus", filed on October 13, 2011, the entire contents of . Technical field

The present invention relates to the field of image retrieval technologies in the field of image processing technologies, and in particular, to an image retrieval method and apparatus. Background technique

With the rapid development of multimedia technology, images are widely used as rich and expressive visual information. Due to the large amount of image information and low level of abstraction, how to quickly and efficiently retrieve the image resources required by users from massive image data has become a new challenge in the field of image processing technology.

In the existing image retrieval technology, most of the text-based image retrieval (text-based image retrieval), the technical idea is to manually text the image, such as keywords, titles and some additional description information, and then文本 Use the traditional database management system method for text retrieval.

With the advent of large-scale digital image libraries, the problem of text-based graphs has become more serious. For example, for large-scale image libraries, manual text annotation is a tedious and time-consuming task; and, manual annotation has certain Subjectivity, different people can have different understandings of image content, even if the same person has different understanding of images in different contexts, so the text annotation of an image is influenced by personal interest and knowledge background, subjectivity Strong; Moreover, the amount of image content is large, and many images are difficult to describe accurately by means of text; and, cultural differences around the world are large, and images marked with different languages are limited in application versatility. Therefore, text-based retrieval results in poor validity of search results, and it is difficult to accurately return images that the user desires. At present, in order to overcome the inefficient manual annotation and ambiguity, Content-Based Image Retrieval (CBIR) has emerged. The technical idea is to use the visual features of the image to retrieve and directly analyze the image content. Image features and semantics are extracted and indexed for retrieval. Feature extraction and matching can be done automatically by the machine.

Content-based maps The search technology combines image processing, pattern recognition, computer vision, image understanding, database management, human-computer interaction and other technologies. It is a fusion of multiple technologies, with a wide range of applications, and has been rapidly developed.

However, the existing image-based image retrieval technology focuses on the global feature of the image for retrieval, and describes the image as an inseparable whole. The color, texture, shape and other features are used to describe the entire image without distinguishing the image. Prospects and backgrounds, and in many cases retrieval based on a single feature.

Therefore, when the meaning of the content expression of an image is large, such as the ability to express landscapes, buildings, and characters, the use of the above-described existing map search technique may result in the extracted features failing to accurately express the meaning of the image content, thereby The search results are less effective, which in turn makes the search efficiency less efficient.

Summary of the invention

The embodiment of the invention provides a method and a device for solving the problem, which are used to solve the problem of poor validity of the retrieval result and low retrieval efficiency when searching based on image content existing in the prior art.

The embodiment of the invention provides a method for drawing a cable, comprising:

Subdividing the image to be retrieved into subgraphs to obtain a plurality of sub-images;

Performing image feature extraction on each of the plurality of sub-images to obtain feature vectors of the specified sub-images;

Determining, for each image in the image library, a feature vector of each sub-image in each sub-image group to be matched of the image, and a feature vector of each of the specified sub-images, determining that the image is similar to the image to be retrieved Degree; wherein, the relative positions between the sub-images in the sub-image group to be matched, and the respective Specify the relative position between the sub-images to be the same;

An image retrieval result corresponding to the image to be retrieved is determined based on a magnitude of similarity of each image in the image library and the image to be retrieved.

An embodiment of the present invention further provides an image retrieval apparatus, including:

a dividing unit, configured to divide the image to be retrieved into sub-pictures to obtain a plurality of sub-images;

An extracting unit, configured to perform image feature extraction on each of the plurality of sub-images to obtain feature vectors of the specified sub-images;

a similarity determining unit, configured to determine, according to each feature in the image library, a feature vector of each sub image in each of the to-be-matched sub-image groups of the image, and a feature vector of each of the specified sub-images a similarity of the image to be retrieved; wherein a relative position between each sub-image in the sub-image group to be matched is the same as a relative position between the specified sub-images;

The search result determining unit is configured to determine a graph result corresponding to the image to be retrieved based on a magnitude of similarity between each image in the image library and the image to be retrieved.

Advantageous effects of the invention include:

In the method provided by the embodiment of the present invention, the image to be retrieved and the image in the image library are divided into subgraphs, and image features are extracted from each of the plurality of subimages of the image to be retrieved, and the specified subimages are obtained. a feature vector, and when performing image retrieval, for each image in the image library, each sub-image group to be matched is determined in the image, and the relative positions between the sub-images in the sub-image group to be matched are compared with The relative positions between the specified sub-images are the same, and then the image is compared with the feature to be retrieved based on the feature vector of each sub-image in each sub-image group to be matched of the image and the feature vector of each specified sub-image. And determining a map search result corresponding to the image to be retrieved based on the magnitude of the similarity between each image in the image library and the image to be retrieved. Since the image is performed for each specified sub-image of the image to be retrieved and for each sub-image in each sub-image group to be matched for each image in the image library, it is compared with the image-based image in the technology. The global feature is searched, and the pertinence is stronger, and the meaning of the expression and the specified sub-images of the image to be retrieved can be effectively retrieved. The expressions match the images, which improves the validity and retrieval efficiency of the search results.

DRAWINGS

FIG. 1 is a flowchart of a method for searching for a picture according to an embodiment of the present invention;

2 is a flowchart of a method for searching for a map provided in Embodiment 1 of the present invention;

3 is a schematic diagram of a sub-picture division and an index window in Embodiment 1 of the present invention;

4 is a schematic diagram of determining a mapping window based on an index window according to Embodiment 1 of the present invention; FIG. 5 is a flowchart of a method for searching for a map provided in Embodiment 2 of the present invention;

Fig. 6 is a schematic structural view of the H cable device provided in Embodiment 3 of the present invention.

detailed description

In order to provide an implementation scheme for improving the validity of the search result and the search efficiency when searching based on the image content, the embodiment of the present invention provides an image retrieval method and apparatus, and the preferred embodiment of the present invention is described below with reference to the accompanying drawings. It is to be understood that the preferred embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. And in the case of no conflict, the features in the embodiments and the embodiments of the present application can be combined with each other.

The embodiment of the present invention provides a method for drawing a cable, as shown in FIG. 1 , including:

Step S101: Perform sub-picture division on the image to be retrieved to obtain a plurality of sub-images.

Step S102: Perform image feature extraction on each of the plurality of sub-images to obtain feature vectors of the specified sub-images.

Step S103: determining, for each image in the image library, a similarity between the image and the image to be retrieved based on a feature vector of each sub-image in each sub-image group to be matched of the image, and a feature vector of each specified sub-image. Wherein, the relative positions between the sub-images in the sub-image group to be matched are the same as the relative positions between the specified sub-images.

Step S104: Based on the similarity between each image in the image library and the image to be retrieved, The image retrieval result corresponding to the retrieved image is determined.

Preferably, when the image feature extraction is performed in the foregoing method, the feature vector of each specified sub-image in each specified sub-image may be obtained, and the feature vector may further include multiple feature vectors, and each feature. The vector may also include a plurality of feature components.

Correspondingly, when determining the similarity between an image in the image library and the image to be retrieved in the foregoing method, the determining may be based on the weight of the sub-image corresponding to each specified sub-image in each specified sub-image, and may also be based on each The weights between the features corresponding to the feature vectors are determined, and may be determined based on the weights within the features corresponding to each feature component.

Preferably, on the basis of the above method, after the image retrieval result is displayed to the user, the user performs a correlation evaluation on each retrieval result image, and based on the user's evaluation result of the correlation of each retrieval result image. Adjusting the weight used in determining the similarity between the image in the image library and the image to be retrieved, obtaining the adjusted weight, and determining the latest similarity between the image in the image library and the image to be retrieved based on the adjusted weight, and based on The latest image similarity between the image in the image library and the image to be retrieved determines the latest image H corresponding to the image to be retrieved.

The method and apparatus provided by the present invention will be described in detail below with reference to the accompanying drawings. Example 1

FIG. 2 is a flowchart of a method for providing a cable according to Embodiment 1 of the present invention, which specifically includes the following steps:

Step S201: Acquire an image to be retrieved submitted by the user, and perform subgraph division on the image to be retrieved to obtain a plurality of sub-images.

Specifically, the sub-picture partitioning may use various methods in the prior art to divide the image to be retrieved into multiple sub-graphs of a rectangle of the same size, for example, as shown in FIG. 3, 16 sub-pictures divided into 4×4. . Among them, multiple subgraphs are the same size and rectangular, which is convenient for subsequent similarity calculation, and is not a strict division condition.

In the embodiment of the present invention, the pseudo-quad tree partitioning method is used to perform sub-picture partitioning, which is as follows: In the first level division, the image is divided into mxn subgraphs of rectangles of the same size, wherein the maximum common divisor of the width and height of the image is first determined, m is the width of the image divided by the greatest common divisor, and n is the image. Divided by the greatest common divisor;

When performing the second level division, for each sub-picture in the mxn sub-pictures, it is divided into 4 sub-pictures of 2 X 2, thereby dividing the image into rectangles of the same size (2*m) (2*) n) Subgraphs; and so on, the image can be divided into (2 ^k *m) (2 ^k *n) subgraphs of the same size rectangle, where k is the number of divisions minus one.

Step S202: Select each designated sub-image from the plurality of sub-images of the image to be retrieved, and the selected designated sub-images constitute a designated sub-image group.

The specified sub-image can be automatically selected by the device according to a certain selection strategy.

Preferably, the plurality of sub-images can be displayed to the user, and the user selects the specified sub-image according to the retrieval requirement of the user. The designated sub-image selected by the user is equivalent to the user's region of interest, that is, the user performs the current image. The purpose of the search is to retrieve an image that matches the meaning of the region of interest. Therefore, the selection of each specified sub-image by the user can improve the validity of the search result.

For example, as shown in Fig. 3, the user selects the designated sub-images 1-5 among the 16 sub-images.

In order to facilitate subsequent determination of each sub-image in the image group to be matched in the image library, in this step, after selecting each of the designated sub-images, an index window including each specified sub-image may also be determined, and the index window is The sub-images of the plurality of sub-images are composed, and include the smallest rectangular window of each of the designated sub-images. For example, as shown in FIG. 3, the index window includes 9 sub-images of 3 x 3 at the upper left of the image to be retrieved.

Step S203: Perform image feature extraction on each of the plurality of sub-images of the image to be retrieved to obtain feature vectors of the designated sub-images.

Specifically, feature vectors of each of the designated sub-images in each of the designated sub-images may be extracted.

Further, the feature vector of the specified sub-image may include a plurality of feature vectors.

Further, each feature vector may include a plurality of feature components. The following is a detailed description of the extraction of three feature vectors of color, texture and shape of an image.

1. Extraction of color feature vectors:

颜色 Using the color moment method, the color distribution is represented by the central moments of the image pixels, and the color information is mostly concentrated on the lower moments, including the first moment is the mean, the second moment is the variance and the third moment is the raid. Used to represent the overall color distribution of an image, where:

确定 Determine the first-order moment mean using the following formula: / = ^∑ ; where / is the first-order moment mean, _Α is the color component value of the ith pixel in the image, and N is the number of pixels included in the image.

确定 Determine the second-order moment variance using the following formula:

1 ^Ν

σ = where _σ is the second-order moment variance. Determine the third-order moment with the following formula:

1 w

S = , where s is the third-order moment. Since each pixel has 3 color components, including a red component, a green component, and a blue component, the corresponding first-order moment mean, second-order moment variance, and third-order moment aura can be determined for each color component. Therefore, the color feature vector of the image proposed in the embodiment of the present invention includes nine color feature components, which respectively correspond to: a red first moment mean value, a red second order moment variance ^, and a red third order moment rake _Sr , and a green first order. the mean torque / _g, and the green second moment variance ^ Rao third moment of green, blue and first order moment mean / _έ, the variance _σ έ second moment blue and blue Rao third moment of _¾.

2. Extraction of texture feature vectors:

Using the gray level co-occurrence matrix method, the corresponding texture features are extracted by using its statistics.

First convert the color image into a grayscale image, and then extract 0 degrees, 90 degrees, 45 degrees, respectively. Gray level co-occurrence matrix of 135 degrees in four directions for the elements in the gray level co-occurrence matrix in each direction, indicating the number of times the pixel points of the direction are 2 and the pair of pixels, wherein the range of gray levels It is 0-256.

For each direction, the gray level co-occurrence matrix based on the direction determines the statistic of energy, entropy, contrast and correlation corresponding to the direction:

确定 Determine the energy texture feature statistic using the following formula:

Among them, is the energy texture feature statistic.确定 Determine the entropy texture feature statistic using the following formula:

ENT = H m _h m _h where ENr is the entropy texture feature statistic; 确定 Determine the contrast texture feature statistic using the following formula:

Where CON is the contrast texture feature statistic; 确定 use the following formula to determine the relevant texture feature statistic:

COR = where CO ? is the associated texture feature statistic,

The mean value of each element of the hth line in the gray level co-occurrence matrix, A is the mean value of each element of the kth column in the gray level co-occurrence matrix, and ^ is the standard deviation of each element of the hth line in the gray level co-occurrence matrix, which is in the gray level co-occurrence matrix The standard deviation of each element of column k.

For each of the four texture feature statistics, the mean and standard deviation of the four texture feature statistics corresponding to the four directions are determined as the texture feature components included in the texture feature vector of the extracted image. A total of 8 texture feature components are included, which are: energy mean texture feature component / _SM , energy standard deviation texture feature component σ _5Μ , entropy mean texture feature component / _£OT , entropy standard deviation texture feature component ^7 , contrast mean texture Feature component / _ecw , contrast standard deviation texture feature component iJc^, correlation mean texture feature component / and associated standard deviation texture feature component σ ₍ 3, extraction of shape feature vector: 提取 Use the shape invariant moment method to extract the shape feature vector of the image.

Specifically, the Canny operator is used to extract the edge information of each pixel in the image / (xj), f{x, y), and the Canny operator is used to extract the edge information of the pixel with the coordinate (x, y). And determine the two-dimensional p+q moment of the image and the normalized central moment with translation invariance, where:

确定 Determine the two-dimensional p+q moment by the following formula:

^ _Pq = ∑∑ ^ ^p y ^q f(x, y) ; where is a two-dimensional p + q moment.

y

确定 Determine the normalized center moment with translation invariance using the following formula:

P _q =∑∑(x - x ₀ Y (y - y ₀ ) ^q f(x,y) ; where ^ is the normalized y with translation invariance

The center moment, (x., j.) is the coordinates of the center of gravity of the image.

Based on "^ and ^, the various second-order and third-order central moments are combined to obtain seven invariant moments which are independent of translation, rotation and scale, and the seven invariant 钜 are taken as the extracted image. The shape feature component of the shape feature vector, 7 invariant, can be calculated by the method in the prior art, and is not described in the JH.

Step S204: Determine, for each image in the image library, each to-be-matched sub-image group of the image, where a relative position between each sub-image in the sub-image group to be matched, and each specified sub-image of the image to be retrieved The relative positions are the same, that is, each group of sub-images is determined in the image, and each group of sub-images is satisfied, the relative position between the sub-images included, and the specified sub-images of the image to be retrieved The relative position is the same.

In the embodiment of the present invention, sub-picture division is performed in advance for each image in the image library, and specifically, the same method of sub-picture division of the image to be retrieved may be performed, and sub-picture division is performed on each image in the image library. The number of the obtained sub-images may be equal to the number of the plurality of sub-images obtained after the image to be retrieved is divided, or may be larger than the number of the plurality of sub-images obtained by dividing the image to be retrieved.

Based on the index window including each specified sub-image determined in the above step S202, specifically, each sub-image group to be matched of one image in the image library may be determined as follows: First, each mapping window is determined in the image. The mapping window satisfies the following conditions: The number and arrangement of the sub-images included in the mapping window are the same as the number and arrangement of the specified sub-images in the index window, for example, as shown in FIG. The index window includes 9 sub-images of 3 x 3 at the upper left of the image to be retrieved, that is, the number of designated sub-images included in the index window is 9, and the arrangement is 3 x 3, and the division is 4 X In the image of the 16 sub-images of 4, four mapping windows satisfying the condition can be determined, which are the upper left, lower left, upper right, and lower right mapping windows including 9 sub-images of 3 x 3. Since the number and arrangement of the sub-images included in the index window and the mapping window are the same, there is a one-to-one correspondence between the sub-images included in the index window and each sub-image included in each mapping window.

Then, a sub-image corresponding to each specified sub-image position included in the index window among the sub-images included in each mapping window is determined to constitute a sub-image group to be matched corresponding to the mapping window.

Step S205: determining, for each image in the image library, each of the image vectors based on the feature vectors of each of the sub-images in the to-be-matched sub-image group of the image, and the feature vectors of the specified sub-images of the image to be retrieved. The similarity between the sub-image group to be matched and the specified sub-image group. The feature vector of each sub-image of each image in the image library may be pre-extracted and stored, and may be directly acquired during use.

Specifically, the similarity between a sub-image group to be matched and a specified sub-image group is determined by the following formula:

S{q, _Pl )= VS{ _Zi ) , where is the similarity of the selected sub-image group of the image P in the image library and the specified sub-image group of the image to be retrieved, ^(ζ,.) The similarity of the i-th sub-image corresponding to the position of the i-th sub-image group to be matched is the sub-image weight corresponding to the i-th designated sub-image, and N is the sub-image included in each designated sub-image. The number of images.

Specifically, the above ^ζ,.) can be determined by the following formula:

^■) =∑^( ) ' where is the i-th sub-image and position j in the sub-image group to be matched

The similarity of the jth feature vector of the corresponding i-th specified sub-image, W _2J is the j-th feature vector pair The inter-feature weight, M is the number of types of feature vectors of the extracted i-th sub-image. For example, when the extracted feature vector includes three feature vectors of color, texture, and shape, M takes a value of 3.

Specifically, the above formula can be determined by the following formula for the sum of the weighted Euclidean distances: '

O is the similarity of the kth feature component of the jth feature vector of the i-th specified sub-image corresponding to the position of the i-th sub-image in the sub-image group to be matched, and is the k-th feature of the j-th feature vector The intra-feature weight corresponding to the component, H is the number of feature components included in the extracted j-th feature vector. For example, the number of feature components included in the color feature vector is nine, and the number of feature components included in the texture feature vector is Eighty, the shape feature vector includes the number of feature components of seven.

Specifically, the above ^) can be determined by the following formula: _r≠ , _Q —r _≠ , _P

O - , where ^ is the kth feature component of the jth feature vector of the i-th specified sub-image of the image to be retrieved q, which is the i-th sub-group of the sub-image group to be matched of the image P in the image library The kth feature component of the jth feature vector of the image.

Step S206: For each image in the image library, after determining the similarity between each sub-image group to be matched and the specified sub-image group of the image to be retrieved, the average value of each similarity is taken as the image and the The similarity of the images is retrieved, or, preferably, the maximum of the similarities may be used as the similarity between the image and the image to be retrieved.

Step S207: Determine an image retrieval result corresponding to the image to be retrieved based on a magnitude of similarity between each image in the image library and the image to be retrieved.

Specifically, the image in the image library whose degree of similarity with the image to be retrieved is greater than the set similarity threshold may be determined as the result of the image corresponding to the image to be retrieved.

Specifically, it may be determined that the image of the image set in the order of the similarity of the image to be retrieved from the highest to the lowest in the image library is determined as the result of the image corresponding to the image to be retrieved.

The above method of searching for the image provided by the first embodiment of the present invention is for each finger of the image to be retrieved. The stator image is performed for each sub-image in each sub-image group to be matched for each image in the image library, so the retrieval is more targeted and more efficient than the global image-based feature in the technology. The meaning of the expression matches the meaning expressed by each specified sub-image of the image to be retrieved, thereby improving the validity of the retrieval result and the retrieval efficiency.

Example 2:

In the second embodiment of the present invention, based on the method in the first embodiment, the correlation evaluation result based on the user's image for each search result is proposed, in order to more accurately return the result of the image required by the user to the user. The weights used in determining the similarity are adjusted, and based on the adjusted weights, the latest similarity between the images is determined to determine the latest image retrieval results. As shown in FIG. 5, the following steps are specifically included:

Step S501, displaying each search result image in the image search result determined in the above embodiment 1 to the user, and providing the user with a relevance evaluation option of the search result image for each search result image, so that the user can perform the search. The resulting image was correlated (3).

Step S502: Acquire a correlation evaluation result of each search result image in the search result of the user, and the correlation evaluation result may specifically include three types, respectively: a related result related to the representation, and a non-judgment of the evaluation

Step S503: Adjust the weights used when determining the similarity between each image in the image library and the image to be retrieved based on the obtained correlation evaluation result of each search result image, and obtain the adjusted weight.

For the three weights used in the above embodiment 1, the corresponding adjusted weights are determined in the following manner:

1. Specify the sub-image weight corresponding to the sub-image:

Step A: Determine the positive example image in which the correlation evaluation result in the figure is the correlation result; for example, the number of images included in the figure search result is T, and the number of positive example images is Tl.

Step Β: Determine the candidate to be matched for the sub-image group to be matched with the highest similarity in each positive image. The similarity of each pair of sub-images corresponding to the positions in the specified sub-image group in the group of game images, since the similarity has been determined in the above embodiment 1, the saved similarity can be directly obtained at this time. In order to facilitate the determination of the subsequent adjusted sub-image weights, the similarity determined for each positive example image and each specified sub-image may be composed into a matrix (z); j _nxjV , where, the matrix

RixN-order matrix, N is the number of sub-images included in the specified sub-image group, and the elements in the matrix (4 is the position of the i-th specified sub-image and the t-th positive example image with the highest similarity among the sub-image groups to be matched The similarity of the corresponding sub-image.

Step C: determining, for each designated sub-image in the specified sub-image group, a reciprocal of the standard deviation of each similarity of the pair of sub-images corresponding to the specified sub-image position among the sub-image groups to be matched with the highest similarity . Wherein, the standard deviation is the above matrix (z); j _{nxjV 标准} the standard deviation of each column element σ, ., σ,. represents the standard deviation corresponding to the i-th specified sub-image, and calculates the reciprocal of the standard deviation.

Since the value of the standard deviation σ,. is smaller, it means that the specified sub-image is more valued by the user, so the reciprocal of the standard deviation σ,. reflects the size of the sub-image weight of the specified sub-image.

Step D: performing normalization processing on each reciprocal corresponding to each of the designated sub-images, and performing a normalized processing result corresponding to each of the designated sub-images as the adjusted sub-image corresponding to the designated sub-image The weight ^,. , can be determined by the following formula: w = \la \ ^, = ^, /∑^,; The adjusted sub-image weight corresponding to the i-th specified sub-image.

2. The weight between features corresponding to the feature vector:

Step a: for each of the plurality of feature vectors, based on the feature vector of each sub-image in each of the sub-image groups to be matched for each image in the image library, and the feature of each specified sub-image The vector determines a feature map search result corresponding to the image to be retrieved corresponding to the feature vector.

That is, in the foregoing embodiment 1, the plurality of feature vectors are used to determine the map search result corresponding to the search image, and in the step a, the map corresponding to the search image is determined based on a single feature vector. As a result, for the sake of distinguishing, the image retrieval result determined based on a single feature vector is referred to as a feature map search result, for example, a color map search result determined based on the color feature vector is determined based on the texture feature vector. The texture image retrieval result is referred to, and the shape feature vector is determined as a shape map search result. The specific determination method can adopt the same method as in the above Embodiment 1, and the difference is only based on a single one feature vector.

Step b: determining each of the search result images existing in the feature image search result and the image search result determined in the embodiment 1.

Step c: determining a first sum value of each score corresponding to the correlation evaluation result of each search result image that exists, and determining a second sum of the initial value of the weight between the first sum value and the feature of the feature vector The value is determined by the following formula:

w _2J = w _2J0 + MARK; wherein, _¾ .. The initial value of the inter-feature weight of the j-th feature vector, for example, the initial value of the inter-feature weight of each feature vector is set to 0; 4 ^ is the first sum value corresponding to the feature vector in the jth, The score corresponding to the correlation evaluation result may be set according to actual needs. For example, in this embodiment, the corresponding result corresponding value is set to 2, the non-evaluation result corresponding score is set to 1, and the uncorrelated result corresponding score is set. 0; is the above second sum value corresponding to the jth feature vector.

Step d: performing normalization processing on each second sum value corresponding to each of the plurality of feature vectors, and performing normalized processing on the result corresponding to each feature vector as the adjusted corresponding to the feature vector The weight between features ^ can be determined by the following formula:

M

W _{2 j} = w _{2 j} I w _{2 j} ; W _{2 j} is the adjusted inter-feature weight corresponding to the j-th feature vector, M is j

The number of types of feature vectors of the extracted sub-images.

3. Intra-feature weights corresponding to feature vectors:

Step 1: Determine the positive example image in which the correlation evaluation result in the image retrieval result is the correlation result; for example, the number of images included in the image retrieval result is T, and the number of the positive example images is T1. Step 2: For each feature vector to be matched in each positive example image, for each feature vector, each feature component included in the feature vector of each sub-image in the to-be-matched sub-image group is determined. In order to facilitate the determination of the sub-image weights after the subsequent adjustment, each of the determined feature components may be composed into a matrix [r^ _nxff , where the matrix [" _n ^ is a 7 XH-order matrix, and H is the extracted j-th feature vector including The number of feature components, the elements in the matrix are the k-th feature component of the j-th feature vector of the i-th sub-image in the to-be-matched sub-image group of the t-th positive example image.

Step 3: determining, for each sub-image and each feature component in the to-be-matched sub-image group, each of the feature components included in the feature vector of the sub-image in the to-be-matched sub-image group of each positive example image The standard deviation is the standard deviation corresponding to the feature component of each sub-image. Wherein, the standard deviation is the standard deviation of each of the elements in the matrix [" _n ^ ^3⁄4 , ^3⁄4 represents a standard corresponding to the i-th sub-image and corresponding to the k-th feature component of the j-th feature vector difference.

Step 4: normalizing, for each sub-image and each feature vector, a reciprocal of each standard deviation corresponding to the plurality of feature components included in the feature vector, and obtaining each sub-image and each feature component The corresponding processing result can be determined by the following formula: w _yk = \ l _yk - W _yk = w _≠ /∑w _vk is the processing result corresponding to the kth feature component of the jth feature vector of the i-th sub-image .

Step 5: Determine, for each feature component, an average value of the processing result corresponding to the feature component of each sub-image, and use the average value as the weight of the adjusted feature corresponding to the feature component, which may be determined by using the following formula: W _3Jk = ∑w _ijk / N ; W _3Jk is the adjusted intra-feature weight corresponding to the k-th eigen component of the j-th eigenvector.

Since the value of the standard deviation ^3⁄4 is smaller, the feature component representing the feature vector is more valued by the user, so the reciprocal of the standard deviation reflects the characteristics of the feature component of the feature vector. The size of the weight inside.

Step S504: Determine, according to the determined adjusted weight, the latest similarity between the image in the image library and the image to be retrieved.

In this step, preferably, only the latest similarity between each image in the image library except the above-mentioned positive example image and the image to be retrieved may be determined.

Step S505: Determine, according to the size of the latest similarity between the image in the image library and the image to be retrieved, the latest image corresponding to the image to be retrieved.

Preferably, the positive example image may be used as a part of the image included in the latest image search result. For the other images included in the latest image search result, the latest similarity between the image library and the image to be retrieved may be greater than An image of the similarity threshold is determined to be other images included in the latest map search result;

Specifically, it is also possible to determine, in the image library, a previously set number of images in the order of the latest similarity of the image to be retrieved from high to low, as other images included in the latest map search result.

The above-described method of Fig. 5 in Embodiment 2 of the present invention can be executed a plurality of times until the set number of loops is satisfied, or until the user instructs the currently determined latest image search result as the final image search result.

According to the image retrieval method provided by Embodiment 2 of the present invention, since the user evaluates the correlation result of the search result image, the weight is adjusted, and based on the adjusted weight, the latest image search result is determined, so that the determined latest image retrieval is performed. The result is closer to the user's needs, which further improves the effectiveness and efficiency of image retrieval.

Example 3:

Based on the same inventive concept, the method of the present invention is provided in accordance with the above-mentioned embodiments of the present invention. Accordingly, the third embodiment of the present invention further provides a drawing device. The structure of the present invention is as shown in FIG.

a dividing unit 601, configured to divide the image to be retrieved into sub-pictures to obtain a plurality of sub-images; The extracting unit 602 is configured to perform image feature extraction on each of the plurality of sub-images to obtain feature vectors of the specified sub-images;

The similarity determining unit 603 is configured to determine, according to each feature in the image library, a feature vector of each sub image in each to-be-matched sub-image group of the image, and a feature vector of each of the specified sub-images. a similarity with the image to be retrieved; wherein a relative position between each sub-image in the sub-image group to be matched is the same as a relative position between the specified sub-images;

The search result determining unit 604 is configured to determine a search result corresponding to the image to be retrieved based on a size of similarity between each image in the image library and the image to be retrieved.

Preferably, the similarity determining unit 603 is specifically configured to determine each of the image based on a feature vector of each sub image in each of the to-be-matched sub-image groups of the image, and a feature vector of each of the specified sub-images. a similarity of a sub-image group to be matched, a specified sub-image group composed of the specified sub-images; a maximum value or an average value of similarities between each sub-image group to be matched and the specified sub-image group The similarity of the image to the image to be retrieved.

Preferably, the extracting unit 602 is configured to perform image feature extraction on each of the plurality of sub-images to obtain a feature vector of each of the specified sub-images; the similarity determining unit 603 Specifically, the feature vector of each sub-image in the sub-image group to be matched, and the feature vector of each specified sub-image in the specified sub-image group, respectively determine the sub-image group to be matched and the designated sub- a similarity of each pair of sub-images corresponding to the positions in the image group; weighting and summing the similarities of each pair of sub-images based on the sub-image weights corresponding to each of the specified sub-images in the specified sub-image group, to obtain the Matching the similarity of the sub-image group with the specified sub-image group.

Preferably, the extracting unit 602 is configured to perform image feature extraction on each of the plurality of sub-images to obtain a plurality of feature vectors of each of the specified sub-images; The unit 603 is specifically configured to use, according to a plurality of feature vectors of one sub-image in the sub-image group to be matched, and a specified sub-picture corresponding to the sub-image position in the specified sub-image group. a plurality of feature vectors of the image, determining a similarity between the sub-image and each feature vector of the specified sub-image; and weighting and summing the similarities of each feature vector based on the inter-feature weights corresponding to each feature vector, The similarity of the sub-image to the specified sub-image.

Preferably, the extracting unit 602 is configured to perform image feature extraction on each of the plurality of sub-images, and obtain a plurality of feature vectors of each of the specified sub-images, each of the plurality of feature vectors. a plurality of feature components included in the feature vector;

The similarity determining unit 603 is specifically configured to determine the sub-image and the designated sub-image based on the plurality of feature components included in one feature vector of the sub-image and the plurality of feature components included in the feature vector of the specified sub-image The similarity of each feature component included in the feature vector of the image;

The sum of the weighted Euclidean distances of the similarities of the various feature components is determined based on the intra-feature weights corresponding to each feature component, and the similarity between the sub-image and a feature vector of the specified sub-image is obtained.

Preferably, the method further includes:

The obtaining unit 605 is configured to obtain a correlation evaluation result of the user for each search result image in the image retrieval result;

The weight adjustment unit 606 is configured to adjust, according to the obtained correlation evaluation result of each search result image, the weight used in determining the similarity between each image in the image library and the image to be retrieved, and obtain an adjustment Post weight

The similarity determining unit 603 is further configured to determine, according to the adjusted weight, an latest similarity between the image in the image library and the image to be retrieved;

The search result determining unit 604 is further configured to determine, according to the size of the latest similarity between the image in the image library and the image to be retrieved, the latest image retrieval result corresponding to the image to be retrieved.

Preferably, the correlation evaluation result obtained by the obtaining unit 605 includes: a correlation correlation result, and a weight adjustment unit 606, configured to determine, when the used weight is a sub-image weight, determine the correlation in the graph result. The results of the sexual evaluation are the positive examples of the relevant results; for each positive example a similarly-matched sub-image group in the image, determining a similarity of each pair of sub-images corresponding to positions in the specified sub-image group in the to-be-matched sub-image group; for each of the specified sub-image groups Specifying a sub-image, determining a reciprocal of a standard deviation of each similarity of the pair of sub-images corresponding to the specified sub-image position among the sub-image groups to be matched with the highest similarity; respectively corresponding to the designated sub-images Performing normalization processing for each reciprocal; and performing a normalized processing result corresponding to each specified sub-image as the adjusted sub-image weight corresponding to the designated sub-image;

When the used weight is an inter-feature weight, for each of the plurality of feature vectors, the feature vector of each sub-image in each sub-image group to be matched for each image in the image library is used. And the feature vector of the specified sub-images, determining a feature map search result corresponding to the image to be retrieved corresponding to the feature vector, determining the feature map search result and the image search result And determining, by each of the search result images, the first sum value of each score corresponding to the correlation evaluation result of each of the search result images, and determining the first sum value and the feature of the feature vector The second sum value of the initial value of the weight; the second sum value corresponding to each of the plurality of feature vectors is normalized; and the result corresponding to each feature vector obtained by normalizing is obtained as The weight of the adjusted features corresponding to the feature vector;

When the used weight is the intra-feature weight, determining the correlation evaluation result in the image retrieval result as each positive example image of the correlation result; for each of the positive example images having the highest similarity to be matched sub-image group, Determining, for each feature vector, each feature component included in the feature vector of each sub-image in the to-be-matched sub-image group; determining, for each sub-image and each feature component in the to-be-matched sub-image group, each positive component a standard deviation of each of the feature components included in the feature vector of the sub-image in the to-be-matched sub-image group of the example image as a standard deviation corresponding to the feature component of each sub-image; for each sub-image and Each feature vector, normalizing the reciprocal of each standard deviation corresponding to the plurality of feature components included in the feature vector, and obtaining a processing result corresponding to each sub-image and each feature component; The feature component determines an average value of the processing result corresponding to the feature component of each sub-image, and the average value is used as the adjusted corresponding to the feature component Inter-feature Heavy.

In summary, the solution provided by the embodiment of the present invention includes: dividing a to-be-searched image into sub-pictures to obtain a plurality of sub-images; and performing image feature extraction on each of the plurality of sub-images to obtain each of the designated sub-images. a feature vector of the image; and for each image in the image library, determining the image and the image to be retrieved based on the feature vector of each sub-image in each sub-image group to be matched of the image, and the feature vector of each specified sub-image The degree of similarity; and determining the result of the map corresponding to the image to be retrieved based on the magnitude of the similarity between each image in the image library and the image to be retrieved. With the solution provided by the embodiment of the present invention, when searching based on image content, the validity of the retrieval result and the retrieval efficiency are improved.

The spirit and scope of the Ming. Thus, it is intended that the present invention cover the modifications and variations of the inventions

Claims

Rights request

A method of drawing a chord, characterized in that it comprises:

Determining, for each image in the image library, a feature vector of each sub-image in each sub-image group to be matched of the image, and a feature vector of each of the specified sub-images, determining that the image is similar to the image to be retrieved a relative position between each sub-image in the sub-image group to be matched, and a relative position between the sub-images;

2. The method according to claim 1, wherein the image is determined based on a feature vector of each sub-image in each of the sub-image groups to be matched of the image, and a feature vector of each of the designated sub-images The similarity of the image to be retrieved specifically includes:

Determining, according to the feature vector of each sub-image in each sub-image group to be matched of the image, and the feature vector of each of the designated sub-images, determining each sub-image group to be matched of the image, and the designated sub-images The similarity of the specified sub-image groups composed;

A maximum value or an average value of similarities between each of the sub-image groups to be matched and the specified sub-image group is taken as the similarity between the image and the image to be retrieved.

The method of claim 2, wherein image feature extraction is performed on each of the plurality of sub-images to obtain a feature vector of each of the specified sub-images, specifically: Image feature extraction is performed on each designated sub-image in the sub-images, and feature vectors of each of the designated sub-images in the specified sub-images are obtained;

Correspondingly, determining, according to the feature vector of each sub-image in the sub-image group to be matched, and the feature vector of each specified sub-image, the similarity between the sub-image group to be matched and the specified sub-image group, Body includes:

Determining a position in the to-be-matched sub-image group and the specified sub-image group respectively based on a feature vector of each sub-image in the sub-image group to be matched, and a feature vector of each of the specified sub-images in the specified sub-image group The similarity of each pair of sub-images corresponding to each other;

The similarity of each pair of sub-images is weighted and summed based on the sub-image weights corresponding to each of the specified sub-images in the specified sub-image group, to obtain the similarity between the sub-image group to be matched and the specified sub-image group.

The method according to claim 3, wherein image feature extraction is performed on each of the plurality of sub-images to obtain a feature vector of each of the specified sub-images, For:

Performing image feature extraction on each of the plurality of sub-images to obtain a plurality of feature vectors of each of the designated sub-images;

Correspondingly, determining the similarity of the pair of sub-images corresponding to the positions in the specified sub-image group in the sub-image group to be matched, specifically includes:

Determining the sub-image and the designation based on a plurality of feature vectors of one sub-image in the sub-image group to be matched, and a plurality of feature vectors of the specified sub-image corresponding to the sub-image position in the specified sub-image group The similarity of each feature vector of the sub-image;

Based on the inter-feature weights corresponding to each feature vector, the similarity of each feature vector is weighted and summed to obtain the similarity between the sub-image and the specified sub-image.

The method according to claim 4, wherein image feature extraction is performed on each of the plurality of sub-images, and a plurality of feature vectors of each of the specified sub-images are obtained. , Specifically:

Performing image feature extraction on each of the plurality of sub-images to obtain a plurality of feature components included in each of the plurality of feature vectors of each of the designated sub-images; Correspondingly, determining a similarity between the sub-image and a feature vector of the specified sub-image includes:

Determining, according to the plurality of feature components included in a feature vector of the sub-image, and the plurality of feature components included in the feature vector of the specified sub-image, determining each of the feature vectors included in the sub-image and the specified sub-image Similarity of feature components;

6. The method of any of claims 3-5, further comprising:

Obtaining a correlation evaluation result of the user for each of the search result images in the map search result; adjusting each image and the image in the image library based on the obtained correlation evaluation result of each search result image Determining a weight used when retrieving the similarity of the image, obtaining an adjusted weight; determining an latest similarity between the image in the image library and the image to be retrieved based on the adjusted weight;

And determining, based on the magnitude of the latest similarity between the image in the image library and the image to be retrieved, the latest image of the image to be retrieved.

7. The method according to claim 6, wherein the correlation evaluation result comprises: the characterization is based on the obtained correlation evaluation result of each retrieval result image, and adjusting each image in the image library is determined The weight used in the similarity of the image to be retrieved specifically includes: determining, when the used weight is a sub-image weight, each positive example image in which the correlation evaluation result in the image retrieval result is a correlation result; Determining a similarity of each pair of sub-images corresponding to positions in the specified sub-image group in the sub-image group to be matched for each sub-image group to be matched with the highest degree of similarity in each positive example image; Determining, in each of the specified sub-images in the image group, a reciprocal of the standard deviation of each similarity of the pair of sub-images corresponding to the specified sub-image position in each sub-image group to be matched with the highest degree of similarity; Sub-images corresponding to each reciprocal, normalized; Normally processing the result corresponding to each specified sub-image obtained as the adjusted sub-image weight corresponding to the designated sub-image;

When the used weight is the intra-feature weight, determining the correlation evaluation result in the image retrieval result as each positive example image of the correlation result; for each of the positive example images having the highest similarity to be matched sub-image group, Determining, for each feature vector, each feature component included in the feature vector of each sub-image in the to-be-matched sub-image group; determining, for each sub-image and each feature component in the to-be-matched sub-image group, each positive component a standard deviation of each of the feature components included in the feature vector of the sub-image in the to-be-matched sub-image group of the example image as a standard deviation corresponding to the feature component of each sub-image; Each feature vector, normalizing the reciprocal of each standard deviation corresponding to the plurality of feature components included in the feature vector, and obtaining a processing result corresponding to each sub-image and each feature component; a feature component that determines an average value of processing results corresponding to the feature components of each sub-image, and uses the average value as the feature component corresponding to the feature component Wherein the weight between adjusted.

8. A figure choking device, comprising:

An extracting unit, configured to perform image feature extraction on each of the plurality of sub-images, Obtaining feature vectors of the specified sub-images;

9. The apparatus according to claim 8, wherein the similarity determining unit is specifically configured to use a feature vector of each sub-image in each sub-image group to be matched based on the image, and the specified a feature vector of the sub-image, determining a similarity of each sub-image group to be matched of the image, and a specified sub-image group composed of the specified sub-images;

The apparatus according to claim 9, wherein the extracting unit is configured to perform image feature extraction on each of the plurality of sub-images to obtain each of the specified sub-images. Specifying a feature vector of the sub-image;

The similarity determining unit is specifically configured to determine the to-be-matched sub-image group respectively based on a feature vector of each sub-image in the sub-image group to be matched, and a feature vector of each specified sub-image in the specified sub-image group a similarity of each pair of sub-images corresponding to positions in the specified sub-image group; weighting the similarity of each pair of sub-images based on sub-image weights corresponding to each of the designated sub-images in the specified sub-image group Summing, obtaining the similarity between the to-be-matched sub-image group and the specified sub-image group.

The apparatus according to claim 10, wherein the extracting unit is configured to perform image feature extraction on each of the plurality of sub-images to obtain each of the specified sub-images Specifying a plurality of feature vectors of the sub-image; The similarity determining unit is specifically configured to use, according to a plurality of feature vectors of one sub-image in the sub-image group to be matched, and a plurality of specified sub-images corresponding to the sub-image position in the specified sub-image group. a feature vector, determining a similarity between the sub-image and each feature vector of the specified sub-image; and weighting and summing the similarity of each feature vector based on the inter-feature weight corresponding to each feature vector, to obtain the sub-image and The similarity of the specified sub-image.

The apparatus according to claim 11, wherein the extracting unit is configured to perform image feature extraction on each of the plurality of sub-images to obtain each of the specified sub-images. Specifying a plurality of feature components included in each feature vector among the plurality of feature vectors of the sub-image; the similarity determining unit is specifically configured to use a plurality of feature components included in a feature vector of the sub-image, and the designation And determining, by the plurality of feature components of the feature vector of the sub-image, a similarity of each feature component included in the feature vector of the specified sub-image;

The apparatus according to any one of claims 10 to 12, further comprising: an obtaining unit, configured to acquire a correlation evaluation result of the user for each of the search result images in the map search result;

a weight adjustment unit, configured to adjust a weight used when determining a similarity between each image in the image library and the image to be retrieved based on the obtained correlation evaluation result of each search result image, and obtain an adjusted Weights;

And the similarity determining unit is further configured to determine, according to the adjusted weight, an latest similarity between the image in the image library and the image to be retrieved;

The search result determining unit is further configured to determine a latest image search result corresponding to the image to be retrieved based on a size of an latest similarity between the image in the image library and the image to be retrieved.

14. The apparatus according to claim 13, wherein the correlation evaluation result obtained by the obtaining unit comprises: characterization related related results, characterization irrelevant irrelevant results, and characterization not commenting Judging the result of non-judgment;

The weight adjustment unit is specifically configured to: when the used weight is a sub-image weight, determine a positive example image in which the correlation evaluation result in the image retrieval result is a correlation result; for each positive example image a similarly-matched sub-image group to be determined, determining a similarity of each pair of sub-images corresponding to positions in the specified sub-image group in the to-be-matched sub-image group; for each of the specified sub-image groups And an image, determining a reciprocal of a standard deviation of each similarity of the pair of sub-images corresponding to the specified sub-image position in each of the to-be-matched sub-image groups having the highest similarity; and corresponding reciprocals corresponding to the respective sub-images Performing normalization processing; the result corresponding to each specified sub-image obtained after the normalization processing is used as the weight of the adjusted sub-image corresponding to the specified sub-image;

When the used weight is the intra-feature weight, determining the correlation evaluation result in the image retrieval result as each positive example image of the correlation result; for each of the positive example images having the highest similarity to be matched sub-image group, Determining, for each feature vector, each feature component included in the feature vector of each sub-image in the to-be-matched sub-image group; determining, for each sub-image and each feature component in the to-be-matched sub-image group, each positive component a standard deviation of each of the feature components included in the feature vector of the sub-image in the to-be-matched sub-image group of the example image as a standard deviation corresponding to the feature component of each sub-image; Each feature vector, a plurality of feature components included in the feature vector Corresponding to the reciprocal of each standard deviation, performing normalization processing to obtain processing results corresponding to each sub-image and each feature component; for each feature component, determining a processing result corresponding to the feature component of each sub-image The average value is used as the adjusted inter-feature weight corresponding to the feature component.