WO2017107866A1

WO2017107866A1 - Image retrieval server and system, related retrieval and troubleshooting method

Info

Publication number: WO2017107866A1
Application number: PCT/CN2016/110356
Authority: WO
Inventors: 陈卓; 李薪宇
Original assignee: 成都理想境界科技有限公司
Priority date: 2015-12-22
Filing date: 2016-12-16
Publication date: 2017-06-29
Also published as: CN106909552A

Abstract

An image retrieval result troubleshooting method, an image retrieval method, an image retrieval server, and an image retrieval system. The image retrieval result troubleshooting method comprises: calculating an attitude relationship between a target image and each retrieval result image according to retrieval results and by means of size and direction information of a matching feature point pair set (S201); converting coordinates of a matching feature point set of the target image into coordinates in a coordinate system of the retrieval result image (S202);performing spatial sorting on feature points in the matching feature point set of the target image according to coordinates after the conversion of the coordinate system, and construct a Delaunay triangular network (S203); obtaining a Delaunay triangular network corresponding to the retrieval result image, and deleting an unmatching feature point subset, so as to obtain the corresponding Delaunay triangular network (S204); and comparing the two Delaunay triangular networks; if the comparison result is that the two Delaunay triangular networks are consistent, determining that the retrieval result is correct, and otherwise determining that the retrieval result is wrong (S205). The accuracy rate of the retrieval results can be effectively improved.

Description

Image retrieval server, system, related retrieval and troubleshooting method

The present application claims priority to Chinese Patent Application No. CN201510974626.7, filed on Dec. 22,,,,,,,,,,,,,,,,,,,,

Technical field

The present invention relates to the field of image retrieval technologies, and in particular, to an image retrieval result troubleshooting method, an image retrieval method, an image retrieval server, and an image retrieval system.

Background technique

In image retrieval based on feature extraction, the feature extraction is first performed on the image and the feature content is described, and the target image is characterized by the feature point set. At the time of retrieval, the retrieval result of the target image is determined based on the matching relationship between the pair of feature points. In extreme cases, there are only a few matching feature point pairs between the search result and the target image (the lowest condition is that the number of matching feature point pairs is greater than or equal to 3). At this time, due to the limitations of the characterization algorithm, the search result may be erroneous (ie, the target image and the search result are two images that are irrelevant). Of course, the search results may also be correct. For the above reasons, there is a need for a method of troubleshooting search results to ensure that the image retrieval results are as accurate as possible.

Summary of the invention

The object of the present invention is to provide an image retrieval result troubleshooting method, an image retrieval method, an image retrieval server and an image retrieval system, which solve the problem of too many retrieval result errors caused by the limitation of the feature description image algorithm, and can effectively improve image retrieval. accuracy.

In order to achieve the above object, the present invention provides an image retrieval result troubleshooting method. The image retrieval is an image retrieval based on feature extraction. When feature extraction is performed on a target image, the extracted feature data includes each feature point. Position information, scale, direction and feature description information in the image region; the image retrieval result refers to one or more sets of search result image data in the image retrieval database that meets the matching algorithm with the target image, and the image retrieval result debugging method is respectively used Troubleshoot each search result image, including:

According to the image retrieval result, the target image and each are calculated by matching the scale and direction information of the feature point pair set. An attitude relationship between the search result images, the matched feature point pair set includes a target image matching feature point set and a retrieval result image matching feature point set;

According to the calculated attitude relationship, the coordinates of the target image matching feature point set and the retrieval result image matching feature point set are converted into the same coordinate system, and the target image matching feature point set and the retrieval result are respectively performed in the converted coordinate system. The image matching feature point set is Delaunay triangulation, and the Delaunay triangulation network corresponding to the target image and the Delaunay triangulation network corresponding to the retrieval result image are obtained.

The two Delaunay triangulation networks are compared. If the alignment results of the two triangular networks are consistent, the image retrieval result is determined to be correct; otherwise, the image retrieval result is determined to be incorrect.

Preferably, the coordinates of the target image matching feature point set and the retrieval result image matching feature point set are converted into the same coordinate system, specifically: converting the coordinates of the target image matching feature point set into the retrieval result image coordinate system, Or convert the coordinates of the search result image matching feature point set into the target image coordinate system.

Preferably, the Delaunay triangulation is performed on the target image matching feature point set and the search result image matching feature point set respectively in the converted coordinate system, specifically: matching the feature image set and the retrieval result image feature to the target image respectively The feature points in the point set are spatially sorted according to the coordinates converted by the coordinate system, and the corresponding Delaunay triangle networks are constructed according to the sorting results.

Preferably, the spatial ordering is a median sorting, and specifically includes: a sorting axis determining step of using an axis having a maximum/minimum diameter on the x-axis and the y-axis as a sorting axis; and an updating step, in the On the sorting axis, calculate the median value of the two feature points constituting the maximum/minimum diameter, and change the original feature point set so that the feature point spatially located to the left of the median value is located to the left of the median point in the data set, and makes the space The feature point on the right side of the median is located to the right of the median point in the data set; the looping step repeats the sorting axis determining step and the point set on the point set formed by the left point and the right point The update step until the number of feature points on the median side is less than two.

Correspondingly, the present invention further provides an image retrieval result troubleshooting method, wherein the image retrieval is image retrieval based on feature extraction, and when feature extraction is performed on the target image, the extracted feature data includes each feature point in the image region. Location information, scale, direction, and feature description information; image retrieval result refers to one or more sets of search result image data in the image retrieval database that meets the matching algorithm with the target image, and the image retrieval result troubleshooting method is used for each Search results images for troubleshooting, including:

According to the image retrieval result, the attitude relationship between the target image and each of the search result images is calculated by using the scale and direction information of the matched feature point pair set, and the matched feature point pair set includes the target image matching feature point set and the search result image. Matching feature point sets;

According to the calculated attitude relationship, the target image is matched with the feature point set and the search result image is matched with the feature point set. The target is converted to the same coordinate system;

According to the position information of the search result image corresponding to the target image matching feature point matching the feature point in the search result image, the subset of the matched feature point pair set is divided, and a plurality of matching feature point pair subsets are obtained, and each matching feature is obtained. The point pair subset includes a target image matching feature point subset and a retrieval result image matching feature point subset;

In the transformed coordinate system, Delaunay triangulation is performed on each target image matching feature point subset and the search result image matching feature point subset respectively to obtain a corresponding Delaunay triangulation network;

Comparing the two Delaunay triangulation networks corresponding to the subset of matching feature points to the subset, if the matching feature point pair of the preset ratio exceeds the matching result of the two triangular networks, the image retrieval result is determined to be correct; otherwise It is determined that the image retrieval result is incorrect.

Preferably, Delaunay triangulation is performed on each subset, specifically: spatially sorting the coordinate points of the feature points in each subset according to the coordinate transformation, and constructing a Delaunay triangular network for each subset according to the sorting result;

The spatial ordering refers to median sorting, and specifically includes:

a sorting axis determining step of using an axis having a maximum/minimum diameter on the x-axis and the y-axis of the feature point set feature point as a sorting axis;

And an updating step of calculating a median value of two feature points constituting the maximum/minimum diameter on the sorting axis, and changing the original feature point set such that the feature point spatially located to the left of the median value is located at the median point in the data set On the left side, and make the feature points on the right side of the median value in the data set to the right of the median point;

In the looping step, the sorting axis determining step and the updating step are repeated for the point set formed by the left point and the point set formed by the right point until the number of feature points on the median side is less than 2.

Correspondingly, the present invention further provides an image retrieval method, comprising: acquiring a target image; performing feature extraction on the target image or the pre-processed target image, and extracting the feature data including the position of each feature point in the image region. Information, scale, direction and feature description information; the extracted feature data is sent to the image retrieval server for image retrieval, and one or more search result images that are initially matched with the target image are obtained; for the preliminary matching search result image, the foregoing An image retrieval result troubleshooting method performs troubleshooting of the retrieval results.

Correspondingly, the present invention also provides an image retrieval server, including an image retrieval database and an image matching module. The image retrieval database stores feature point set data of a plurality of sample images, and the feature point set data of each sample image includes position information, scale, direction and feature description information of each feature point in the feature point set in the image region; The image matching module is configured to receive a retrieval request from the image retrieval client, and match the target image data included in the retrieval request in the image retrieval database according to the set matching algorithm to obtain one or more groups and target images. The search result image data of the matching algorithm is satisfied, the image retrieval server further includes a debugging module, and the debugging module includes:

The attitude calculation and coordinate system conversion unit is configured to calculate the attitude relationship between the target image and each of the retrieval result images according to the image retrieval result, and use the matching feature point pair scale and direction information, and according to the calculated posture relationship, Converting the coordinates of the target image matching feature point set and the retrieval result image matching feature point set into the same coordinate system, the matching feature point pair set including the target image matching feature point set and the retrieval result image matching feature point set;

A triangular network construction unit is configured to perform Delaunay triangulation on the target image matching feature point set and the search result image matching feature point set respectively in the converted coordinate system, and obtain a Delaunay triangular network corresponding to the target image and the search result image corresponding to the image. Delaunay triangle network;

The determining unit is configured to compare the two Delaunay triangular networks, and if the two triangular network comparison results are consistent, the image retrieval result is determined to be correct; otherwise, the image retrieval result is determined to be incorrect.

Preferably, the Delaunay triangulation is performed on the target image matching feature point set and the search result image matching feature point set respectively in the converted coordinate system, specifically: matching the feature image set and the retrieval result image feature point respectively to the target image The concentrated feature points are spatially sorted according to the coordinates converted by the coordinate system, and the corresponding Delaunay triangular networks are constructed according to the sorting results.

Correspondingly, the present invention further provides an image retrieval server, comprising an image retrieval database and an image matching module, wherein the image retrieval database stores feature point set data of a plurality of sample images, and the feature point set data of each sample image includes features. Position information, scale, direction and feature description information of each feature point in the image area; the image matching module is configured to receive a retrieval request from the image retrieval client, and the retrieval request is according to the set matching algorithm The included target image data is matched in the image retrieval database, and one or more sets of search result image data satisfying the matching algorithm with the target image are obtained, and the image retrieval server further includes a debugging module, and the debugging module includes:

An attitude calculation and coordinate system conversion unit for using the scale of the matching feature point pair set according to the image retrieval result And the direction information, calculating the attitude relationship between the target image and each of the retrieval result images, and converting the coordinates of the target image matching feature point set and the retrieval result image matching feature point set into the same coordinate system according to the calculated posture relationship And the matching feature point pair set includes a target image matching feature point set and a retrieval result image matching feature point set;

The sub-division unit obtains a subset of the matched feature point pair set according to the position information of the search result image corresponding to the target image matching feature point matching the feature point in the search result image, and obtains a plurality of matching feature point pair subsets. Each of the matching feature point pair subsets includes a target image matching feature point subset and a retrieval result image matching feature point subset;

A triangular network building unit is configured to perform Delaunay triangulation on each target image matching feature point subset and the search result image matching feature point subset in the converted coordinate system to obtain a corresponding Delaunay triangular network;

The determining unit compares the two Delaunay triangulation networks corresponding to the subset of the matching feature points, and if the matching feature point pair of the preset ratio exceeds the matching result of the two triangular networks, the image retrieval result is determined. Correct; otherwise the image retrieval result is determined to be incorrect.

Preferably, the Delaunay triangulation is performed on each subset, specifically: spatially sorting the coordinate points of the feature points in each subset according to the coordinate transformation, and constructing a Delaunay triangular network for each subset according to the sorting result.

Correspondingly, the present invention further provides an image retrieval system, comprising an image retrieval client and an image retrieval server, wherein the image retrieval server is any one of the foregoing image retrieval servers;

The image retrieval client includes an image acquisition module, a feature extraction module, a retrieval request sending module, and a retrieval result receiving module, wherein:

The image acquisition module is configured to acquire a target image;

The feature extraction module is configured to perform feature extraction on the target image, and the extracted feature data includes location information, scale, direction, and feature description information of each feature point in the image region;

The search request sending module is configured to send the feature data extracted by the feature extraction module to an image retrieval server for image retrieval;

The search result receiving module is configured to receive the search result information returned from the image retrieval server.

The invention uses the Delaunay triangulation network to characterize the internal relationship of the image feature point set, and uses the unique property of the Delaunay triangulation network to debug (correct) the search result, and the algorithm is correct (the bottom line of the constraint condition is satisfied), but the human recognition Knowing that the search results that are determined to be erroneous are eliminated, can effectively improve the accuracy of the search results.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present invention, and other drawings can be obtained according to these drawings for those skilled in the art without any inventive labor:

1 is a schematic flow chart of a method for generating an image retrieval database according to an embodiment of the present invention;

2 is a schematic diagram of a feature point set in an embodiment of the present invention;

3 is a first schematic flowchart of a method for troubleshooting an image retrieval result according to an embodiment of the present invention;

4 is a second schematic flowchart of a method for troubleshooting an image retrieval result according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of locations of corresponding matching feature points in a search result image and a target image according to an embodiment of the present invention; FIG.

FIG. 6 is a schematic diagram of a first structure of an image retrieval server according to an embodiment of the present invention; FIG.

FIG. 7 is a schematic diagram showing a second structure of an image retrieval server according to an embodiment of the present invention; FIG.

FIG. 8 is a third schematic flowchart diagram of a method for troubleshooting an image retrieval result according to an embodiment of the present invention; FIG.

FIG. 9 is a fourth schematic flowchart of a method for troubleshooting an image retrieval result according to an embodiment of the present invention; FIG.

FIG. 10 is a third schematic structural diagram of an image retrieval server according to an embodiment of the present invention; FIG.

FIG. 11 is a fourth schematic structural diagram of an image retrieval server according to an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

The present invention uses a Delaunay triangulation network to characterize the internal relationship of image feature point sets, using the Delaunay triangle The uniqueness characteristics of the network are used to debug (correct) the search results, and the algorithm results are correct (the bottom line that satisfies the constraint), but the search results that are cognitively determined to be erroneous are rejected.

First, let's introduce the Delaunay triangulation network: The Delaunay triangulation network is a network formed by Delaunay triangulation of point sets. To meet the definition of Delaunay triangulation, two important criteria must be met:

1) Empty circle characteristics: The Delaunay triangular network is unique (any four points cannot be co-circular), and no other points exist in the circumscribed circle of any triangle in the Delaunay triangular network;

2) Maximizing the minimum angular characteristic: In the triangulation that the scatter set may form, the minimum angle of the triangle formed by the Delaunay triangulation is the largest. In this sense, the Delaunay triangle network is the "closest to the regularized" triangular network. Specifically, it means that the diagonal lines of the convex quadrilaterals are formed by two adjacent triangles, and after the mutual exchange, the minimum angles of the six inner corners are no longer increased.

The Delaunay Triangulation Network has the following features:

1) closest: a triangle is formed with the last three points, and each line segment (the sides of the triangle) does not intersect;

2) Uniqueness: no matter where the construction begins, the final result will be consistent;

3) Optimality: if the diagonals of the convex quadrilateral formed by any two adjacent triangles are interchangeable, then the smallest angle among the six inner corners of the two triangles does not become large;

4) The most rule: If the minimum angle of each triangle in the triangular network is sorted in ascending order, the Delaunay triangulation network has the largest value;

5) Regionality: adding, deleting, and moving a certain vertex will only affect adjacent triangles;

6) Shell with convex polygons: The outermost boundary of the triangular network forms a convex polygonal outer shell.

In the method for troubleshooting the retrieval result of the present invention, it is necessary to construct a Delaunay triangular network separately for the target image matching feature point set and the retrieval result image matching feature point set. Since the triangular network comparison is needed, when the Delaunay triangular network is established, the coordinates of the target image matching feature point and the search result image matching feature point need to be transformed into the same coordinate system. Then, in the converted coordinate system, the Delaunay triangulation is performed on the target image matching feature point set and the search result image matching feature point set respectively, and the Delaunay triangular network corresponding to the target image and the Delaunay triangular network corresponding to the retrieval result image are obtained. Finally, the correctness of the search results is judged by comparing the above two Delaunay triangulation networks.

In the present invention, the retrieval result image refers to a sample image that satisfies the matching algorithm. In one embodiment, the Delaunay triangulation network corresponding to the target image and the retrieval result image is generated online. In another implementation manner, the Delaunay triangulation network corresponding to each sample image may be generated offline when the image retrieval database is constructed. Improve the efficiency of image retrieval.

The following is a first embodiment of the present invention by taking the Delaunay triangulation network corresponding to the sample image offline as an example with reference to FIG. 1 to FIG. 7 .

1 is a schematic flowchart of a method for generating an image retrieval database according to an embodiment of the present invention. This method is used to construct a dedicated image retrieval database with Delaunay triangular network data corresponding to the sample image. The image retrieval database generating method includes the following steps S101 to S104.

In S101, feature extraction is performed on the sample image or the pre-processed sample image to obtain a feature point set corresponding to each sample image. In this step feature extraction method, a feature-based extraction method based on scale invariance, such as ORB, SIFT, SURF, etc., may be employed. The preprocessed sample image refers to a sample image preprocessed by one or more methods of uniform size processing, redundant area culling, Gaussian blur processing, and affine transformation, and the preprocessing is to improve retrieval precision.

In S102, the position information, the scale, the direction and the feature description information of each feature point in the feature point set corresponding to each sample image in the image region are recorded in the search data corresponding to the sample image in the image retrieval database. . After feature extraction, the order in which feature point data is recorded into the image retrieval database is arbitrary. The characterization information can be an 8-byte content description. The feature point position information can be represented by two-dimensional coordinates.

In S103, according to the position information of each feature point in the image region in the feature point corresponding to each sample image, the feature points in the feature point set corresponding to the sample image are spatially sorted. Feature points that are closer in spatial position are still closer together after sorting.

In S104: constructing a Delaunay triangular network corresponding to the sample image according to the spatial ordering result of each sample image obtained in S103 (the triangular network is unique), that is, constructing the same point set, no matter which point is started A consistent result is obtained; at the same time, the same subset of the point set is deleted, and the resulting triangular network is also consistent). Then, the triangular sequence data of the constructed Delaunay triangulation network is recorded in the retrieval data corresponding to the sample image in the image retrieval database. The triangular sequence data includes the sequence numbers of three points and three sides corresponding to each triangle in the Delaunay triangular network.

The spatial ordering in step S103 may be a median ordering. The median ranking refers to performing median ordering based on location information of feature points within the image region. Specifically, the axis having the feature points in the feature points on the x-axis and the y-axis has the largest/minimum diameter as the sorting axis. Calculating a median value of two feature points constituting the maximum/minimum diameter on the sorting axis, and changing a set of original feature points such that a feature point spatially located to the left of the median value is located at a median point in the data set Side, and feature points that are spatially to the right of the median are located to the right of the median point in the data set. Then, the above-mentioned recursive processing is performed on the point set formed by the left point and the point set formed by the right point until the number of feature points on the median side is less than 2 stop. The x-axis diameter refers to the absolute value of the difference between the maximum value and the minimum value of the x-coordinates of each feature point in the feature point set; the y-axis diameter refers to the difference between the maximum value and the minimum value of the y-coordinate of each feature point in the feature point set. The absolute value.

See Figure 2 for a set of points, including the following seven points: [(-2,2)(2.5,-5)(2,1)(-4,-1.5)(-7.5,2.5)(7,2 ) (1,-2.5)]. The set of 7 points consists of a x-axis diameter of 14 and a y-axis diameter of 7.5. Assume that the median sorting uses the larger of the x and y-axis circumference diameters as the sorting axis. In the first sorting, the x-axis is used as the sorting axis, and the median is 0. Three points (-7.5, 2.5), (-2, 2), (-4, -1.5) are placed to the left of the median point, and the other four points are placed to the right of the median point. Then, the left point set and the right point set are recursively processed, that is, the left and right side point sets are re-searched for the larger diameter axis in the xy axis, and the median values of the two feature points constituting the diameter are calculated, and the original feature point set is changed. So that the feature points that are spatially to the left of the median are located to the left of the median point in the data set, and the feature points that are spatially to the right of the median are located to the right of the median point in the data set.

In the image retrieval database generation method of the embodiment, a large number of sample images may be processed on the server side to generate a corresponding image retrieval database, and new sample image data may be added to the existing image separately or in groups in the added mode. Retrieve the database. For different application requirements, in order to improve the retrieval accuracy, the sample image can be preprocessed in various ways, including uniform size processing, redundant area culling, Gaussian blur processing, affine transformation and so on.

According to the image retrieval database generation method described above, a dedicated image retrieval database used in the embodiment of the present invention can be generated. The image retrieval database is stored on the image retrieval server side. The image retrieval database stores retrieval data of a plurality of sample images. The retrieval data of each sample image includes feature point set data of the sample image and triangular sequence data of a Delaunay triangular network constructed from the feature point set. The feature point set data includes position information, scale, direction, and feature description information of each feature point in the feature point set in the image region. The triangular sequence data includes the sequence numbers of three points and three sides corresponding to each triangle in the Delaunay triangular network. The image retrieval database referred to in the embodiment of the present invention in conjunction with the embodiments of Figs. 3 to 7 is the dedicated image retrieval database described in this embodiment.

FIG. 3 is a schematic diagram of a first flow of a method for troubleshooting an image retrieval result according to an embodiment of the present invention. The image retrieval is an image retrieval based on feature extraction. When feature extraction is performed on the target image, the extracted feature data includes position information, scale, direction, and feature description information of each feature point in the image region. The image retrieval result refers to one or more sets of retrieval result image data in the image retrieval database that satisfy the matching algorithm with the target image. Each of the search result images is separately debugged by using the image search result troubleshooting method shown in FIG. 3, and includes the following steps S201 to S205.

In S201: according to the image retrieval result, the attitude relationship between the target image and each of the retrieval result images is calculated by using the scale and direction information of the pair of matching feature points. The matched feature point pair set includes a target image matching feature point set and a retrieval result image matching feature point set. A target image matching feature point and a search result image matching feature point constitute a matching feature point pair.

In S202: the coordinates of the target image matching feature point set are converted into coordinates in the search result image coordinate system according to the calculated posture relationship.

In S203: spatially sorting the feature points in the target image matching feature point set according to the coordinates converted by the coordinate system, and constructing a Delaunay triangular network corresponding to the target image matching feature point set according to the sorting result. In this step, the feature point spatial sorting manner is consistent with the sample image feature point spatial sorting manner when the image retrieval database is generated. For example, when generating an image retrieval database, the feature points of the sample image are sorted by median ordering and the largest diameter among the x and y axis diameters is the sorting axis. In this step, when the feature points in the target image matching feature point set are spatially sorted, it is also required to be performed in the same manner.

In S204, the Delaunay triangulation network corresponding to the search result image is obtained from the image retrieval database, and the feature point subsets on the unmatched are deleted in the Delaunay triangulation network corresponding to the search result image, and the search result image matching feature point set is obtained. Corresponding Delaunay triangle network.

In S205, the Delaunay triangular network corresponding to the target image matching feature point set and the Delaunay triangular network corresponding to the search result image matching feature point set are compared. If the results of the two triangular network alignments are consistent (the so-called results are consistent, that is, the corresponding point pairs in the point pair set are in the same position in the two Delaunay triangular networks), it is determined that the image retrieval result is correct; otherwise, the image retrieval result is determined to be incorrect. . The image matching algorithm is not limited in the embodiment of the present invention, and the image retrieval based on the feature extraction may be performed by using the method of the embodiment of the present invention.

In step S201, the calculated attitude relationship between the target image and the search result image can be described by a vector of length 6, which is affine[6]. In step S202, the coordinates of the feature point set of the target image are converted into coordinates in the image coordinate system of the search result according to affine [6], and the conversion expression is as follows:

Xr=Xo*affine[0]+Yo*affine[1]+affine[2]

Yr=Xo*affine[3]+Yo*affine[4]+affine[5]

Where (Xr, Yr) is the coordinate of the feature point of the target image in the image coordinate system corresponding to the search result, and (Xo, Yo) is the original coordinate of the feature point of the target image. In step S203, the point set of (Xr, Yr) is spatially ordered.

In the embodiment shown in FIG. 3, the troubleshooting condition is very severe due to the alignment troubleshooting performed by the Delaunay triangulation constructed by matching the feature point set in the entire map. As long as a set of feature points are mismatched, the entire search result is judged as an error.

In the actual image retrieval process, the target image may be subject to distortion or the like. In this case, if the process method shown in Figure 3 is used to troubleshoot, the error will be too large. In view of this situation, the present invention proposes an improvement.

FIG. 4 is a second schematic flowchart of a method for troubleshooting an image retrieval result according to an embodiment of the present invention. This troubleshooting method is improved on the embodiment illustrated in FIG. The image retrieval result troubleshooting method includes the following steps S301 to S306.

In S301, according to the image retrieval result, the attitude relationship between the target image and each of the search result images is calculated by using the scale and direction information of the pair of matching feature points. The matched feature point pair set includes a target image matching feature point set and a retrieval result image matching feature point set. A target image matching feature point and a search result image matching feature point constitute a matching feature point pair.

In S302: the coordinates of the target image matching feature point set are converted into coordinates in the search result image coordinate system according to the calculated posture relationship. The conversion method is the same as the conversion method in step S201.

In S303, according to the position of the image point of the search result corresponding to the target image matching feature point in the search result image, the subset of the feature points in the target image matching feature point after the coordinate system conversion is divided into subsets, and more The target images match a subset of feature points. Generally, the block is divided into 3*3 blocks to 7*7 blocks, and the feature point subset set in 9 to 49 blocks is subjected to subsequent step processing in units of subsets (ie, the processes in steps S304 to S306 are all in subsets). In order to avoid the feature point set matching alignment, the error of the calculation result is too large due to the different postures of the feature points. Referring to Figure 5, the left side is the search result image, and the right side is the target image. The two matching feature point pairs include A A', B B', C C', D D', E E', F F'. When the sub-region is divided into the matching feature point set, the search result image feature point A B C D E F corresponding to the target image matching feature point A'B'C'D'E'F' is located in the search result image. The location is divided into subsets. As shown in FIG. 5, the matching feature points A B C D corresponding to the four points of A'B'C'D' are located in the same region fast in the search result image, and the matching feature points E F corresponding to the two points of E'F' are The search result image is located in the same area fast. Therefore, the four points of A'B'C'D' are divided into the same target image matching feature point subset in the target image matching feature point set, and the E'F' two points are divided into another in the target image matching feature point set. A target image matches a subset of feature points. Similarly, in the search result image, the A B C D four points are divided into the same search result image matching feature point subset, and the E F is divided into another search result image matching feature point subset. A target image matching feature point subset corresponds to a search result image matching feature point subset. The corresponding target image matching feature point subset and the retrieval result image matching feature point subset are referred to as a subset pair (ie, matching feature point pair subset). In a subset pair, the feature points in the target image matching feature point subset are completely matched with the feature points in the search result image matching feature point subset. For example, a target image matching feature point subset composed of four points A'B'C'D' and a search result image matching feature point subset formed by A B C D four points are referred to as a subset pair. In this step, the reason is as follows: the position of the feature point of the search result image corresponding to the target image matching feature point in the search result image is selected, and the feature point of the coordinate image converted target image matching feature point set is subjected to a subset. The division is because the image retrieval is based on the sample images stored in the database. The sample image is a complete image, and the target image may not exist during the shooting process. In the case of a complete image or the like (that is, only a part of the entire image is taken), if the target image is used as a basis for the division, there is a high possibility of an error.

In S304, the feature points in each of the target image matching feature point subsets are spatially sorted according to the coordinates converted by the coordinate system. According to the sorting result, a Delaunay triangulation network is constructed for each target image matching feature point subset. In this step, the feature point spatial sorting manner is consistent with the sample image feature point spatial sorting manner when the image retrieval database is generated.

In S305, a Delaunay triangular network corresponding to the search result image is obtained from the image retrieval database, and the feature point subsets on the unmatched are deleted in the Delaunay triangular network, and the matching feature points of the search result image in the matching feature point pair set are obtained. The Delaunay triangle network corresponding to the episode.

In S306: comparing the two pairs of Delaunay triangulation networks corresponding to each pair of matched feature points (the above two Delaunay triangulation networks refer to the pairs of matching feature points obtained in steps S304 and S305 respectively) Corresponding to the two Delaunay triangle networks). If the subset of the preset ratio exceeds the alignment result of the two triangular networks, the image retrieval result is determined to be correct; otherwise, the image retrieval result is determined to be incorrect. In this step, the preset ratio can be freely set according to the actual situation, and the setting range is preferably between 1/3 and 1/6. Assumption: The preset ratio is set to 2/3. At this time, if the subset of more than 2/3 matches the two triangular network alignment results, it is determined that the image retrieval result is correct.

Using the flow method of FIG. 4, the influence of the distorted image on the retrieval result can be effectively reduced, and the accuracy of the retrieval result can be further improved. The embodiment of FIG. 4 does not limit the image matching algorithm, as long as the image retrieval based on feature extraction can be performed by using the method of the embodiment of the present invention.

FIG. 6 is a schematic diagram of a first structure of an image retrieval server according to an embodiment of the present invention. The image retrieval server of the present embodiment includes an image retrieval database 10 and an image matching module 11. The image retrieval database 10 is a dedicated image retrieval database described in the foregoing embodiment. The image matching module 11 is configured to receive a retrieval request from the image retrieval client, and match the target image data included in the retrieval request in the image retrieval database according to the set matching algorithm to obtain one or more groups and targets. The image satisfies the search result image data of the matching algorithm.

The image retrieval server also includes a debug module 12. The troubleshooting module 12 includes a posture calculation and coordinate system conversion unit 121, a triangular network construction unit 122, and a determination unit 123.

The attitude calculation and coordinate system conversion unit 121 is configured to calculate the attitude relationship between the target image and each of the retrieval result images by using the scale and direction information of the pair of matching feature points according to the image retrieval result, and according to the calculated posture relationship. Converting the coordinates of the target image matching feature point set to the coordinates in the search result image coordinate system. The matched feature point pair set includes a target image matching feature point set and a retrieval result image matching feature point set. a target image Matching feature points and a search result image matching feature points constitute a matching feature point pair.

The triangular network construction unit 122 is configured to spatially sort the feature points in the target image matching feature point set according to the coordinate converted by the coordinate system (the sorting manner is consistent with the sorting manner used in the image retrieval database generation), and construct the target image according to the sorting result. Match the Delaunay triangle network corresponding to the feature point set.

The determining unit 123 is configured to obtain a Delaunay triangulation network corresponding to the search result image from the image retrieval database 10, and delete the unmatched feature point subset in the Delaunay triangulation network corresponding to the search result image to obtain a search result image matching. The Delaunay triangular network corresponding to the feature point set compares the Delaunay triangular network corresponding to the search result image matching feature point set with the Delaunay triangular network corresponding to the target image matching feature point set constructed by the triangular network construction unit 122. If the alignment results of the two triangular networks are consistent, it is determined that the image retrieval result is correct; otherwise, the image retrieval result is determined to be incorrect.

FIG. 7 is a schematic diagram of a second structure of an image retrieval server according to an embodiment of the present invention. The image retrieval server structure differs from that of FIG. 6 in that the debug module 12 is different in structure. In this embodiment, a subset division unit 124 is added to the debug module 12, and the functions of the triangle network construction unit 125 and the determination unit 126 are also different from those of the corresponding unit in FIG. 6, as follows:

The posture calculation and coordinate system conversion unit 121 in FIG. 7 is completely consistent with the functions of the posture calculation and the coordinate system conversion unit in FIG. 6, and is used for calculating the target image by using the scale and direction information of the matching feature point pair set according to the image retrieval result. The attitude relationship between each of the search result images is converted, and the coordinates of the target image matching feature point set are converted into coordinates in the search result image coordinate system according to the calculated posture relationship.

The subset dividing unit 124 is configured to perform a subset of the feature points in the target image matching feature point set after the coordinate system is converted according to the position of the image point of the search result corresponding to the target image matching the feature point in the search result image. Dividing, obtaining a plurality of target image matching feature point subsets. A detailed description of the subset partitioning is provided in the section describing the FIG.

The triangular network construction unit 125 is configured to spatially sort the feature points of each target image matching feature point subset according to the coordinates converted by the coordinate system (the sorting manner is consistent with the sorting manner used in the image retrieval database generation), according to the sorting result Each target image matches a subset of feature points to construct a Delaunay triangulation network.

The determining unit 126 is configured to obtain a Delaunay triangulation network corresponding to the search result image from the image retrieval database, and delete the unmatched feature point subset in the Delaunay triangulation network corresponding to the search result image to obtain a matching feature point pair set. Each of the search result images matches the Delaunay triangular network corresponding to the subset of feature points, and compares the two Delaunay triangular networks corresponding to each subset pair. If the subset of the preset ratio exceeds the alignment result of the two triangular networks, the image retrieval result is determined to be correct; otherwise, the image retrieval result is determined to be incorrect.

The Delaunay triangulation network corresponding to the target image and the search result map are generated online in conjunction with FIG. 8 to FIG. A specific embodiment of the present invention will be described by taking a corresponding Delaunay triangular network as an example. In the later paragraphs of the present specification, the image retrieval database mentioned in the embodiments introduced in conjunction with FIGS. 8 to 11 is a conventional image retrieval database in which Delaunay triangular network data of sample images is not stored in advance, and only sample images need to be stored. The feature point set data can be. The feature point set data of each sample image includes position information, scale, direction, and feature description information of each feature point in the feature point set in the image region.

FIG. 8 is a third schematic flowchart of a method for troubleshooting image retrieval results according to an embodiment of the present invention. The image retrieval is also an image retrieval based on feature extraction. When feature extraction is performed on the target image, the extracted feature data includes position information, scale, direction, and feature description information of each feature point in the image region. The image retrieval result refers to one or more sets of retrieval result image data in the image retrieval database that satisfy the matching algorithm with the target image. The image retrieval result debugging method shown in FIG. 8 performs error correction for each of the search result images, respectively, and includes the following steps S401 to S404.

In S401: calculating a posture relationship between the target image and each of the search result images by using the matching feature point pair scale and direction information according to the image retrieval result. The matched feature point pair set includes a target image matching feature point set and a retrieval result image matching feature point set. A target image matching feature point and a search result image matching feature point constitute a matching feature point pair.

In S402: the coordinates of the target image and the search result image matching feature point set are converted into the same coordinate system according to the calculated posture relationship. In a specific implementation, the coordinates of the target image matching feature point set may be converted into the search result image coordinate system, or the coordinates of the search result image matching feature point set may be converted into the target image coordinate system. The coordinate system conversion method refers to the description of step S201 in the foregoing embodiment.

In S403, Delaunay triangulation is performed on the target image matching feature point set and the search result image matching feature point set respectively in the converted coordinate system, and the Delaunay triangular network corresponding to the target image and the Delaunay triangular network corresponding to the search result image are obtained. . The Delaunay triangulation is specifically as follows: matching the feature points in the feature points of the target image and the feature points in the image feature points of the search result, spatially sorting according to the coordinates converted by the coordinate system, and constructing corresponding Delaunay triangles according to the sorting results. The internet. The spatial ordering can be a median ordering. The specific method of median sorting refers to the description of the median ordering in step S103 in the previous embodiment.

In S404: the above two Delaunay triangular networks are compared. If the results of the two triangular network alignments are consistent (the so-called results are consistent, that is, the corresponding point pairs in the point pair set are in the same position in the two Delaunay triangular networks), it is determined that the image retrieval result is correct; otherwise, the image retrieval result is determined to be incorrect. . The image matching algorithm is not limited in the embodiment of the present invention, and the image retrieval based on the feature extraction may be performed by using the method of the embodiment of the present invention.

FIG. 9 is a fourth schematic flowchart of a method for troubleshooting an image retrieval result according to an embodiment of the present invention. The wrong side The method is improved on the embodiment illustrated in Fig. 8. The method flow illustrated in FIG. 8 is to perform a Delaunay triangulation on the entire matching feature point set of the target image and the search result image, respectively. The method flow illustrated in FIG. 9 is to first divide a subset of matching feature points of the target image and the search result image, and then perform Delaunay triangulation on each of the matched feature point subsets respectively, and finally use the subset pair corresponding to the corresponding The Delaunay triangle network is compared. The troubleshooting process of FIG. 9 specifically includes the following steps S501 to S505.

In S501: calculating the attitude relationship between the target image and each of the search result images by using the matching feature point pair scale and direction information according to the image retrieval result.

In S502, the coordinates of the target image and the search result image matching feature point set are converted into the same coordinate system according to the calculated posture relationship. Similarly, in a specific implementation, the coordinates of the target image matching feature point set may be converted into the search result image coordinate system, or the coordinates of the search result image matching feature point set may be converted into the target image coordinate system. The coordinate system conversion method refers to the description of step S201 in the foregoing embodiment.

In S503, according to the position information of the search result image corresponding to the target image matching feature point matching the feature point in the search result image, the subset of the matched feature point pair set after the coordinate system conversion is divided, and several matching features are obtained. Point to subset. Each pair of matching feature point pairs includes a target image matching feature point subset and a retrieval result image matching feature point subset. The specific subset division manner can be combined with FIG. 5, and the sub-set division description in step S303 is referred to. The mode of the subset division in S503 is substantially the same as that of step S303, except that the manner of description is slightly different.

In S504, in the converted coordinate system, each of the target image matching feature point subset and the search result image matching feature point subset are respectively subjected to Delaunay triangulation to obtain a corresponding Delaunay triangular network. In this step, Delaunay triangulation is performed for each subset, specifically: spatially sorting the coordinate points of the feature points in each subset according to the coordinate transformation, and constructing a Delaunay triangular network for each subset according to the sorting result. The spatial ordering can be a median ordering. The specific method of median sorting refers to the description of the median ordering in step S103 in the previous embodiment.

In S505: comparing two Delaunay triangulation networks corresponding to the subset of matching feature points (a Delaunay triangular network corresponding to a target image matching feature point subset and a Delaunay triangular network corresponding to a search result image matching feature point subset) . If the matching feature point pair of the preset ratio exceeds the matching result of the two triangular networks, the image retrieval result is determined to be correct; otherwise, the image retrieval result is determined to be incorrect. In this step, the preset ratio can be freely set according to the actual situation, and the setting range is preferably between 1/3 and 1/6. Assumption: The preset ratio is set to 2/3. At this time, if the subset of more than 2/3 matches the two triangular network alignment results, it is determined that the image retrieval result is correct.

FIG. 10 is a schematic diagram of a third structure of an image retrieval server according to an embodiment of the present invention. Image retrieval service The apparatus includes an image retrieval database 20 and a matching module 21. The image retrieval database 20 stores feature point set data of a plurality of sample images. The feature point set data of each sample image includes position information, scale, direction, and feature description information of each feature point in the feature point set in the image region. The matching module 21 is configured to receive a retrieval request from the image retrieval client, and match the target image data included in the retrieval request in the image retrieval database according to the set matching algorithm to obtain one or more groups and target images. The search result image data satisfying the matching algorithm. The image retrieval server also includes a debug module 22. The troubleshooting module 22 includes a posture calculation and coordinate system conversion unit 221, a triangular network construction unit 222, and a determination unit 223, wherein:

The posture calculation and coordinate system conversion unit 221 is configured to calculate a posture relationship between the target image and each of the retrieval result images by using the matching feature point pair scale and direction information according to the image retrieval result, and according to the calculated posture The relationship converts the coordinates of the target image and the search result image matching feature point set into the same coordinate system. In a specific implementation, the coordinates of the target image matching feature point set may be converted into the search result image coordinate system, or the coordinates of the search result image matching feature point set may be converted into the target image coordinate system. The coordinate system conversion method refers to the description of step S201 in the foregoing embodiment.

The triangular network construction unit 222 is configured to perform Delaunay triangulation on the target image matching feature point set and the retrieval result image matching feature point set respectively in the converted coordinate system, to obtain the Delaunay triangulation network corresponding to the target image and the retrieval result. The image corresponds to the Delaunay triangle network. The Delaunay triangulation is specifically as follows: matching the feature points in the feature points of the target image and the feature points in the image feature points of the search result, spatially sorting according to the coordinates converted by the coordinate system, and constructing corresponding Delaunay triangles according to the sorting results. The internet. The spatial ordering can be a median ordering. The specific method of median sorting refers to the description of the median ordering in step S103 in the previous embodiment.

The determining unit 223 is configured to compare the two Delaunay triangulation networks. If the two triangle network comparison results are consistent, the image retrieval result is determined to be correct; otherwise, the image retrieval result is determined to be incorrect.

FIG. 11 is a schematic diagram of a fourth structure of an image retrieval server according to an embodiment of the present invention. The image retrieval server structure differs from that of FIG. 10 in that the debug module 22 is different in structure. In this embodiment, a subset division unit 224 is added to the debug module 22, and the functions of the triangle network construction unit 225 and the determination unit 226 are also different from the corresponding unit functions in FIG. 10, as follows:

The posture calculation and coordinate system conversion unit 221 is completely consistent with the functions of the posture calculation and the coordinate system conversion unit in FIG. 10, and is used for calculating the target image and each by using the scale and direction information of the pair of matching feature points according to the image retrieval result. A pose relationship between the search result images, and according to the calculated pose relationship, the coordinates of the target image matching feature point set and the search result image matching feature point set are converted into the same coordinate system. Similarly, in the specific implementation: the coordinates of the target image matching feature point set may be converted into the search result image coordinate system, or the retrieval result may be The coordinates of the image matching feature point set are converted into the target image coordinate system. The coordinate system conversion method refers to the description of step S201 in the foregoing embodiment.

The subset dividing unit 224 is configured to perform a subset of the matched feature point pair set after the coordinate system is converted according to the position information of the search result image corresponding to the target image matching feature point matching the feature point in the search result image. Dividing, a number of matching feature point pairs are obtained. Each pair of matching feature point pairs includes a target image matching feature point subset and a retrieval result image matching feature point subset. The specific subset division manner can be combined with FIG. 5, and the sub-set division description in step S303 is referred to. The mode of the subset division in S503 is substantially the same as that of step S303, except that the manner of description is slightly different.

The triangular network construction unit 225 is configured to perform Delaunay triangulation on each of the target image matching feature point subset and the search result image matching feature point subset in the converted coordinate system to obtain a corresponding Delaunay triangular network. Delaunay triangulation is performed for each subset. Specifically, the feature points in each subset are spatially sorted according to the coordinates after coordinate transformation, and a Delaunay triangulation network is constructed for each subset according to the sorting result. The spatial ordering can be a median ordering. The specific method of median sorting refers to the description of the median ordering in step S103 in the previous embodiment.

The determining unit 226 is configured to compare the two Delaunay triangulation networks corresponding to the subset of the matching feature points, and if the matched feature point pair subsets exceeding the preset ratio satisfy the consistency of the two triangular network comparison results, It is determined that the image retrieval result is correct; otherwise, the image retrieval result is determined to be incorrect. The preset ratio can be freely set according to the actual situation, and the setting range is preferably between 1/3 and 1/6. Assumption: The preset ratio is set to 2/3. At this time, if the subset of more than 2/3 matches the two triangular network alignment results, it is determined that the image retrieval result is correct.

An embodiment of the present invention further provides an image retrieval method, including the following steps S601 to S604.

In S601: the target image is acquired.

In S602: feature extraction is performed on the target image or the pre-processed target image. The extracted feature data includes position information, scale, direction, and feature description information of each feature point in the image area. The preprocessed sample image refers to a sample image preprocessed by one or more methods of uniform size processing, redundant area culling, Gaussian blur processing, and affine transformation.

In S603: the extracted feature data is sent to the image retrieval server for image retrieval, and one or more search result images that are initially matched with the target image are obtained.

In S604, the search result is debugged by using the image search result troubleshooting method in any of the embodiments of the present invention.

An embodiment of the present invention further provides an image retrieval system including an image retrieval client and an image retrieval server. The image retrieval client is installed on the mobile terminal. The image retrieval server is an image retrieval server illustrated in any of Figs. 6, 7, 10, and 11. The image retrieval client includes an image acquisition module, a feature extraction module, a retrieval request sending module, and a retrieval result receiving module. The image acquisition module is configured to acquire a target image. The feature extraction module is configured to perform feature extraction on the target image, and the extracted feature data includes location information, scale, direction, and feature description information of each feature point in the image region. The search request sending module is configured to send the feature data extracted by the feature extraction module to the image retrieval server for image retrieval. The search result receiving module is configured to receive the search result information returned from the image retrieval server.

The module or unit in the embodiment of the present invention may be implemented by a general-purpose integrated circuit, such as a CPU (Central Processing Unit) or an ASIC (Application Specific Integrated Circuit).

One of ordinary skill in the art can understand that all or part of the process of implementing the foregoing embodiments can be completed by a computer program to instruct related hardware, and the program can be stored in a computer readable storage medium. When executed, the flow of an embodiment of the methods as described above may be included. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

The above is only the preferred embodiment of the present invention, and the scope of the present invention is not limited thereto, and those skilled in the art can understand all or part of the process of implementing the above embodiments, and according to the claims of the present invention. The equivalent change is still within the scope of the invention.

Claims

An image retrieval result troubleshooting method, the image retrieval is an image retrieval based on feature extraction, and when the feature image is extracted from the target image, the extracted feature data includes position information, scale, and direction of each feature point in the image region. And the feature description information; the image retrieval result refers to one or more sets of search result image data in the image retrieval database that meets the matching algorithm with the target image, and is characterized in that each of the search result images is separately used by the image retrieval result debugging method. Troubleshoot, including:

According to the image retrieval result, the attitude relationship between the target image and each of the search result images is calculated by using the scale and direction information of the matched feature point pair set, and the matched feature point pair set includes the target image matching feature point set and the search result image. Matching feature point sets;

According to the calculated attitude relationship, the coordinates of the target image matching feature point set and the retrieval result image matching feature point set are converted into the same coordinate system, and the target image matching feature point set and the retrieval result are respectively performed in the converted coordinate system. The image matching feature point set is Delaunay triangulation, and the Delaunay triangulation network corresponding to the target image and the Delaunay triangulation network corresponding to the retrieval result image are obtained.

The two Delaunay triangulation networks are compared. If the alignment results of the two triangular networks are consistent, the image retrieval result is determined to be correct; otherwise, the image retrieval result is determined to be incorrect.
The method according to claim 1, wherein the coordinates of the target image matching feature point set and the retrieval result image matching feature point set are converted into the same coordinate system, specifically: matching the target image with the feature point set The coordinates are converted into the search result image coordinate system, or the coordinates of the search result image matching feature point set are converted into the target image coordinate system.
The method according to claim 2, wherein said Delaunay triangulation is performed on the target image matching feature point set and the retrieval result image matching feature point set in the converted coordinate system, respectively: The image matching feature point set and the feature points in the image feature point of the retrieval result are spatially sorted according to the coordinates converted by the coordinate system, and the corresponding Delaunay triangular network is constructed according to the sorting result.
The method of claim 3, wherein the spatial ordering is a median ordering, specifically comprising:

a sorting axis determining step of using an axis having a maximum/minimum diameter on the x-axis and the y-axis of the feature point set feature point as a sorting axis;

And an updating step of calculating a median value of two feature points constituting the maximum/minimum diameter on the sorting axis, and changing the original feature point set such that the feature point spatially located to the left of the median value is located at the median point in the data set On the left side, and make the feature points on the right side of the median value in the data set to the right of the median point;

In the looping step, the sorting axis determining step and the updating step are repeated for the point set formed by the left point and the point set formed by the right point until the number of feature points on the median side is less than 2.
An image retrieval result troubleshooting method, the image retrieval is an image retrieval based on feature extraction, and when the feature image is extracted from the target image, the extracted feature data includes position information, scale, and direction of each feature point in the image region. And the feature description information; the image retrieval result refers to one or more sets of search result image data in the image retrieval database that meets the matching algorithm with the target image, and is characterized in that each of the search result images is separately used by the image retrieval result debugging method. Troubleshoot, including:

According to the image retrieval result, the attitude relationship between the target image and each of the search result images is calculated by using the scale and direction information of the matched feature point pair set, and the matched feature point pair set includes the target image matching feature point set and the search result image. Matching feature point sets;

Converting the coordinates of the target image matching feature point set and the retrieval result image matching feature point set to the same coordinate system according to the calculated posture relationship;

According to the position information of the search result image corresponding to the target image matching feature point matching the feature point in the search result image, the subset of the matched feature point pair set is divided, and a plurality of matching feature point pair subsets are obtained, and each matching feature is obtained. The point pair subset includes a target image matching feature point subset and a retrieval result image matching feature point subset;

In the transformed coordinate system, Delaunay triangulation is performed on each target image matching feature point subset and the search result image matching feature point subset respectively to obtain a corresponding Delaunay triangulation network;

Comparing the two Delaunay triangulation networks corresponding to the subset of matching feature points to the subset, if the matching feature point pair of the preset ratio exceeds the matching result of the two triangular networks, the image retrieval result is determined to be correct; otherwise It is determined that the image retrieval result is incorrect.
The method according to claim 5, wherein the coordinates of the target image matching feature point set and the retrieval result image matching feature point set are converted into the same coordinate system, specifically: matching the target image with the feature point set The coordinates are converted into the search result image coordinate system, or the coordinates of the search result image matching feature point set are converted into the target image coordinate system.
The method according to claim 6, wherein each of the subsets is subjected to Delaunay triangulation, specifically: spatially sorting the coordinate points of the feature points in each subset according to the coordinate conversion, according to the sorting result for each Construct a Delaunay triangulation network in a subset;

The spatial ordering refers to median sorting, and specifically includes:

The sorting axis determining step is to take the axis having the largest/minimum diameter on the x-axis and the y-axis of the feature point in the feature point Sorting axis

And an updating step of calculating a median value of two feature points constituting the maximum/minimum diameter on the sorting axis, and changing the original feature point set such that the feature point spatially located to the left of the median value is located at the median point in the data set On the left side, and make the feature points on the right side of the median value in the data set to the right of the median point;

In the looping step, the sorting axis determining step and the updating step are repeated for the point set formed by the left point and the point set formed by the right point until the number of feature points on the median side is less than 2.
An image retrieval method, comprising:

Obtain the target image;

Feature extraction is performed on the target image or the pre-processed target image, and the extracted feature data includes position information, scale, direction and feature description information of each feature point in the image region;

Sending the extracted feature data to an image retrieval server for image retrieval, to obtain one or more search result images that initially match the target image;

The search result image of the preliminary matching is subjected to the image search result troubleshooting method according to any one of claims 1 to 7, and the search result is debugged.
An image retrieval server includes an image retrieval database and an image matching module. The image retrieval database stores feature point set data of a plurality of sample images, and the feature point set data of each sample image includes each feature point in the feature point set. Location information, scale, direction, and feature description information in the image area; the image matching module is configured to receive a retrieval request from the image retrieval client, and image the target image included in the retrieval request according to the set matching algorithm Searching the database for matching, and obtaining one or more sets of search result image data that meets the matching algorithm with the target image, wherein the image retrieval server further includes a debugging module, and the debugging module includes:

The attitude calculation and coordinate system conversion unit is configured to calculate the attitude relationship between the target image and each of the retrieval result images according to the image retrieval result, and use the matching feature point pair scale and direction information, and according to the calculated posture relationship, Converting the coordinates of the target image matching feature point set and the retrieval result image matching feature point set into the same coordinate system, the matching feature point pair set including the target image matching feature point set and the retrieval result image matching feature point set;

A triangular network construction unit is configured to perform Delaunay triangulation on the target image matching feature point set and the search result image matching feature point set respectively in the converted coordinate system, and obtain a Delaunay triangular network corresponding to the target image and the search result image corresponding to the image. Delaunay triangle network;

The determining unit is configured to compare the two Delaunay triangular networks, and if the two triangular network comparison results are consistent, the image retrieval result is determined to be correct; otherwise, the image retrieval result is determined to be incorrect.
The image retrieval server according to claim 9, wherein the coordinates of the target image matching feature point set and the retrieval result image matching feature point set are converted into the same coordinate system, specifically: matching the target image with the feature point The set coordinates are converted into the search result image coordinate system, or the coordinates of the search result image matching feature point set are converted into the target image coordinate system.
The image retrieval server according to claim 10, wherein said Delaunay triangulation is performed on the target image matching feature point set and the search result image matching feature point set in the converted coordinate system, specifically: respectively The target image matching feature point set and the feature points in the image point set of the search result are spatially sorted according to the coordinates converted by the coordinate system, and the corresponding Delaunay triangle networks are constructed according to the sorting result.
An image retrieval server includes an image retrieval database and an image matching module. The image retrieval database stores feature point set data of a plurality of sample images, and the feature point set data of each sample image includes each feature point in the feature point set. Location information, scale, direction, and feature description information in the image area; the image matching module is configured to receive a retrieval request from the image retrieval client, and image the target image included in the retrieval request according to the set matching algorithm Searching the database for matching, and obtaining one or more sets of search result image data that meets the matching algorithm with the target image, wherein the image retrieval server further includes a debugging module, and the debugging module includes:

The attitude calculation and coordinate system conversion unit is configured to calculate the attitude relationship between the target image and each of the retrieval result images according to the image retrieval result, and use the matching feature point pair scale and direction information, and according to the calculated posture relationship, Converting the coordinates of the target image matching feature point set and the retrieval result image matching feature point set into the same coordinate system, the matching feature point pair set including the target image matching feature point set and the retrieval result image matching feature point set;

The sub-division unit obtains a subset of the matched feature point pair set according to the position information of the search result image corresponding to the target image matching feature point matching the feature point in the search result image, and obtains a plurality of matching feature point pair subsets. Each of the matching feature point pair subsets includes a target image matching feature point subset and a retrieval result image matching feature point subset;

A triangular network building unit is configured to perform Delaunay triangulation on each target image matching feature point subset and the search result image matching feature point subset in the converted coordinate system to obtain a corresponding Delaunay triangular network;

The determining unit compares the two Delaunay triangulation networks corresponding to the subset of the matching feature points, and if the matching feature point pair of the preset ratio exceeds the matching result of the two triangular networks, the image retrieval result is determined. Correct; otherwise the image retrieval result is determined to be incorrect.
The image retrieval server according to claim 12, wherein the coordinates of the target image matching feature point set and the retrieval result image matching feature point set are converted into the same coordinate system, specifically: matching the target image The coordinates of the point set are converted into the search result image coordinate system, or the coordinates of the search result image matching feature point set are converted into the target image coordinate system.
The image retrieval server according to claim 13, wherein the Delaunay triangulation is performed for each subset, specifically: spatially sorting the coordinate points of the feature points in each subset according to the coordinate conversion, according to the sorting result Construct a Delaunay triangle network for each subset.
An image retrieval system comprising an image retrieval client and an image retrieval server, characterized in that:

The image retrieval server is the image retrieval server according to any one of claims 9 to 14;

The image retrieval client includes an image acquisition module, a feature extraction module, a retrieval request sending module, and a retrieval result receiving module, wherein:

The image acquisition module is configured to acquire a target image;

The feature extraction module is configured to perform feature extraction on the target image, and the extracted feature data includes location information, scale, direction, and feature description information of each feature point in the image region;

The search request sending module is configured to send the feature data extracted by the feature extraction module to an image retrieval server for image retrieval;

The search result receiving module is configured to receive the search result information returned from the image retrieval server.