WO2020186777A1

WO2020186777A1 - Image retrieval method, apparatus and device, and computer-readable storage medium

Info

Publication number: WO2020186777A1
Application number: PCT/CN2019/117191
Authority: WO
Inventors: 陈世喆; 王威; 吴力丰; 王昊; 何维; 谢树铭; 龚阳
Original assignee: 平安科技（深圳）有限公司
Priority date: 2019-03-16
Filing date: 2019-11-11
Publication date: 2020-09-24
Also published as: CN110059212A

Abstract

The present application belongs to the technical field of image retrieval. Disclosed is an image retrieval method, comprising: acquiring an image to be retrieved, and performing main feature extraction and non-main feature extraction on the image to be retrieved, so as to determine, based on extracted main features and non-main features, an image tag corresponding to the image to be retrieved; performing retrieval in a preset image database based on the image tag, and determining an image set corresponding to the image tag; determining image features of the image to be retrieved based on a preset image feature extraction algorithm, performing feature matching in the image set according to the image features, and determining alternative images corresponding to the image to be retrieved; and performing precision sorting on the alternative images based on a preset sorting algorithm, and outputting the alternative images according to the sequence after sorting. Further disclosed are an image retrieval apparatus and device, and a computer-readable storage medium. According to the present application, the accuracy of picture retrieval is improved.

Description

Image retrieval method, device, equipment and computer readable storage medium To

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office, the application number is 201910206127.1, and the invention title is "image retrieval method, device, equipment and computer-readable storage medium" on March 16, 2019. The reference is incorporated in this application.

Technical field

This application relates to the field of image retrieval technology, and in particular to an image retrieval method, device, equipment and computer-readable storage medium.

Background technique

Due to the rapid development and wider application of multimedia, image information, and network technologies, the scale of image databases is getting larger and larger, and how to effectively manage visual information has become an urgent problem to be solved. However, the existing image retrieval technology is still in the experimental stage, and the accuracy of image retrieval is not high enough, especially when there are image occlusions, low sample quantity and quality, and lack of information, it is even more necessary to improve the accuracy of image retrieval.

Summary of the invention

The main purpose of this application is to provide an image retrieval method, device, equipment, and computer-readable storage medium, aiming to solve the technical problem of low image retrieval accuracy in the existing image retrieval technology.

To achieve the above objective, this application provides an image retrieval method, the image retrieval method includes:

Acquiring the image to be retrieved, and performing subject feature extraction and non-subject feature extraction on the image to be retrieved, so as to determine the image label corresponding to the image to be retrieved based on the extracted subject feature and non-subject feature;

Searching in a preset image database based on the image tag, and determining the image set corresponding to the image tag;

Determine the image feature of the image to be retrieved based on a preset image feature extraction algorithm, and perform feature matching in the image set according to the image feature to determine the candidate image corresponding to the image to be retrieved;

The candidate images are accurately sorted based on a preset sorting algorithm, and the candidate images are output in the sorted order.

In addition, in order to achieve the above-mentioned object, the present application also provides an image retrieval device, which includes:

The feature extraction module is used to obtain the image to be retrieved, and perform subject feature extraction and non-subject feature extraction on the image to be retrieved, so as to determine the image label corresponding to the image to be retrieved based on the extracted subject feature and non-subject feature;

The tag retrieval module is configured to perform retrieval in a preset image database based on the image tag, and determine the image set corresponding to the image tag;

The feature matching module is used to determine the image feature of the image to be retrieved based on a preset image feature extraction algorithm, and perform feature matching in the image set according to the image feature to determine the candidate corresponding to the image to be retrieved image;

The image sorting module is configured to accurately sort the candidate images based on a preset sorting algorithm, and output the candidate images in the sorted order.

In addition, in order to achieve the above object, the present application also provides an image retrieval device, the image retrieval device includes: a memory, a processor, and computer-readable instructions stored on the memory and running on the processor, When the computer-readable instructions are executed by the processor, the steps of the image retrieval method described above are implemented.

In addition, in order to achieve the above objective, this application also provides a computer-readable storage medium. The computer-readable storage medium may be a non-volatile readable storage medium on which computer-readable instructions are stored. When the computer-readable instructions are executed by the processor, the steps of the image retrieval method described above are realized.

This application belongs to the field of image retrieval technology, and proposes an image retrieval method. First, the image to be retrieved is acquired, and the corresponding image tag is determined based on the subject and non-subject features of the image to be retrieved. The image tag is used in the preset Search in the image database, determine the image set corresponding to the image tag, and further determine the image feature of the image to be retrieved based on the preset image feature extraction algorithm, so as to perform feature matching in the determined image set according to the image feature to determine the image feature to be retrieved. The candidate images corresponding to the retrieved images are finally sorted with accuracy through a preset sorting algorithm, and the final sorted candidate images are output as the retrieval result. The image retrieval method proposed in this application determines image tags by extracting subject features and non-subject features to perform preliminary retrieval to narrow the retrieval range, and then accurately obtain retrieval results that match the image to be retrieved through image feature matching. Finally, The search results are reordered through the sorting algorithm to provide users with more accurate search results and improve the efficiency and accuracy of image search.

Description of the drawings

FIG. 1 is a schematic diagram of the hardware structure of an image retrieval device involved in a solution of an embodiment of the application;

2 is a schematic flowchart of the first embodiment of the image retrieval method of this application;

FIG. 3 is a detailed flowchart of step S20 in FIG. 2;

4 is a schematic flowchart of a second embodiment of an image retrieval method according to this application;

FIG. 5 is a schematic diagram of the functional modules of the first embodiment of the image retrieval device of this application.

The realization, functional characteristics, and advantages of the purpose of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

detailed description

It should be understood that the specific embodiments described here are only used to explain the application, and are not used to limit the application.

The main solution of the embodiment of the application is to obtain the image to be retrieved, and perform subject feature extraction and non-subject feature extraction on the image to be retrieved, so as to determine the corresponding image to be retrieved based on the extracted subject feature and non-subject feature Image tags; search based on the image tags in a preset image database to determine the image set corresponding to the image tags; determine the image features of the image to be retrieved based on the preset image feature extraction algorithm, and according to the Image features perform feature matching in the image set to determine the candidate image corresponding to the image to be retrieved; perform accurate sorting on the candidate images based on a preset sorting algorithm, and output the sorted images in the sorted order Alternative image. Through the technical solutions of the embodiments of the present application, the technical problem of low image retrieval accuracy in the existing image retrieval technology is solved.

As shown in FIG. 1, FIG. 1 is a schematic diagram of the hardware structure of the image retrieval device involved in the solution of the embodiment of the application.

The image retrieval method involved in the embodiments of the present application is mainly applied to an image retrieval device, and the image retrieval device may be a device with display and processing functions such as a PC, a portable computer, and a mobile terminal.

As shown in FIG. 1, the image retrieval device may include a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to implement connection and communication between these components. The user interface 1003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 can be a high-speed RAM memory or a stable memory (non-volatile memory), such as disk storage. Optionally, the memory 1005 may also be a storage device independent of the foregoing processor 1001.

Optionally, the image retrieval device may also include a camera, RF (Radio Frequency, radio frequency) circuits, sensors, audio circuits, Wi-Fi modules, etc. Among them, sensors such as light sensors, motion sensors and other sensors. Of course, the image retrieval device can also be equipped with other sensors such as gyroscope, barometer, hygrometer, thermometer, infrared sensor, etc., which will not be described here.

Those skilled in the art can understand that the structure of the image retrieval device shown in FIG. 1 does not constitute a limitation on the image retrieval device, and may include more or less components than shown in the figure, or a combination of certain components, or different components Layout.

As shown in FIG. 1, a memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and computer readable instructions. In Figure 1, the network interface 1004 is mainly used to connect to the back-end server and communicate with the back-end server; the user interface 1003 is mainly used to connect to the client (user side) and communicate with the client; and the processor 1001, the memory 1005 It may be set in an image retrieval device, which calls computer-readable instructions stored in the memory 1005 through the processor 1001, and executes the image retrieval method provided in the embodiment of the present application.

The solution provided in this embodiment first obtains the image to be retrieved, and extracts and determines the corresponding image label based on the subject feature and non-subject feature of the image to be retrieved, and uses the image tag to search in a preset image database to determine the image label Corresponding image set, and further determine the image feature of the image to be retrieved based on the preset image feature extraction algorithm, so as to perform feature matching in the determined image set according to the image feature to determine the candidate image corresponding to the image to be retrieved, Finally, the candidate images are accurately sorted by a preset sorting algorithm, and the final sorted candidate images are output as the retrieval result. The image retrieval method proposed in this application determines image tags by extracting subject features and non-subject features to perform preliminary retrieval to narrow the retrieval range, and then accurately obtain retrieval results that match the image to be retrieved through image feature matching. Finally, The search results are reordered through the sorting algorithm to provide users with more accurate search results and improve the efficiency and accuracy of image search.

Based on the above hardware structure, an embodiment of the image retrieval method of this application is proposed.

Referring to Fig. 2, Fig. 2 is a schematic flowchart of a first embodiment of an image retrieval method according to this application. In this embodiment, the method includes:

Step S10: Obtain the image to be retrieved, and perform subject feature extraction and non-subject feature extraction on the image to be retrieved, so as to determine the image label corresponding to the image to be retrieved based on the extracted subject feature and non-subject feature;

In order to solve the problem of low accuracy of image retrieval in the existing image retrieval technology, this application proposes an image retrieval method. First, when an image retrieval request is received, the image to be retrieved is acquired and based on a preset The feature extraction algorithm extracts subject features and non-subject features, and determines the corresponding image label through subject feature and non-subject feature extraction. Specifically, in this embodiment, the feature extraction of the image to be retrieved includes subject feature extraction and non-subject feature extraction. The subject is the subject with the subject, and the non-subject includes some special objects, such as landmark buildings. Therefore, the subject feature and non-subject feature of the image to be retrieved can be obtained through subject feature extraction and non-subject feature extraction.

In this embodiment, the preset feature extraction algorithm used for extracting subject features and extracting non-subject features of the image to be retrieved specifically refers to Mask The R-CNN algorithm, the process of extracting the main features of the image to be retrieved through Mask R-CNN is as follows:

First, preprocess the image to be retrieved in order to enhance the subject features in the image to be retrieved; secondly, input the preprocessed image to be retrieved into the preset convolutional neural network to perform the subject feature map of the image to be retrieved Finally, use RPN (Region Candidate Network) to determine and classify subject features, specifically, by setting a preset number of ROIs ((region) for each point in the feature map obtained in the previous step of interest, region of interest), you can obtain multiple ROIs, filter these multiple ROIs, filter out a part of the ROI, and finally perform semantic segmentation on the effective ROI area to identify the subject features in the image to be retrieved.

Similarly, in this embodiment, the non-subject features in the image to be retrieved are also extracted using Mask The difference between the R-CNN algorithm is that the non-subject feature is strengthened in the preprocessing process, and the non-subject feature map is extracted using the preset convolutional neural network, and the non-subject feature is finally determined through the RPN network Judgment and classification.

It is understandable that, in this embodiment, when passing the Mask After the R-CNN algorithm extracts the subject feature and non-subject feature in the image to be retrieved, it can further label the retrieved image according to the subject feature and non-subject feature. The image tag includes the subject feature. The corresponding subject label and the non-subject label corresponding to the non-subject feature. The label represents the image type corresponding to the image to be retrieved. Because of the existence of the subject and non-subject in the image to be retrieved, the image to be retrieved can be brought with There is more than one image tag.

Step S20, searching in a preset image database based on the image tag, and determining the image set corresponding to the image tag;

Further, when the image tag corresponding to the image to be retrieved is determined, it is matched with the tag corresponding to the image stored in the preset image database. If the tag matching is successful, it indicates that the image to be retrieved is the same as the preset image. The types of images in the database are the same or similar and can be put into the image collection for further feature matching. Specifically, as shown in FIG. 3, the step S20 specifically includes:

Step S21, acquiring a preset image tag corresponding to an image in a preset image database;

It is understandable that in this embodiment, a corresponding image database is pre-established, and the pictures in the preset image database have corresponding preset image tags. Specifically, in this embodiment, by acquiring a large number of samples For images, a corresponding image database is pre-established. When a preset image database is established based on the sample image, the image label corresponding to each sample image can be determined in advance, that is, the preset image label, so as to follow up with the image of the image to be retrieved. The label is matched, and the determination of the preset image label can also be realized by the corresponding feature extraction algorithm, such as Mask R-CNN algorithm.

Further, in addition to determining the preset image label, the preset image feature corresponding to the image of the preset image database can also be determined, so that the subsequent matching with the image feature of the image to be retrieved can more accurately determine the image to be retrieved The corresponding search result.

Step S22, matching the subject label and the preset image label, and determining the subject image set corresponding to the subject label;

Step S23, matching the non-subject label and the preset image label, and determine the non-subject image set corresponding to the non-subject label;

By Mask The R-CNN algorithm recognizes the subject characteristics of the image to be retrieved, and can tag the retrieved image with the corresponding subject label, and by recognizing the non-subject feature, the corresponding non-subject label can be marked. By performing the label matching degree calculation in the preset image database through the subject label, the image set corresponding to the subject feature of the image to be retrieved can be obtained, and the image set corresponding to the non subject feature can be obtained by matching with the non-subject label. In this embodiment, the calculation of the label matching degree may set a corresponding matching degree threshold, and the images in the preset image database corresponding to the result of the matching degree calculation higher than the matching degree threshold are put into the corresponding image collection.

By extracting subject features, and using subject tag information corresponding to subject features to perform image tag matching in a preset image database, reducing background interference to improve accuracy; by extracting non-subject features and using corresponding non-subjects The tag is used for retrieval and retrieval, which also reduces the scope of image search to speed up retrieval.

Step S24: Obtain the union of the subject image set and the non-subject image set, and determine it as the image set corresponding to the image tag.

The subject tag is calculated in the preset image database to determine the subject image set corresponding to the subject feature; the non subject feature is matched to obtain the image set corresponding to the non subject feature. It can be understood that these two images The set may contain the same image or different images. Therefore, the union of the two image sets is taken as the image set corresponding to the image to be retrieved, so that the retrieval result can be further determined from the union.

Step S30: Determine the image feature of the image to be retrieved based on a preset image feature extraction algorithm, and perform feature matching in the image set according to the image feature to determine a candidate image corresponding to the image to be retrieved;

When the image database is established in advance, the preset image features corresponding to the images in the image database are determined, and further, through the preset image feature extraction algorithm, the image features of the image to be retrieved are extracted, based on the image of the image to be retrieved Feature: Perform feature matching calculation with the preset image features of the images contained in the image collection, and use the image with the feature matching degree, that is, the higher matching degree, as the candidate image corresponding to the image to be retrieved, further reducing image retrieval Range.

In step S40, the candidate images are accurately sorted based on a preset sorting algorithm, and the candidate images are output in the sorted order.

Through the calculation of the matching degree of image features, several candidate images with higher matching degree are selected from the determined image set. Assuming that the number of candidate images is n, then the n candidate images are further processed with high precision Reorder, and finally output the reordered n candidate images in order.

Specifically, the preset sorting algorithm used for re-sorting n candidate images is the RANSAC algorithm. The RANSAC algorithm randomly selects 4 samples from the matching data set (candidate images) and ensures that there is no difference between the 4 samples. Collinear, calculate the homography matrix corresponding to the 4 samples, record it as model M, then use this model to test all the data, and calculate the number of data points that meet this model and the projection error. If the corresponding projection error is the smallest, Then this model is the optimal model, and the candidate images are re-sorted according to the optimal model. Finally, the candidate images are output in the determined order, which is the retrieval result of the image to be retrieved.

In this embodiment, the image to be retrieved is first obtained, and the corresponding image tag is determined based on the subject feature and non-subject feature of the image to be retrieved. The image tag is used to search in the preset image database to determine the image tag corresponding And further determine the image features of the image to be retrieved based on the preset image feature extraction algorithm, so as to perform feature matching in the determined image set according to the image feature, determine the candidate image corresponding to the image to be retrieved, and finally The candidate images are sorted accurately through a preset sorting algorithm, and the final sorted candidate images are output as the search result. The image retrieval method proposed in this application determines image tags by extracting subject features and non-subject features to perform preliminary retrieval to narrow the retrieval range, and then accurately obtain retrieval results that match the image to be retrieved through image feature matching. Finally, The search results are reordered through the sorting algorithm to provide users with more accurate search results and improve the efficiency and accuracy of image search.

Further, referring to FIG. 4, based on the above-mentioned embodiment, a second embodiment of the image retrieval method of the present application is proposed. In this embodiment, the step S30 specifically includes:

Step S31: Determine the image feature of the image to be retrieved based on a preset image feature extraction algorithm;

When the image database is established in advance, the preset image features corresponding to the images in the image database are determined; when feature matching is required, the image features of the images to be retrieved are determined based on the preset image feature extraction algorithm. In this embodiment In the above two types of image feature extraction, the same image feature extraction algorithm can be used, specifically, it refers to the SIFT algorithm. Image feature extraction mainly extracts multiple representative keywords (feature points) from the picture to form one Dictionary, and then count the number of keywords appearing in each picture to obtain the feature vector of the picture. The process of constructing the preset image features in the preset image database is as follows:

First, obtain a large number of sample images, and perform image feature extraction on them through SIFT, determine the image features corresponding to the sample image, and represent the corresponding sample image with a vector to obtain the image feature vector of the sample image. At the same time, the k-means clustering method is used to cluster all image features, and image features with the same feature are clustered into the same category. Specifically, k feature points are randomly selected as cluster centers, and other features are selected. The distance between a point and K central points, associate it with the closest central point, all points associated with the same central point form a cluster, and calculate the mean of each group of clusters to cluster the group The associated center point moves to the position of the average value, and then repeats the steps of calculating the distance between other feature points and K center points so as to put them into the associated cluster to change the position of the cluster center point until The position of the center point does not change anymore, so that all the feature points are divided into different clusters.

Specifically, the process of using SIFT for image feature extraction is as follows:

The image that needs image feature extraction is convolved with a two-dimensional Gaussian function to obtain multiple Gaussian images with different σ values. These multiple Gaussian images with different σ values constitute the scale space of the image, indicating that it is at different scales In the image below, the purpose of constructing the scale space is to detect the feature points that exist at different scales.

In this embodiment, in order to improve the accuracy of feature point extraction, the image can be downsampled first to obtain image pyramids at different resolutions, and then Gaussian convolution is performed on each layer of the image. In this way, the original image Each layer of the pyramid has only one image, and after Gaussian convolution, images of different scales are added to each layer.

Furthermore, to optimize the images of each layer in the above image pyramid, DOG (Difference of Gaussian (Gaussian difference) to construct a Gaussian difference image, specifically includes: after acquiring images at different scales in each layer of the image pyramid, subtracting them with adjacent images, and obtaining all images to reconstruct the pyramid is the DOG pyramid. After the DOG pyramid is obtained, the DOG extreme point is further searched, that is, each pixel point is compared with the surrounding pixels. When it is greater than or less than all adjacent points, it is an extreme point, specifically, adjacent points Includes pixels in the same scale space where the pixel is located and pixels in adjacent scale spaces. Through the above process, the detection and extraction of the feature points of the image are completed, and the image features of the image to be retrieved can be determined.

Step S32: Obtain the preset image features corresponding to the images in the image collection from the preset image database, and perform feature evaluation on the image features and the preset image features corresponding to the images in the image collection The matching calculation determines a matching score, so as to determine the candidate image corresponding to the image to be retrieved based on the matching score.

Obtain the preset image features corresponding to the images in the image collection from the preset image database, and perform feature matching calculations between the image features of the image to be retrieved and the preset image features in the image collection to determine the image to be retrieved A feature matching score corresponding to the image feature of and each preset image feature in the image set, and the candidate image corresponding to the image to be retrieved is determined based on the matching score.

Specifically, after the matching score is determined, a matching list can be established according to the level of the matching scores, the determined matching scores are sorted from high to low, and the preset number from the first in the matching list is selected in order The matching score of, takes several images in the corresponding image set as candidate images, that is, the search result corresponding to the image to be retrieved, and further uses the sorting algorithm to readjust the ranking of the candidate images to make the output search The result is more accurate.

In this embodiment, after a preliminary search is performed through the image tags to determine the corresponding image set, the image feature of the image to be retrieved is determined by the preset image feature extraction algorithm, and the image feature corresponding to the image in the image set is determined Perform feature matching calculation on image features, determine the matching score, and finally determine the candidate images corresponding to the image to be retrieved, and perform high-precision sorting on the candidate images. The sorted order is used as the final output retrieval result, which greatly improves the image The accuracy of retrieval.

In addition, the embodiment of the present application also provides an image retrieval device.

Referring to FIG. 5, FIG. 5 is a schematic diagram of the functional modules of the first embodiment of the image retrieval device of this application.

In this embodiment, the image retrieval device includes:

The feature extraction module 10 is configured to obtain the image to be retrieved, and perform subject feature extraction and non-subject feature extraction on the image to be retrieved, so as to determine the image label corresponding to the image to be retrieved based on the extracted subject feature and non-subject feature;

The tag retrieval module 20 is configured to retrieve in a preset image database based on the image tag, and determine the image set corresponding to the image tag;

The feature matching module 30 is configured to determine the image feature of the image to be retrieved based on a preset image feature extraction algorithm, and perform feature matching in the image set according to the image feature to determine the device corresponding to the image to be retrieved Select image

The image sorting module 40 is configured to accurately sort the candidate images based on a preset sorting algorithm, and output the candidate images in the sorted order.

Further, the feature extraction module 10 specifically includes:

The request receiving unit is configured to obtain the image to be retrieved corresponding to the image retrieval request when the image retrieval request is received;

The feature extraction unit is configured to perform subject feature extraction and non-subject feature extraction on the image to be retrieved based on a preset feature extraction algorithm to determine subject features and non-subject features corresponding to the image to be retrieved, and based on the subject The feature and the non-subject feature determine the image tag corresponding to the image to be retrieved.

Further, the tag retrieval module 20 specifically includes:

A preset image tag acquiring unit, configured to acquire a preset image tag corresponding to an image in a preset image database;

A subject label matching unit, configured to match the subject label with the preset image label, and determine the subject image set corresponding to the subject label;

The non-subject tag matching unit is configured to match the non-subject tag with the preset image tag, and determine the non-subject image set corresponding to the non-subject tag;

The image collection acquisition unit is configured to acquire the union of the subject image collection and the non-subject image collection, and determine it as the image collection corresponding to the image tag.

Further, the image retrieval device further includes:

An image database establishment unit for acquiring sample images and establishing a preset image database based on the sample images;

The preset image tag determination unit is used to determine the preset image tag corresponding to the image in the preset image database, and determine the preset image tag corresponding to the image in the preset image database based on the preset image feature extraction algorithm. Set image characteristics.

Further, the feature matching module 30 specifically includes:

An image feature determining unit, configured to determine the image feature of the image to be retrieved based on a preset image feature extraction algorithm;

The feature degree matching unit is configured to obtain a preset image feature corresponding to an image in the image collection from the preset image database, and compare the image feature with the preset image corresponding to the image in the image collection The feature performs feature matching calculation to determine a matching score, so as to determine the candidate image corresponding to the image to be retrieved based on the matching score.

Further, the feature matching unit further includes:

The matching score sorting subunit is used to sort the matching scores from high to low to obtain a matching list, and select a preset number of matching scores from the top of the matching list, and set the preset The image in the image set corresponding to the number of matching scores is determined as the candidate image corresponding to the image to be retrieved.

Among them, each module in the above-mentioned image retrieval device corresponds to each step in the above-mentioned image retrieval method embodiment, and its functions and implementation processes will not be repeated here.

In addition, the embodiment of the present application also proposes a computer-readable storage medium, and the computer-readable storage medium may be a non-volatile readable storage medium.

The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the image retrieval method described above are realized.

For the method implemented when the computer-readable instruction is executed, please refer to the various embodiments of the image retrieval method of this application, which will not be repeated here.

It should be noted that in this article, the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, It also includes other elements that are not explicitly listed, or elements inherent to the process, method, article, or system. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, article or system that includes the element.

The serial numbers of the foregoing embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

Through the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiments can be implemented by means of software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is better.的实施方式。 Based on this understanding, the technical solution of this application essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM/RAM) as described above. , Magnetic disk, optical disk), including several instructions to make a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the method described in each embodiment of the present application.

The above are only preferred embodiments of this application, and do not limit the scope of this application. Any equivalent structure or equivalent process transformation made using the content of the description and drawings of this application, or directly or indirectly used in other related technical fields , The same reason is included in the scope of patent protection of this application.

Claims

An image retrieval method, wherein the image retrieval method includes the following steps:

Acquiring the image to be retrieved, and performing subject feature extraction and non-subject feature extraction on the image to be retrieved, so as to determine the image label corresponding to the image to be retrieved based on the extracted subject feature and non-subject feature;

Searching in a preset image database based on the image tag, and determining the image set corresponding to the image tag;

Determine the image feature of the image to be retrieved based on a preset image feature extraction algorithm, and perform feature matching in the image set according to the image feature to determine the candidate image corresponding to the image to be retrieved;

The candidate images are accurately sorted based on a preset sorting algorithm, and the candidate images are output in the sorted order.
5. The image retrieval method according to claim 1, wherein the image to be retrieved is acquired, and subject feature extraction and non-subject feature extraction are performed on the subject image to be retrieved, so as to determine the subject feature and non-subject feature The steps of the image label corresponding to the image to be retrieved include:

When an image retrieval request is received, acquiring the image to be retrieved corresponding to the image retrieval request;

Perform subject feature extraction and non-subject feature extraction on the image to be retrieved based on a preset feature extraction algorithm to determine the subject feature and non-subject feature corresponding to the image to be retrieved, and based on the subject feature and the non-subject feature The feature determines the image tag corresponding to the image to be retrieved.
3. The image retrieval method according to claim 2, wherein the image tag includes a subject label corresponding to the subject feature and a non-subject label corresponding to the non-subject feature.
8. The image retrieval method according to claim 1, wherein the step of performing retrieval in a preset image database based on the image tag, and determining the image set corresponding to the image tag comprises:

Acquiring the preset image tag corresponding to the image in the preset image database;

Matching the subject label and the preset image label to determine a subject image set corresponding to the subject label;

Matching the non-subject label and the preset image label to determine a non-subject image set corresponding to the non-subject label;

A union of the subject image set and the non-subject image set is acquired, and the union is determined as the image set corresponding to the image tag.
5. The image retrieval method according to claim 1, wherein before the step of performing retrieval in a preset image database based on the image tags and determining the image set corresponding to the image tags, the method further comprises:

Acquiring sample images, and establishing a preset image database based on the sample images;

The preset image tag corresponding to the image in the preset image database is determined, and the preset image feature corresponding to the image in the preset image database is determined based on a preset image feature extraction algorithm.
The image retrieval method according to claim 1, wherein the image feature extraction algorithm based on a preset image feature determines the image feature of the image to be retrieved, and performs feature matching in the image collection according to the image feature to determine The steps of the candidate image corresponding to the image to be retrieved include:

Determining the image feature of the image to be retrieved based on a preset image feature extraction algorithm;

Acquiring preset image features corresponding to the images in the image collection from the preset image database, and performing feature matching calculations on the image features and the preset image features corresponding to the images in the image collection, The matching score is determined, so as to determine the candidate image corresponding to the image to be retrieved based on the matching score.
8. The image retrieval method according to claim 6, wherein the step of determining the candidate image corresponding to the image to be retrieved based on the matching score comprises:

Sort the matching scores from high to low to obtain a matching list, and select a preset number of matching scores from the top of the matching list, and assign the preset number of matching scores to all The images in the image set are determined to be candidate images corresponding to the image to be retrieved.
An image retrieval device, wherein the image retrieval device includes:

The feature extraction module is used to obtain the image to be retrieved, and perform subject feature extraction and non-subject feature extraction on the image to be retrieved, so as to determine the image label corresponding to the image to be retrieved based on the extracted subject feature and non-subject feature;

The tag retrieval module is configured to perform retrieval in a preset image database based on the image tag, and determine the image set corresponding to the image tag;

The feature matching module is used to determine the image feature of the image to be retrieved based on a preset image feature extraction algorithm, and perform feature matching in the image set according to the image feature to determine the candidate corresponding to the image to be retrieved image;

The image sorting module is configured to accurately sort the candidate images based on a preset sorting algorithm, and output the candidate images in the sorted order.
8. The image retrieval device according to claim 8, wherein the feature extraction module specifically comprises:

The request receiving unit is configured to obtain the image to be retrieved corresponding to the image retrieval request when the image retrieval request is received;

The feature extraction unit is configured to perform subject feature extraction and non-subject feature extraction on the image to be retrieved based on a preset feature extraction algorithm to determine subject features and non-subject features corresponding to the image to be retrieved, and based on the subject The feature and the non-subject feature determine the image tag corresponding to the image to be retrieved.
8. The image retrieval device according to claim 8, wherein the tag retrieval module specifically comprises:

A preset image tag acquiring unit, configured to acquire a preset image tag corresponding to an image in a preset image database;

A subject label matching unit, configured to match the subject label with the preset image label, and determine the subject image set corresponding to the subject label;

The non-subject tag matching unit is configured to match the non-subject tag with the preset image tag, and determine the non-subject image set corresponding to the non-subject tag;

The image collection acquisition unit is configured to acquire the union of the subject image collection and the non-subject image collection, and determine it as the image collection corresponding to the image tag.
8. The image retrieval device according to claim 8, wherein the image retrieval device further comprises:

An image database establishment unit for acquiring sample images and establishing a preset image database based on the sample images;

The preset image tag determination unit is used to determine the preset image tag corresponding to the image in the preset image database, and determine the preset image tag corresponding to the image in the preset image database based on the preset image feature extraction algorithm. Set image characteristics.
8. The image retrieval device according to claim 8, wherein the feature matching module specifically comprises:

An image feature determining unit, configured to determine the image feature of the image to be retrieved based on a preset image feature extraction algorithm;

The feature degree matching unit is configured to obtain a preset image feature corresponding to an image in the image collection from the preset image database, and compare the image feature with the preset image corresponding to the image in the image collection The feature performs feature matching calculation to determine a matching score, so as to determine the candidate image corresponding to the image to be retrieved based on the matching score.
The image retrieval device according to claim 12, wherein the image retrieval device comprises: a memory, a processor, and computer-readable instructions stored on the memory and executable on the processor, and the computer can The following steps are implemented when the read instruction is executed by the processor:

Acquiring the image to be retrieved, and performing subject feature extraction and non-subject feature extraction on the image to be retrieved, so as to determine the image label corresponding to the image to be retrieved based on the extracted subject feature and non-subject feature;

Searching in a preset image database based on the image tag, and determining the image set corresponding to the image tag;

Determine the image feature of the image to be retrieved based on a preset image feature extraction algorithm, and perform feature matching in the image set according to the image feature to determine the candidate image corresponding to the image to be retrieved;

The candidate images are accurately sorted based on a preset sorting algorithm, and the candidate images are output in the sorted order.
The image retrieval device according to claim 13, wherein the image to be retrieved is acquired, and subject feature extraction and non-subject feature extraction are performed on the subject image to be retrieved, so as to determine the The steps of the image label corresponding to the image to be retrieved include:

When an image retrieval request is received, acquiring the image to be retrieved corresponding to the image retrieval request;

Perform subject feature extraction and non-subject feature extraction on the image to be retrieved based on a preset feature extraction algorithm to determine the subject feature and non-subject feature corresponding to the image to be retrieved, and based on the subject feature and the non-subject feature The feature determines the image tag corresponding to the image to be retrieved.
The image retrieval device according to claim 12, wherein the step of performing retrieval in a preset image database based on the image tag, and determining the image set corresponding to the image tag comprises:

Acquiring the preset image tag corresponding to the image in the preset image database;

Matching the subject label and the preset image label to determine a subject image set corresponding to the subject label;

Matching the non-subject label and the preset image label to determine a non-subject image set corresponding to the non-subject label;

A union of the subject image set and the non-subject image set is acquired, and the union is determined as the image set corresponding to the image tag.
The image retrieval device according to claim 12, wherein, before the step of performing retrieval in a preset image database based on the image tags, and determining the image set corresponding to the image tags, the method further comprises:

Acquiring sample images, and establishing a preset image database based on the sample images;

The preset image tag corresponding to the image in the preset image database is determined, and the preset image feature corresponding to the image in the preset image database is determined based on a preset image feature extraction algorithm.
A computer-readable storage medium, wherein computer-readable instructions are stored on the computer-readable storage medium, and when the computer-readable instructions are executed by a processor, the following steps are implemented:

Acquiring the image to be retrieved, and performing subject feature extraction and non-subject feature extraction on the image to be retrieved, so as to determine the image label corresponding to the image to be retrieved based on the extracted subject feature and non-subject feature;

Searching in a preset image database based on the image tag, and determining the image set corresponding to the image tag;

Determine the image feature of the image to be retrieved based on a preset image feature extraction algorithm, and perform feature matching in the image set according to the image feature to determine the candidate image corresponding to the image to be retrieved;

The candidate images are accurately sorted based on a preset sorting algorithm, and the candidate images are output in the sorted order.
The computer-readable storage medium according to claim 17, wherein said acquiring the image to be retrieved, and performing subject feature extraction and non-subject feature extraction on the retrieved image, so as to determine based on the extracted subject feature and non-subject feature The steps of the image label corresponding to the image to be retrieved include:

When an image retrieval request is received, acquiring the image to be retrieved corresponding to the image retrieval request;

Perform subject feature extraction and non-subject feature extraction on the image to be retrieved based on a preset feature extraction algorithm to determine the subject feature and non-subject feature corresponding to the image to be retrieved, and based on the subject feature and the non-subject feature The feature determines the image tag corresponding to the image to be retrieved.
17. The computer-readable storage medium according to claim 17, wherein the step of searching in a preset image database based on the image tag, and determining the image set corresponding to the image tag comprises:

Acquiring the preset image tag corresponding to the image in the preset image database;

Matching the subject label and the preset image label to determine a subject image set corresponding to the subject label;

Matching the non-subject label and the preset image label to determine a non-subject image set corresponding to the non-subject label;

A union of the subject image set and the non-subject image set is acquired, and the union is determined as the image set corresponding to the image tag.
17. The computer-readable storage medium of claim 17, wherein before the step of searching in a preset image database based on the image tags and determining the image set corresponding to the image tags, the method further comprises:

Acquiring sample images, and establishing a preset image database based on the sample images;

The preset image tag corresponding to the image in the preset image database is determined, and the preset image feature corresponding to the image in the preset image database is determined based on a preset image feature extraction algorithm. To