WO2021196551A1

WO2021196551A1 - Image retrieval method and apparatus, computer device, and storage medium

Info

Publication number: WO2021196551A1
Application number: PCT/CN2020/119714
Authority: WO
Inventors: 炊文伟; 畅婧文
Original assignee: 北京旷视科技有限公司
Priority date: 2020-03-31
Filing date: 2020-09-30
Publication date: 2021-10-07
Also published as: CN111522969A

Abstract

Provided are an image retrieval method and apparatus, a computer device, and a storage medium. The method comprises: receiving a retrieval instruction from a user (S202); acquiring an image to be retrieved, wherein said image includes multiple targets of different detection types (S204); by means of a target detection algorithm, performing target detection on said image to obtain a detection type corresponding to each detected target and a distribution area corresponding to each detected target (S206); according to the distribution area corresponding to each detected target, performing image matting in said image to obtain a sub-image corresponding to each detected target (S208); with regard to the detected target of each detection type, retrieving, in an image base library corresponding to the detection type of the detected target, a base library sub-image matching the sub-image corresponding to the detected target, wherein the image base library stores the base library sub-image, and the base library sub-image includes a target with the same detection type as the detected target (S210); and according to the base library sub-image matching the sub-image corresponding to each detected target, obtaining a target image corresponding to the retrieval instruction (S212). By means of the method, the image retrieval efficiency can be improved.

Description

Image retrieval method, device, computer equipment and storage medium

Technical field

This application relates to the technical field of image data processing, in particular to an image retrieval method, device, computer equipment and storage medium.

Background technique

With the development of multimedia technology, image retrieval technology emerged. Image retrieval refers to the process of retrieving similar images in the database through a piece of image to be retrieved. Since the emergence of image retrieval technology, three important branches have been formed: text-based image retrieval, content-based image retrieval, and semantic-based image retrieval.

In the traditional image retrieval method, the user first needs to manually intercept the sub-images of each target in the image to be retrieved, and then the user inputs the target sub-images into the retrieval system to realize the image retrieval function. However, in the retrieval process, the user can only input one target sub-picture at a time, and the retrieval system can only complete retrieval for one target at a time. When there are multiple targets, the retrieval system needs to perform multiple image retrieval tasks, resulting in longer retrieval time and lower retrieval efficiency.

Summary of the invention

Based on this, it is necessary to provide an image retrieval method, device, computer equipment, and storage medium that can improve the efficiency of image retrieval in response to the above technical problems.

An image retrieval method, the method includes:

Receiving user's retrieval instructions;

Acquiring an image to be retrieved; the image to be retrieved includes multiple targets of different detection types;

Target detection is performed on the image to be retrieved by the target detection algorithm, and the detection type corresponding to each detected target and the distribution area corresponding to each detected target are obtained; the distribution area is the detected target in the to-be-detected The area in the image;

Matting in the image to be retrieved according to the distribution area corresponding to each detected target to obtain a sub-image corresponding to each detected target;

For the detected targets of each detection type, in the image base database corresponding to the detection type of the detected target, search the base library sub-images that match the sub-images corresponding to the detected targets; in the image base library Storing a base library sub-picture, the base library sub-picture contains the same target as the detection type of the detected target;

The target image corresponding to the retrieval instruction is obtained according to the base library sub-image that matches the sub-image corresponding to each detected target.

In one of the embodiments, the user's search instruction is an associated search instruction, and the associated search instruction includes a first detection type and a second detection type; the subgraph corresponding to the detected target of the first detection type matches The base library sub-image of is the first base library sub-image, the base library sub-image that matches the sub-image corresponding to the detection target of the second detection type is the second base library sub-image, and the image base library is stored in association There are the base library sub-picture and the image identifier of the panorama corresponding to the base library sub-picture;

The obtaining the target image corresponding to the retrieval instruction according to the base library sub-image matching the sub-image corresponding to each detected target includes:

If the first base library sub-image and the second base library sub-image are associated with a first associated panoramic image with the same image identifier, then the first associated panoramic image is used as the target image corresponding to the retrieval instruction, Or use the first associated panoramic image, the first base library sub-image, and the second base library sub-image as the target image corresponding to the retrieval instruction.

In one of the embodiments, the method further includes:

The detection type corresponding to each detected target is displayed to the user, so that the user selects the first detection type and the second detection type from the detection types corresponding to each detected target.

In one of the embodiments, the method further includes:

According to the first similarity between the sub-image corresponding to the detected target of the first detection type and the first base library sub-image, the sub-image corresponding to the detected target of the second detection type is compared with the second The second degree of similarity of the base library sub-images obtains the degree of similarity of each target image corresponding to the retrieval instruction.

In one of the embodiments, the retrieval instruction includes the identification of the detected target used for retrieval; the method further includes:

Display the distribution area corresponding to each detected target or the sub-image corresponding to each detected target to the user, so that the user selects the detected target for image retrieval from the detected targets ；

For the detection targets of each detection type, in the image base library corresponding to the detection type of the detection target, searching the base library sub-images that match the sub-images corresponding to the detected targets includes:

For the third detection target corresponding to the identification of the detection target used for image retrieval, search for the third detection target in the image base corresponding to the detection type of the third detection target The third base library sub-image matched by the corresponding sub-image.

In one of the embodiments, the retrieval instruction includes at least two identifications of the detected targets used for retrieval; the detected targets of each detection type are displayed on the bottom of the image corresponding to the detection types of the detected targets. In the library, retrieving the base library subgraph matching the subgraph corresponding to the detected target, further includes:

For the fourth detection target corresponding to the identification of the detection target used for image retrieval, search for the fourth detection target in the image base corresponding to the detection type of the fourth detection target The fourth base library subgraph matched by the corresponding subgraph;

If the third base library sub-image and the fourth base library sub-image are associated with a second associated panorama with the same image identifier, then the second associated panorama is used as the target image corresponding to the retrieval instruction, Or use the second associated panoramic image, the third base library sub-image, and the fourth base library sub-image as the target image corresponding to the retrieval instruction.

In one of the embodiments, in the image base library corresponding to the detection type of the detected target, retrieving the base library sub-image that matches the sub-image corresponding to the detected target includes:

In the image base library corresponding to the detection type of the detected target, filter the base library sub-image set that meets the set time and/or set location input by the user;

In the set of base library subgraphs, search base library subgraphs matching the subgraph corresponding to the detected target.

In one of the embodiments, before the screening of the set of base library submaps that meet the set time and/or set location input by the user, the method further includes:

Acquiring the preset time and the preset duration input by the user;

Use a time period including the preset time as the set time, and the duration of the set time is the preset duration; or;

Acquiring the start time and end time input by the user;

Use the time period between the start time and the end time as the set time; or;

Acquire the start date and end date, daily start time and daily end time entered by the user,

In each date between the start date and the end date, a time period between the start time of the day and the end time of the day is taken as the set time.

An image retrieval device, the device includes:

The instruction receiving module is used to receive the user's retrieval instruction;

An image acquisition module for acquiring an image to be retrieved; the image to be retrieved contains multiple targets of different detection types;

The target detection module is used to perform target detection on the image to be retrieved through a target detection algorithm to obtain the detection type corresponding to each detected target and the distribution area corresponding to each detected target; the distribution area is the detected target The area where the target is located in the image to be detected;

A matting module for matting in the image to be retrieved according to the distribution area corresponding to each detected target to obtain a sub-image corresponding to each detected target;

The image retrieval module is used to retrieve, in the image base database corresponding to the detection type of the detected target, the base library sub-image that matches the sub-image corresponding to the detected target for the detected targets of each detection type; The image base library stores base library sub-images, and the base library sub-images include targets of the same detection type as the detected targets;

The image determination module is used to obtain the target image corresponding to the retrieval instruction according to the base library sub-image matched with the sub-image corresponding to each detected target.

A computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when the processor executes the computer program:

Receiving user's retrieval instructions;

A computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

Receiving user's retrieval instructions;

The above-mentioned image retrieval method, device, computer equipment and storage medium can realize target detection for multiple targets of different detection types in the image to be retrieved at one time, so as to obtain the detection type corresponding to each detected target and the corresponding distribution of each detected target Region, and for each detection target of each detection type, the target image is retrieved from the image base corresponding to the detection type of the detection target. In the embodiment of the application, the user does not need to manually intercept the sub-images of each target in the image to be retrieved and input each target sub-image into the retrieval system. The user only needs to upload the image once to complete the retrieval of all detected targets of detection types at one time. Thus, the complexity of user operations is effectively reduced, and retrieval efficiency is improved.

Description of the drawings

Figure 1 is an application environment diagram of an image retrieval method in an embodiment;

Figure 2 is a schematic flowchart of an image retrieval method in an embodiment;

FIG. 3 is an effect diagram showing the sub-picture corresponding to each detected target to the user in an embodiment;

Fig. 4 is an effect diagram showing the target image to the user in an embodiment;

Fig. 5 is a schematic flowchart of an image retrieval method in another embodiment;

FIG. 6 is a schematic flowchart of an image retrieval method in another embodiment;

FIG. 7 is a schematic flow diagram of screening a collection of base library sub-images that meet the time and location search conditions set by the user in an embodiment;

Figure 8 is a structural block diagram of an image retrieval device in an embodiment;

Fig. 9 is an internal structure diagram of a computer device in an embodiment.

Detailed ways

In order to make the purpose, technical solutions, and advantages of the application more clear, the application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

In an exemplary embodiment, the image retrieval method involved in the present application is applied to an image retrieval device for illustration. It is understandable that, as shown in FIG. 1, the image retrieval device may be a terminal 102, a server 104, or a system including the terminal 102 and the server 104, and is implemented through the interaction between the terminal 102 and the server 104. The terminal 102 may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices. The server 104 may be implemented by an independent server or a server cluster composed of multiple servers.

In an exemplary embodiment, as shown in FIG. 2, an image retrieval method is provided. The image retrieval device is the server in FIG. 1 as an example for description, which can be specifically implemented by the following steps:

Step S202: Receive a search instruction from the user.

Among them, the user's retrieval instruction is generated by the user based on the image retrieval requirement by performing corresponding user operations on the terminal.

Specifically, the image retrieval device may receive a user's retrieval instruction sent by the terminal.

Step S204: Obtain the image to be retrieved.

Among them, the image to be retrieved contains multiple targets of different detection types. The detection type is the target type that can be detected by the target detection algorithm used by the image retrieval device. For example, the target detection algorithm can detect the face type target, the human body type target and the motor vehicle type target, but the cat face, telephone pole type target cannot be detected, and the detection type is the face type, the human body type and the motor vehicle type. , Cat faces and telephone poles are not the detection type.

Specifically, the image retrieval device acquires the image to be retrieved. Wherein, the image to be retrieved may be an image input by the user through the terminal, and the image contains multiple detection types of targets. For example, the image to be retrieved A contains a face, a human body, and a non-motor vehicle, that is, it contains a person. Three types of targets are detected: face, human body, and non-motor vehicle.

In step S206, target detection is performed on the image to be retrieved by the target detection algorithm, and the detection type corresponding to each detected target and the distribution area corresponding to each detected target are obtained.

Among them, the target detected by the target detection algorithm is called the detected target, and the target detection algorithm can detect multiple detection types of detected targets at one time. The distribution area is the area where the detected target is located in the image to be detected, that is, the position of the target in the image to be detected.

Specifically, the image retrieval device performs target detection on the image to be retrieved through a preset target detection algorithm, and obtains the detection type corresponding to each detected target and the distribution area corresponding to each detected target. For example, target detection is performed on the image A to be retrieved, and a human face, a human body, and a non-motor vehicle can be detected at the same time. If the target detection algorithm cannot detect this type of target due to some reasons (such as occlusion of the target, unclear image quality, etc.), then this type of target does not belong to the aforementioned "detected target".

Optionally, after acquiring the distribution area corresponding to each detected target, the image retrieval device displays the distribution area corresponding to each detected target on the image to be detected to the user, so that the user knows which targets have been detected .

Step S208: Cut out the image in the image to be retrieved according to the distribution area corresponding to each detected target to obtain a sub-image corresponding to each detected target.

Specifically, the image retrieval device performs matting in the image to be retrieved for each detected target according to the distribution area corresponding to each detected target, and obtains a sub-image corresponding to each detected target for subsequent retrieval steps. Optionally, the distribution area is represented by the coordinates of the upper left, lower left, upper right, and lower right four vertices of the detection box (boundingbox) where the detected target is located. picture. Optionally, for each detected target, the target sub-image needs to be intercepted from the original image (that is, the image to be retrieved).

Optionally, after obtaining the sub-images corresponding to each detected target, the image retrieval device displays the sub-images corresponding to each detected target to the user, so that the user knows which targets have been detected and will be used for subsequent retrieval. Among them, the effect picture can be seen in Figure 3.

Step S210: For the detected targets of each detection type, in the image base library corresponding to the detection type of the detected target, search for base library sub-images that match the sub-image corresponding to the detected target.

Among them, the image base library stores base library sub-images, and the base library sub-images include targets with the same detection type as the detection type of the detected target.

Specifically, the image retrieval device first searches for a matching image base based on the detection type of the detected target, and then searches the image base matching the detection type of the detected target for the sub-image that matches the detected target. Bottom library submap. For example, in the image A to be retrieved, the detected targets include face targets, human targets, and non-motorized vehicle targets, and the detection types corresponding to the detected targets are face type, human body type, and non-motorized vehicle type, respectively. The image detection equipment finds the image database of the motor vehicle type according to the type of motor vehicle, the image database of the non-motor vehicle type according to the type of non-motor vehicle, and the image database of the human body type according to the type of human body. In various types of image bases, for example, in the image base of the face type, search the base sub-images that match the face sub-images, and in the non-motor vehicle type image bases, search the sub-images that match the non-motor vehicle sub-images Base library sub-image, in the image base library of the human body type, search for the base library sub-image matching the human body sub-image.

Optionally, the matching method of the target sub-image and the base library sub-image may be as follows: firstly, extract the image features of the target sub-image and the image features of the base library sub-image respectively, and then determine the image features of the target sub-image and the base library sub-image. For the similarity between the image features of the images, the base library sub-image with the similarity greater than the preset threshold is used as the base library sub-image matching the sub-image corresponding to the detected target. Alternatively, the similarity can be determined by calculating the distance between feature vectors.

In step S212, the target image corresponding to the retrieval instruction is obtained according to the base library sub-image that matches the sub-image corresponding to each detected target.

Specifically, the image retrieval device may directly use the base library sub-images matching the sub-images corresponding to each detected target as the target image corresponding to the retrieval instruction, or may directly use the base library sub-images matching the sub-images corresponding to each detected target as the target image. Further screening and processing are performed to obtain the target image corresponding to the retrieval instruction. For example, the panoramic image corresponding to the base library sub-image (the base library sub-image is obtained by target detection and matting in the panoramic image) can be used as the target image.

Continuing the previous example, optionally, for the image A to be detected, the image retrieval device can retrieve the face target, the human target, and the non-motorized vehicle target at one time, and obtain the face retrieval result, the human body retrieval result, and the non-motorized vehicle target. As a result of the search, the set of the three is used as the target image corresponding to the search instruction.

Optionally, the image retrieval device can arrange the multiple matched base library sub-images in descending order according to the similarity between the sub-image corresponding to the detected target and the matched base library sub-image, and select the top N bits as the search command The corresponding target image. The rendering of the target image to the user can be seen in Figure 4.

It should be noted that the embodiment of the present application does not limit the sequence of the steps. For example, step S202 may follow step S204, S206, or S208.

In the above image retrieval method, multiple targets of different detection types in the image to be retrieved can be detected at one time, so as to obtain the detection type corresponding to each detected target and the distribution area corresponding to each detected target, and then for each detection type The target image is retrieved in the image base corresponding to the detection type of the detected target. In the embodiment of the application, the user does not need to manually intercept the sub-images of each target in the image to be retrieved and input each target sub-image into the retrieval system. The user only needs to upload the image once to complete the retrieval of all detected targets of detection types at one time. Thus, the complexity of user operations is effectively reduced, and retrieval efficiency is improved.

In an exemplary embodiment, the user's search instruction is an associated search instruction, and the associated search instruction includes a first detection type and a second detection type. The base library subgraph matching the subgraph corresponding to the detection target of the first detection type is the first base library subgraph, and the base library subgraph matching the subgraph corresponding to the detection target of the second detection type is the second base Library map. The image base library is associated with the base library sub-image and the image identifier of the panorama corresponding to the base library sub-image.

To continue the previous example, the image A to be retrieved contains the face F, the human body P and the non-motor vehicle V. In the previous embodiment, the target image corresponding to the retrieval instruction contains the face F1 retrieved by the face F, and retrieved by the human body P The human body P1 and the non-motor vehicle V1 retrieved by the non-motor vehicle V. The target image can be a sub-image of the base library where the human face F1, the human body P1 and the non-motorized vehicle V1 are located (see FIG. 4), or it can be a panoramic image where the human face F1, the human body P1 and the non-motorized vehicle V1 are located. Since the face F1 is only retrieved from the face F, it is impossible to know whether the human body is photographed in the panoramic image where the face F1 is located, and whether the photographed human body matches the human body P. However, in some cases, the user wants to retrieve such a target image, since it contains a human face F2 matching the human face F and a human body P2 matching the human body P in the same image. Or, the user wants to retrieve such a target image, since it contains a human body P2 matching the human body P and a non-motor vehicle V2 matching the non-motor vehicle V in the same image. That is, the user hopes that there are at least two detection types of targets in the same target image, and the targets of each detection type are matched with the corresponding types of targets in the image to be retrieved. At this time, the user can give an associated retrieval instruction including the first detection type and the second detection type. For example, when the user hopes that since the same target image contains a human face matching the human face F and a human body matching the human body P, an associated retrieval instruction including the face detection type and the human body retrieval type can be given. Correspondingly, after the image retrieval device obtains F1, F2 matching the human face F and P1, P2 matching the human body P, it will not return F1, F2, P1, P2 directly to the user as the target image, but will It is further judged whether F1, F2 and P1, P2 appear in the same image, that is, whether F1, F2 and P1, P2 correspond to the same panoramic image.

Based on this, referring to FIG. 5, step S212 can be specifically implemented through the following steps:

Step S212a: If the first associated panoramic image with the same image identifier is associated with the first base library sub-image and the second base library sub-image, then the first associated panoramic image is used as the target image corresponding to the retrieval instruction, or the first associated The panoramic image, the first base library sub-image, and the second base library sub-image are used as the target image corresponding to the retrieval instruction.

Specifically, the first base library sub-image and the second base library sub-image are associated with the first associated panorama with the same image identifier, indicating that the first base library sub-image and the second base library sub-image are both from the first associated panorama The image is cut out in the figure, that is, the target of the first detection type in the first base library submap and the target of the second detection type in the second base library submap are located in the same panoramic image. The image retrieval device can retrieve a panoramic image containing two detection types at the same time by determining the relationship between the base library sub-image and the panoramic image, so that it is easier to determine the relationship between different detection types of targets in different base library sub-images.

For example, in the image A to be retrieved, if the target in the first base library sub-image is the face F2, the target in the second base library sub-image is the non-motorized vehicle V2, if the first base library sub-image and the second base sub-image are The panorama corresponding to the base library sub-image has the same image identifier, indicating that the face F2 and the non-motor vehicle V2 were originally located in the same first associated panorama B. At this time, due to the correlation between the base library sub-image and the panorama , The first associated panoramic image B can also be used as the target image corresponding to the retrieval instruction.

In the embodiment of the present application, such a target image can be obtained. There are at least two detection types of targets in the same target image, and each detection type of target matches the corresponding type of target in the image to be retrieved. In this way, a richer and multi-dimensional search method can be provided, and the user experience can be improved.

In an exemplary embodiment, the method for acquiring the first detection type and the second detection type may be: displaying the detection type corresponding to each detected target to the user, so that the user can select from the detection type corresponding to each detected target The first detection type and the second detection type.

Specifically, after the image retrieval device obtains the detection type corresponding to each detected target, the image retrieval device displays the detection type corresponding to each detected target to the user. The display method can be displayed to the user through the display screen of the image retrieval device or through It is first transmitted to the terminal and then displayed to the user through the display screen of the terminal, so that the user selects the first detection type and the second detection type from the detection types corresponding to each detected target. Optionally, the user can manually select the first detection type and the second detection type according to the detection type prompt option. Optionally, the user can also select one detection type or more than two detection types.

In the embodiment of the present application, the detection type corresponding to the detected target is displayed to the user, so that the user can select the target of the detection type with the retrieval requirement, and the retrieved target image can be quickly obtained. Moreover, this can prevent all detection types of targets from being retrieved, so that the amount of retrieved target image data is too large, thereby increasing the time for the user to view the retrieval results, and undoubtedly increasing the amount of tasks for the user. Therefore, the efficiency and accuracy of image retrieval can be further improved.

In an exemplary embodiment, the method further includes the following steps:

Step S2112, according to the first similarity between the subgraph corresponding to the detected target of the first detection type and the first base library subgraph, and the difference between the subgraph corresponding to the detected target of the second detection type and the second base library subgraph The second degree of similarity obtains the degree of similarity of each target image corresponding to the retrieval instruction.

Specifically, the similarity between the images can be obtained by extracting the image features in each image and calculating the feature distance of the corresponding position. The image retrieval device is based on the first similarity between the sub-image corresponding to the detected target of the first detection type and the first base library sub-image, and the difference between the sub-image corresponding to the detected object of the second detection type and the second base library sub-image The second degree of similarity obtains the degree of similarity between each target image and the image to be retrieved. Optionally, the image retrieval device may linearly superimpose the first similarity and the second similarity according to a preset weight to obtain the similarity of each target image corresponding to the retrieval instruction. The image retrieval device may also perform non-linear superposition of the first similarity and the second similarity to obtain the similarity of each target image corresponding to the retrieval instruction. Of course, the similarity between each target image and the image to be retrieved can also be calculated in other ways, and the present application is not limited to this.

For example, the user hopes that since the same target image contains a human face matching the human face F and a human body matching the human body P, an associated retrieval instruction including the face detection type and the human body retrieval type is given. Among them, the face matching similarity threshold is 90%, and the human body matching similarity threshold is 70%. Normally, the face retrieval result is more accurate, and a higher weight can be set, such as 0.8, and a lower weight, such as 0.2, can be set for the human body due to clothing or occlusion. The user image retrieval device retrieves F1 (90% similarity to F, corresponding to panoramic image identification 1), F2 (95% similarity to F, corresponding to panoramic image identification 2), and P1 that matches human body P (90% similarity to P, corresponding to panoramic image identification 1), P2 (75% similarity to F, corresponding to panoramic image identification 2), the target image M1 (including F1 and P1) with panoramic image identification 1 can be obtained, with The target image M2 (including F2 and P2) of the panoramic image identifier 2, where the similarity between M1 and the image to be retrieved is 90%, and the similarity between M2 and the image to be retrieved is 91%.

In the embodiment of the present application, since the first base library sub-image and the second base library sub-image are images in the same first associated panoramic image, the associated similarity calculated based on the first similarity and the second similarity , Can be used to characterize the similarity between the first associated panoramic image and the detected targets of the first detection type and the second detection type. In this way, more image information can be obtained, which helps to find the image between the two types of targets at the same time. Relevance.

Further, after the similarity of each target image corresponding to the retrieval instruction is obtained, the similarities under different detection types may be sorted in descending order, and the top N bits may be selected as the target image corresponding to the retrieval instruction. In this way, displaying target images with a higher degree of similarity to the user reduces the amount of tasks the user has to view on the one hand, and on the other hand provides more accurate target images, which is conducive to improving user experience.

In an exemplary embodiment, the retrieval instruction includes the identification of the detected target used for retrieval. Based on this, please refer to Figure 6, the method also includes the following steps:

In step S209, the distribution area corresponding to each detected target or the sub-image corresponding to each detected target is displayed to the user, so that the user selects a detected target for image retrieval from the detected targets.

Specifically, after obtaining the distribution area corresponding to each detected target and/or the sub-map corresponding to each detected target, the image retrieval device displays the distribution area corresponding to each detected target and/or the sub-map corresponding to each detected target To the user, the display mode can be displayed to the user through the display screen of the image retrieval device or through the first transmission to the terminal and then displayed to the user through the terminal display, so that the user can select the checkout for image retrieval from each checkout target Target.

It is understandable that after the image retrieval device performs target detection on the image to be retrieved through the preset target detection algorithm and obtains each detected target, it assigns an identifier (for example, target ID) to each detected target, and the user selects a certain detected target. After the target is used as a checkout target for search, it returns a search instruction containing the identification of the checkout target used for search.

Further, step S210 can be specifically implemented through the following steps:

Step S2102, for the third detection target corresponding to the identification of the detection target used for image retrieval, in the image base corresponding to the detection type of the third detection target, search for the child corresponding to the third detection target. The third base library sub-image that the image matches.

Specifically, after the image retrieval device receives the detection target selected by the user, it determines the third detection target according to the identification of the detection target, and searches the image database corresponding to the detection type of the third detection target. The third base library subgraph matching the subgraph corresponding to the third detection target. That is, in this embodiment, the search is performed only for the checkout targets selected by the user, and it is not necessary to search for all checkout targets.

It should be noted that there may be multiple third detection targets, and multiple third detection targets may correspond to the same or different detection types. For example, the user selects the face F, the non-motor vehicle V from the face F, the human body P, and the non-motorized vehicle V in the image to be retrieved as the third detection target. The base library sub-image F4 matched by F is searched for the base library sub-image P4 matching P in the non-motor vehicle image base library.

In another embodiment, the image retrieval device may only show two different targets of the same detection type to the user for selection when the image to be retrieved contains two different targets of the same detection type. For a detection type that contains only one target, The target contained in it can be used for retrieval by default. For example, if the image to be detected contains two human faces and one non-motorized vehicle, the two human faces can be displayed to the user so that the user can select the face that needs to be used for retrieval without showing the non-motorized vehicle to the user. After that, the image retrieval device uses the non-motorized vehicle contained in the image A to be retrieved and a face selected by the user for retrieval.

In the embodiment of the present application, the distribution area corresponding to each detected target and/or the sub-graph corresponding to each detected target is displayed to the user, so that the user can select the target with the retrieval requirement, so as to quickly retrieve the user with the retrieval requirement. The retrieval image of the target further improves retrieval efficiency.

In an exemplary embodiment, the retrieval instruction includes at least two identifications of the detected targets used for retrieval. Based on this, step S210 can also be specifically implemented through the following steps:

Step S2104, for the fourth detection target corresponding to the identification of the detection target used for image retrieval, in the image base corresponding to the detection type of the fourth detection target, search for the child corresponding to the fourth detection target The fourth base library sub-picture of the picture matching.

Specifically, for the fourth detection target corresponding to the identification of the detection target used for image retrieval, the image retrieval device retrieves the fourth detection target in the image base corresponding to the detection type of the fourth detection target The corresponding subgraph matches the fourth base library subgraph.

In some cases, the user wants to retrieve images that contain both the third detection target and the fourth detection target. For example, the image to be retrieved contains face A and face B, and the user wants to find other images where face A and face B are captured at the same time. At this time, step S212 can be specifically implemented through the following steps:

Step S2122: If the third base library sub-image and the fourth base library sub-image are associated with a second associated panorama with the same image identifier, use the second associated panorama as the target image corresponding to the retrieval instruction, or associate the second associated panorama with The panoramic image, the third base library sub-image, and the fourth base library sub-image are used as the target image corresponding to the retrieval instruction.

Specifically, if the third base library sub-image and the fourth base library sub-image are associated with the second associated panorama with the same image identifier, it indicates that the third base library sub-image and the fourth base library sub-image are both from the second association. The map is obtained in the panorama, that is, the target in the third base library sub-map and the target in the fourth base library sub-map are in the same panoramic view. At this time, the image retrieval device may use the second associated panorama as the target image corresponding to the retrieval instruction, or the image retrieval device may use the second associated panorama, the third base library sub-image, and the fourth base library sub-image as the retrieval instruction corresponding to the retrieval instruction. Target image.

In this way, the image retrieval device will use the second associated panoramic image in which the face A and the face B are captured at the same time as the target image.

It should be noted that the detection types of the third detection target and the fourth detection target may be the same or different.

In the embodiment of this application, if the different base library sub-images belong to the partial images in the same second associated panorama, it indicates that these base library sub-images were captured by the same image acquisition device at the same time. Therefore, these The base library sub-image and the second associated panoramic image are used as the final target image, which is beneficial for the user to know the correlation between the images, and thereby learn more image information.

In an exemplary embodiment, referring to FIG. 7, step S210 may be specifically implemented through the following steps:

Step S232, in the image base library corresponding to the detection type of the detected target, filter the base library sub-image set that meets the set time and/or set location input by the user;

Step S234, searching for a base library subgraph matching the subgraph corresponding to the detected target in the base library subgraph set.

Specifically, the user can set the search time period and the search location range in the search condition option in the search interface displayed on the terminal. Based on the set time and/or set location, the image corresponding to the detection type of the detected target In the base library, filter the set of base library submaps that meet the set time and/or set location input by the user. Next, the target detection device searches for the base library subgraph matching the subgraph corresponding to the detected target in the base library subgraph set.

In the embodiment of the present application, the search is only performed in the base library sub-map at the set time and/or set location, which is beneficial to quickly obtain the search image.

In an exemplary embodiment, there are multiple ways to implement time setting, and only a few of them are listed below:

Implementation method 1: Obtain the preset time and preset duration input by the user; use the time period including the preset time as the set time, where the duration of the set time is the preset duration. Optionally, the preset time is a time determined based on the shooting time of the image to be retrieved, and the set time is a time period centered on the preset time and 1/2 of the preset time length before and after.

Under normal circumstances, it is easier to retrieve the target within a period of time before and after the time the image to be retrieved is taken, and the preset duration can be set according to the moving speed of the target.

Implementation method 2: Obtain the start time and end time entered by the user; use the time period between the start time and the end time as the set time. Optionally, the start time and the end time are determined based on the shooting time of the image to be retrieved.

For example, in the continuous time period option, the user enters the start time at the start time, such as 2018-05-02 00:00:00, and the end time at the end time, such as 2018-05-09 00 :00:00, after setting, the image retrieval device will use 2018-05-02 00:00:00 ~ 2018-05-09 00:00:00 as the set time.

Implementation method 3: Get the start date and end date entered by the user, the start time of each day, and the end time of each day, and add the start time and the end time of each day between the start date and the end date The time period between the end times of each day is regarded as the set time. Optionally, the start date and end date, the start time of each day, and the end time of each day are determined based on the shooting time of the image to be retrieved.

For example, in the regular time period option, the user enters the start date and end date at the date, such as 2018-05-02, 2018-05-09, and enters the daily start time and every day at the time. The end time of the day, such as 01:00:00, 04:00:00, after setting, the image retrieval equipment will be 01:00:00～04:00:00 in 2018-05-02～2018-05-09 As the set time.

In this embodiment, by setting multiple search time search methods, different actual needs can be met, the functions are more diversified, and the applicability is stronger.

It should be understood that although the various steps in the flowcharts of FIGS. 2-7 are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless specifically stated in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least part of the steps in Figures 2-7 can include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution of these steps or stages The sequence is not necessarily performed sequentially, but may be performed alternately or alternately with other steps or at least a part of the steps or stages in other steps.

In an exemplary embodiment, as shown in FIG. 8, an image retrieval device is provided, including: an instruction receiving module 302, an image acquisition module 304, a target detection module 306, a matting module 308, an image retrieval module 310, and an image Determine module 312, where:

The instruction receiving module 302 is used to receive a user's retrieval instruction.

The image acquisition module 304 is used to acquire the image to be retrieved; the image to be retrieved contains multiple targets of different detection types.

The target detection module 306 is used to perform target detection on the image to be retrieved through a target detection algorithm to obtain the detection type corresponding to each detected target and the distribution area corresponding to each detected target; the distribution area is the location of the detected target in the image to be detected. Area.

The matting module 308 is used for matting in the image to be retrieved according to the distribution area corresponding to each detected target to obtain a sub-image corresponding to each detected target.

The image retrieval module 310 is used to retrieve the base library sub-images that match the sub-images corresponding to the detected targets in the image base database corresponding to the detection type of the detected target for the detected targets of each detection type; The base library sub-map is stored in the library, and the base library sub-map contains targets of the same detection type as the detected target.

The image determination module 312 is configured to obtain the target image corresponding to the retrieval instruction according to the base library sub-image matched with the sub-image corresponding to each detected target.

In the above image retrieval device, multiple targets of different detection types in the image to be retrieved can be detected at one time, so as to obtain the detection type corresponding to each detected target and the distribution area corresponding to each detected target, and then for each detection type The target image is retrieved in the image base corresponding to the detection type of the detected target. In the embodiment of the application, the user does not need to manually intercept the sub-images of each target in the image to be retrieved and input each target sub-image into the retrieval system. The user only needs to upload the image once to complete the retrieval of all detected targets of detection types at one time. Thus, the complexity of user operations is effectively reduced, and retrieval efficiency is improved.

In an exemplary embodiment, the user's search instruction is an associated search instruction, and the associated search instruction includes a first detection type and a second detection type; a base library sub-image that matches a sub-image corresponding to a detection target of the first detection type It is the first base library sub-image, and the base library sub-image that matches the sub-image corresponding to the detection target of the second detection type is the second base library sub-image. The image base library is associated with the base library sub-image and the base library sub-image. The image identifier of the panoramic image corresponding to the image; the image determining module 312 is specifically configured to associate the first associated panoramic image with the same image identifier if the first base library subimage and the second base library subimage are associated with the first associated panoramic image. The image is used as the target image corresponding to the retrieval instruction, or the first associated panoramic image, the first base library sub-image, and the second base library sub-image are used as the target image corresponding to the retrieval instruction.

In an exemplary embodiment, it further includes: a display module (not shown) for displaying the detection type corresponding to each detected target to the user, so that the user can select the first detection type from the detection type corresponding to each detected target. Detection type and second detection type.

In an exemplary embodiment, it further includes: a similarity calculation module, which is used to calculate the first similarity between the sub-image corresponding to the detected target of the first detection type and the first base library sub-image, and the second detection type The second similarity between the sub-image corresponding to the target and the second base library sub-image is detected, and the similarity of each target image corresponding to the retrieval instruction is obtained.

In an exemplary embodiment, the retrieval instruction includes the identification of the detected target used for retrieval; the display module is also used to display the distribution area corresponding to each detected target or the sub-image corresponding to each detected target to the user, The user is allowed to select a detected target for image retrieval from among the detected targets; the image retrieval module 310 is specifically used for the third detected target corresponding to the identification of the detected target used for image retrieval, and the third detected target In the image base library corresponding to the detection type of the detected target, a third base library sub-image that matches the sub-image corresponding to the third detected target is retrieved.

In an exemplary embodiment, the retrieval instruction includes at least two identifications of the detected targets used for retrieval; the image retrieval module 310 is specifically configured to determine the fourth detection corresponding to the identifications of the detected targets used for image retrieval. Target, in the image base corresponding to the detection type of the fourth detected target, retrieve the fourth base library sub-image that matches the sub-image corresponding to the fourth detected target; the image determining module 312 is specifically configured to: If the base library sub-image and the fourth base library sub-image are associated with the second associated panoramic image with the same image identifier, the second associated panoramic image is used as the target image corresponding to the retrieval instruction, or the second associated panoramic image and the third associated panoramic image are The library sub-image and the fourth base library sub-image are used as the target image corresponding to the retrieval instruction.

In an exemplary embodiment, the image retrieval module 310 is specifically configured to filter the image base library corresponding to the detection type of the detected target to filter the base library sub-images that meet the set time and/or set location input by the user. Set; in the base library subgraph collection, search for base library subgraphs that match the subgraph corresponding to the detected target.

For the specific definition of the image retrieval device, please refer to the above definition of the image retrieval method, which will not be repeated here. Each module in the above-mentioned image retrieval device can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules may be embedded in the form of hardware or independent of the processor in the computer equipment, or may be stored in the memory of the computer equipment in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. 9. The computer equipment includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer equipment is used to store the sub-picture data of the base library. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer program is executed by the processor to realize an image retrieval method.

Those skilled in the art can understand that the structure shown in FIG. 9 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may Including more or fewer parts than shown in the figure, or combining some parts, or having a different arrangement of parts.

In an exemplary embodiment, a computer device is provided, including a memory and a processor, and a computer program is stored in the memory, and the processor implements the following steps when the processor executes the computer program:

Receiving user's retrieval instructions;

Obtain the image to be retrieved; the image to be retrieved contains multiple targets of different detection types;

Target detection is performed on the image to be retrieved through the target detection algorithm, and the detection type corresponding to each detected target and the distribution area corresponding to each detected target are obtained; the distribution area is the area where the detected target is located in the image to be detected;

Cut out the image in the image to be retrieved according to the distribution area corresponding to each detected target, and obtain the sub-image corresponding to each detected target;

For detection targets of each detection type, in the image base corresponding to the detection type of the detected target, retrieve the base library sub-images that match the sub-image corresponding to the detected target; the image base library stores the base library sub-images , The base library sub-image contains the same target as the detection type of the detected target;

According to the base library sub-images matching the sub-images corresponding to each detected target, the target image corresponding to the retrieval instruction is obtained.

In the above-mentioned computer equipment, target detection can be performed on multiple targets of different detection types in the image to be retrieved at one time, so as to obtain the detection type corresponding to each detected target and the distribution area corresponding to each detected target. The target is detected, and the target image is retrieved in the image base corresponding to the detection type of the detected target. In the embodiment of the application, the user does not need to manually intercept the sub-images of each target in the image to be retrieved and input each target sub-image into the retrieval system. The user only needs to upload the image once to complete the retrieval of all detected targets of detection types at one time. Thus, the complexity of user operations is effectively reduced, and retrieval efficiency is improved.

In an exemplary embodiment, the user's search instruction is an associated search instruction, and the associated search instruction includes a first detection type and a second detection type; a base library sub-image that matches a sub-image corresponding to a detection target of the first detection type It is the first base library sub-image, and the base library sub-image that matches the sub-image corresponding to the detection target of the second detection type is the second base library sub-image. The image base library is associated with the base library sub-image and the base library sub-image. The image identifier of the panoramic image corresponding to the image; the processor also implements the following steps when executing the computer program: if the first base library sub-image and the second base library sub-image are associated with the first associated panoramic image with the same image identifier, the An associated panoramic image is used as the target image corresponding to the retrieval instruction, or the first associated panoramic image, the first base library sub-image, and the second base library sub-image are used as the target image corresponding to the retrieval instruction.

In an exemplary embodiment, the processor further implements the following step when executing the computer program: displaying the detection type corresponding to each detected target to the user, so that the user selects the first detection type from the detection types corresponding to each detected target And the second detection type.

In an exemplary embodiment, the processor further implements the following steps when executing the computer program: according to the first similarity between the subgraph corresponding to the detected target of the first detection type and the first base library subgraph, and the second detection type The second similarity between the sub-image corresponding to the detected target and the second base library sub-image is obtained to obtain the similarity of each target image corresponding to the retrieval instruction.

In an exemplary embodiment, the retrieval instruction includes the identification of the detected target used for retrieval; the processor further implements the following steps when executing the computer program: the distribution area corresponding to each detected target or the subgraph corresponding to each detected target It is displayed to the user so that the user can select a detection target for image retrieval from each detection target; for the third detection target corresponding to the identification of the detection target used for image retrieval, the third detection target In the image base library corresponding to the detection type of, search for a third base library sub-image that matches the sub-image corresponding to the third detection target.

In an exemplary embodiment, the retrieval instruction includes at least two identifications of the detected targets used for retrieval; the processor further implements the following steps when executing the computer program: Four detection targets, in the image base corresponding to the detection type of the fourth detection target, search for the fourth base library sub-image that matches the sub-image corresponding to the fourth detection target; if the third base library sub-image and If the fourth base library sub-image is associated with the second associated panoramic image with the same image identifier, the second associated panoramic image is used as the target image corresponding to the retrieval instruction, or the second associated panoramic image, the third base library sub-image and the first The four-base library sub-image is used as the target image corresponding to the retrieval instruction.

In an exemplary embodiment, when the processor executes the computer program, the following steps are further implemented: in the image database corresponding to the detection type of the detected target, filter the image database corresponding to the set time and/or set location input by the user. Library subgraph collection; in the base library subgraph collection, search the base library subgraph matching the subgraph corresponding to the detected target.

In an exemplary embodiment, the processor further implements the following steps when executing the computer program: obtaining the preset time and the preset duration input by the user; taking the time period including the preset time as the set time, and the duration of the set time is Preset duration; or; obtain the start time and end time entered by the user; use the time period between the start time and the end time as the set time; or; obtain the start date and end date entered by the user, and the daily For the start time and the end time of each day, the time period between the start time of the day and the end time of each day in each date between the start date and the end date is used as the set time.

In an exemplary embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

Receiving user's retrieval instructions;

For the detected targets of each detection type, in the image base corresponding to the detection type of the detected target, retrieve the base library sub-images that match the sub-images corresponding to the detected targets; the base library sub-images are stored in the image base library , The base library sub-image contains the same target as the detection type of the detected target;

In the above-mentioned computer-readable storage medium, target detection can be performed on multiple targets of different detection types in the image to be retrieved at one time, so as to obtain the detection type corresponding to each detected target and the distribution area corresponding to each detected target. For the detection targets of the detection type, the target images are retrieved in the image base corresponding to the detection type of the detection target. In the embodiment of the application, the user does not need to manually intercept the sub-images of each target in the image to be retrieved and input each target sub-image into the retrieval system. The user only needs to upload the image once to complete the retrieval of all detected targets of detection types at one time. Thus, the complexity of user operations is effectively reduced, and retrieval efficiency is improved.

In an exemplary embodiment, the user's search instruction is an associated search instruction, and the associated search instruction includes a first detection type and a second detection type; a base library sub-image that matches a sub-image corresponding to a detection target of the first detection type It is the first base library sub-image, and the base library sub-image that matches the sub-image corresponding to the detection target of the second detection type is the second base library sub-image. The image base library is associated with the base library sub-image and the base library sub-image. The image identifier of the panoramic image corresponding to the image; the computer program is executed by the processor to implement the following steps: if the first base library sub-image and the second base library sub-image are associated with the first associated panoramic image with the same image identifier, then The first associated panoramic image is used as the target image corresponding to the retrieval instruction, or the first associated panoramic image, the first base library sub-image, and the second base library sub-image are used as the target image corresponding to the retrieval instruction.

In an exemplary embodiment, when the computer program is executed by the processor, the following steps are further implemented: the detection type corresponding to each detected target is displayed to the user, so that the user selects the first detection from the detection type corresponding to each detected target. Type and second detection type.

In an exemplary embodiment, when the computer program is executed by the processor, the following steps are further implemented: according to the first similarity between the subgraph corresponding to the detected target of the first detection type and the first base library subgraph, and the second detection The second similarity between the sub-image corresponding to the detected target of the type and the second base library sub-image is obtained to obtain the similarity of each target image corresponding to the retrieval instruction.

In an exemplary embodiment, the retrieval instruction includes the identification of the detected target used for retrieval; when the computer program is executed by the processor, the following steps are also implemented: the distribution area corresponding to each detected target or the child corresponding to each detected target The picture is shown to the user, so that the user can select the detection target for image retrieval from each detection target; for the third detection target corresponding to the identification of the detection target used for image retrieval, the third detection target In the image base library corresponding to the detection type of the target, a third base library sub-image that matches the sub-image corresponding to the third detected target is retrieved.

In an exemplary embodiment, the retrieval instruction includes at least two identifications of the detected targets used for retrieval; when the computer program is executed by the processor, the following steps are also implemented: The fourth detection target, in the image base corresponding to the detection type of the fourth detection target, search for the fourth base library sub-image that matches the sub-image corresponding to the fourth detection target; if the third base library sub-image If the second associated panoramic image with the same image identifier is associated with the fourth base library sub-image, the second associated panoramic image is used as the target image corresponding to the retrieval instruction, or the second associated panoramic image, the third base library sub-image and The fourth base library sub-image is used as the target image corresponding to the retrieval instruction.

In an exemplary embodiment, when the computer program is executed by the processor, the following steps are further implemented: in the image base corresponding to the detection type of the detected target, filter those that meet the set time and/or set location input by the user. Base library subgraph collection; in the base library subgraph collection, search for base library subgraphs that match the subgraph corresponding to the detected target.

In an exemplary embodiment, when the computer program is executed by the processor, the following steps are further implemented: obtaining the preset time and the preset duration input by the user; using the time period including the preset time as the set time, and the duration of the set time Is the preset duration; or; obtains the start time and end time entered by the user; takes the time period between the start time and the end time as the set time; or; obtains the start date and end date entered by the user, and every day The start time and the end time of each day, the time period between the start time of the day and the end time of each day in each date between the start date and the end date is used as the set time.

A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage. In the medium, when the computer program is executed, it may include the processes of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, or optical storage. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not a limitation, RAM may be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc.

The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be It is considered as the range described in this specification.

The above-mentioned embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the scope of the invention patent. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can be made, and these all fall within the protection scope of this application. Therefore, the scope of protection of the patent of this application shall be subject to the appended claims.

Claims

An image retrieval method, characterized in that the method includes:

Receiving user's retrieval instructions;

Acquiring an image to be retrieved; the image to be retrieved includes multiple targets of different detection types;

Target detection is performed on the image to be retrieved by the target detection algorithm, and the detection type corresponding to each detected target and the distribution area corresponding to each detected target are obtained; the distribution area is the detected target in the to-be-detected The area in the image;

Matting in the image to be retrieved according to the distribution area corresponding to each detected target to obtain a sub-image corresponding to each detected target;

For the detected targets of each detection type, in the image base database corresponding to the detection type of the detected target, search the base library sub-images that match the sub-images corresponding to the detected targets; in the image base library Storing a base library sub-picture, the base library sub-picture contains the same target as the detection type of the detected target;

The target image corresponding to the retrieval instruction is obtained according to the base library sub-image that matches the sub-image corresponding to each detected target.
The method according to claim 1, wherein the user's search instruction is an associated search instruction, and the associated search instruction includes a first detection type and a second detection type; and the detection of the first detection type The base library subgraph matched by the subgraph corresponding to the target is the first base library subgraph, and the base library subgraph matched with the subgraph corresponding to the detected target of the second detection type is the second base library subgraph. The image base library is associated and stored with the base library sub-image and the image identifier of the panorama corresponding to the base library sub-image;

The obtaining the target image corresponding to the retrieval instruction according to the base library sub-image matching the sub-image corresponding to each detected target includes:

If the first base library sub-image and the second base library sub-image are associated with a first associated panoramic image with the same image identifier, then the first associated panoramic image is used as the target image corresponding to the retrieval instruction, Or use the first associated panoramic image, the first base library sub-image, and the second base library sub-image as the target image corresponding to the retrieval instruction.
The method according to claim 2, wherein the method further comprises:

The detection type corresponding to each detected target is displayed to the user, so that the user selects the first detection type and the second detection type from the detection types corresponding to each detected target.
The method according to claim 2 or 3, wherein the method further comprises:

According to the first similarity between the sub-image corresponding to the detected target of the first detection type and the first base library sub-image, the sub-image corresponding to the detected target of the second detection type is compared with the second The second degree of similarity of the base library sub-images obtains the degree of similarity of each target image corresponding to the retrieval instruction.
The method according to any one of claims 1 to 4, wherein the retrieval instruction includes an identification of a detected target used for retrieval; the method further comprises:

Display the distribution area corresponding to each detected target or the sub-image corresponding to each detected target to the user, so that the user selects the detected target for image retrieval from the detected targets ；

For the detection targets of each detection type, in the image base library corresponding to the detection type of the detection target, searching the base library sub-images that match the sub-images corresponding to the detected targets includes:

For the third detection target corresponding to the identification of the detection target used for image retrieval, search for the third detection target in the image base corresponding to the detection type of the third detection target The third base library sub-image matched by the corresponding sub-image.
The method according to claim 5, wherein the retrieval instruction includes at least two identifications of the detected targets used for retrieval; In the image base library corresponding to the detection type, retrieving the base library sub-image that matches the sub-image corresponding to the detected target, further includes:

For the fourth detection target corresponding to the identification of the detection target used for image retrieval, search for the fourth detection target in the image base corresponding to the detection type of the fourth detection target The fourth base library subgraph matched by the corresponding subgraph;

The obtaining the target image corresponding to the retrieval instruction according to the base library sub-image matching the sub-image corresponding to each detected target includes:

If the third base library sub-image and the fourth base library sub-image are associated with a second associated panorama with the same image identifier, then the second associated panorama is used as the target image corresponding to the retrieval instruction, Or use the second associated panoramic image, the third base library sub-image, and the fourth base library sub-image as the target image corresponding to the retrieval instruction.
The method according to any one of claims 1 to 6, characterized in that, in the image base corresponding to the detection type of the detected target, search for the sub-images corresponding to the detected target The base library submap, including:

In the image base library corresponding to the detection type of the detected target, filter the base library sub-image set that meets the set time and/or set location input by the user;

In the set of base library subgraphs, search base library subgraphs matching the subgraph corresponding to the detected target.
The method according to claim 7, characterized in that, before the screening of the set of base library sub-images that meet the set time and/or set location input by the user, the method further comprises:

Acquiring the preset time and the preset duration input by the user;

Use a time period including the preset time as the set time, and the duration of the set time is the preset time;

or;

Acquiring the start time and end time input by the user;

Use the time period between the start time and the end time as the set time;

or;

Acquire the start date and end date, daily start time and daily end time entered by the user,

In each date between the start date and the end date, a time period between the start time of the day and the end time of the day is taken as the set time.
An image retrieval device, characterized in that the device includes:

The instruction receiving module is used to receive the user's retrieval instruction;

An image acquisition module for acquiring an image to be retrieved; the image to be retrieved contains multiple targets of different detection types;

The target detection module is used to perform target detection on the image to be retrieved through a target detection algorithm to obtain the detection type corresponding to each detected target and the distribution area corresponding to each detected target; the distribution area is the detected target The area where the target is located in the image to be detected;

A matting module for matting in the image to be retrieved according to the distribution area corresponding to each detected target to obtain a sub-image corresponding to each detected target;

The image retrieval module is used to retrieve, in the image base database corresponding to the detection type of the detected target, the base library sub-image that matches the sub-image corresponding to the detected target for the detected targets of each detection type; The image base library stores base library sub-images, and the base library sub-images include targets of the same detection type as the detected targets;

The image determination module is used to obtain the target image corresponding to the retrieval instruction according to the base library sub-image matched with the sub-image corresponding to each detected target.
A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method according to any one of claims 1 to 8 when the computer program is executed by the processor.
A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the method according to any one of claims 1 to 8 when the computer program is executed by a processor.