CN113806577A - Image searching method and device, storage medium and electronic equipment - Google Patents
Image searching method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN113806577A CN113806577A CN202110974274.0A CN202110974274A CN113806577A CN 113806577 A CN113806577 A CN 113806577A CN 202110974274 A CN202110974274 A CN 202110974274A CN 113806577 A CN113806577 A CN 113806577A
- Authority
- CN
- China
- Prior art keywords
- image
- original image
- target
- panoramic
- compressed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 230000015654 memory Effects 0.000 claims description 28
- 238000004590 computer program Methods 0.000 claims description 21
- 230000003068 static effect Effects 0.000 claims description 13
- 238000004891 communication Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 4
- 238000010845 search algorithm Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 8
- 238000005516 engineering process Methods 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 16
- 239000000284 extract Substances 0.000 description 8
- 230000006870 function Effects 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 3
- 238000007906 compression Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000006835 compression Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Library & Information Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Processing Or Creating Images (AREA)
Abstract
The present application relates to the field of image processing technologies, and in particular, to an image search method, an image search device, a storage medium, and an electronic device. The image searching method provided by the embodiment of the application can receive an image searching request sent by a client based on a panoramic compressed image, wherein the image searching request carries marquee information and is based on the marquee information, corresponding target object compressed images are intercepted from the panoramic compressed images, compressed image features of the target object compressed images are extracted, matching degrees between the compressed image features and original image features of all prestored local original images are obtained, target original image features with matching degrees meeting preset matching conditions are selected based on the obtained matching degrees, image searching is carried out based on the target original image features, and corresponding searching results are obtained, so that the accuracy of the image searching results can be improved.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to an image search method, an image search device, a storage medium, and an electronic device.
Background
With the development of Artificial Intelligence (AI) technology, fast searching based on images has gradually become an important requirement, and accordingly, various image searching technologies have been widely applied.
Specifically, when an image search technology is used, an intelligent server sends a panoramic compressed image containing a plurality of objects (such as human faces) and a selection frame to a client, the client performs frame selection on at least one object in the panoramic compressed image to obtain at least one object compressed image, the selected at least one object compressed image is sent to one or more intelligent servers, the intelligent servers extract image features of the obtained object compressed image, and search is performed in an image database based on the obtained image features to obtain a matched object associated image.
However, with the above scheme, although the matched target-related image can be obtained quickly, the following defects exist:
in the related art, the panoramic compressed image source has two parts: the method comprises the steps that firstly, an intelligent server compresses panoramic original images stored locally at the intelligent server or obtained from other servers connected with the intelligent server to obtain corresponding panoramic compressed images; secondly, the intelligent server or other servers connected with the intelligent server perform Luminance and Chrominance (YUV) coding on the locally preset video frame, and then generate a corresponding panoramic compressed image through compression coding.
In general, generating a panoramic compressed image by a compression method causes loss of pixels of the image during the compression process, so that image features of a target object compressed image extracted based on the panoramic compressed image are partially missing, thereby affecting the accuracy of a search result during an image search process.
In view of the above, it is desirable to provide a new image searching method to overcome the above-mentioned drawbacks.
Disclosure of Invention
The embodiment of the application provides an image searching method, an image searching device, a storage medium and electronic equipment, which can obtain accurate image searching results.
In order to achieve the above purpose, the technical solution of the embodiment of the present application is implemented as follows:
in a first aspect, an embodiment of the present application provides an image search method, where the method includes:
receiving an image search request sent by a client based on a panoramic compressed image, wherein the image search request carries marquee information; the panoramic compressed image is sent to the client in advance by an intelligent server;
based on the marquee information, intercepting a corresponding target object compressed image from the panoramic compressed image, and extracting the compressed image characteristics of the target object compressed image;
respectively obtaining the matching degrees between the compressed image features and the original image features of each pre-stored local original image, and selecting target original image features with matching degrees meeting preset matching conditions based on the obtained matching degrees, wherein the local original images are intercepted from the appointed sample panoramic original images;
and searching images based on the target original image characteristics to obtain corresponding search results.
In a second aspect, an embodiment of the present application further provides an image search apparatus, including:
the communication module is used for receiving an image search request sent by a client based on the panoramic compressed image, wherein the image search request carries marquee information; the panoramic compressed image is sent to the client in advance by an intelligent server;
the extraction module is used for intercepting a corresponding target object compressed image from the panoramic compressed image based on the marquee information and extracting the compressed image characteristics of the target object compressed image;
the matching module is used for respectively obtaining the compressed image features and the matching degrees between the pre-stored original image features of each local original image, and selecting target original image features with the matching degrees meeting preset matching conditions based on the obtained matching degrees, wherein the local original images are intercepted from the appointed sample panoramic original images;
and the searching module is used for searching images based on the target original image characteristics to obtain corresponding searching results.
In an optional embodiment, before receiving an image search request sent by a client based on a panoramic compressed image, the extracting module is specifically configured to perform at least one of the following operations:
acquiring a specified static image, taking the static image as a sample panoramic original image, intercepting a local original image from the sample panoramic original image, extracting original image characteristics of the local original image, and storing the original image characteristics in a storage area associated with the panoramic compressed image;
acquiring a specified video data stream, decoding the video data stream, taking an obtained video frame image as a sample panoramic original image, intercepting a local original image from the sample panoramic original image, extracting original image characteristics of the local original image, and storing the original image characteristics in a storage area associated with the panoramic compressed image.
In an optional embodiment, when the local original image is intercepted from the sample panoramic original image, the extracting module is specifically configured to:
extracting the position information of a specified target in the sample panoramic original image by adopting a sliding window method SWA;
and according to the position information, intercepting a local original image with a preset size from the sample panoramic original image.
In an optional embodiment, when the matching degrees between the compressed image features and the pre-stored original image features of the local original images are respectively obtained, the matching module is specifically configured to:
respectively aiming at the original image characteristics of each local original image, executing the following operations:
calculating cosine values of included angles between original image features of a local original image and compressed image features of a compressed image of a target object by adopting a cosine similarity method;
taking the cosine value as the matching degree between the original image characteristic of the local original image and the compressed image characteristic of the target object compressed image;
sequentially reading the obtained matching degrees, executing the following operations every time when one matching degree is read, and outputting the target original image characteristics until all the matching degrees are read:
comparing the matching degree read currently with the matching degree read last time;
if the matching degree read currently is judged to be larger than the matching degree read last time, the original image features related to the matching degree read currently are used as new target original image features;
and if the matching degree read currently is judged to be smaller than the matching degree read last time, keeping the original image characteristics associated with the matching degree read last time as target original image characteristics.
In an optional embodiment, when the target original image feature with the matching degree meeting the preset matching condition is selected based on the obtained matching degrees, the search module is specifically configured to:
and performing image search in an image database based on the target original image characteristics by adopting a scale invariant feature transform matching method SIFT or a fast nearest neighbor search algorithm FLANN, and taking the obtained target image as the search result.
In an optional embodiment, after obtaining the corresponding search result, the communication module is specifically configured to perform any one of the following operations:
and if the number of the target images is 1, directly sending the obtained target images to the client.
And if the number of the target images is more than 1, sorting the target images according to the alphabetical order according to the initial letters of the names of the target images, and sending the target images to the client according to a sorting result. In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements any one of the image search methods described in the first aspect above.
In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the image search method of the first aspect is implemented.
In a fourth aspect, an embodiment of the present application further provides an electronic device, including a memory and a processor, where the memory stores a computer program executable on the processor, and when the computer program is executed by the processor, the processor is enabled to implement any one of the image search methods in the first aspect.
In the embodiment of the application, an intelligent server intercepts corresponding target object compressed images from panoramic compressed images and extracts compressed image features based on the information of a marquee carried by an image search request sent by a client, and performs image search based on target original features, of the original image features corresponding to each preset local original image, of which the matching degree with the compressed image features meets preset matching conditions, so as to obtain the target images; therefore, the target original image features which are most similar to the compressed image features are adopted to replace the compressed image features for image search, so that more accurate target images can be obtained, the image search accuracy is effectively improved on the premise of not increasing the system operation load, and meanwhile, the search efficiency is also ensured.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1 is a diagram of a service system architecture for image search according to an embodiment of the present application;
fig. 2a and fig. 2b are exemplary diagrams of a local original feature extraction situation provided in an embodiment of the present application;
3a, 3b and 3c are exemplary diagrams for setting a sample panoramic original image based on a video data stream according to an embodiment of the present application;
fig. 4 is a diagram illustrating an example of a storage form and a transmission manner of original image features in a system according to an embodiment of the present application;
fig. 5 is a flowchart of an image searching method provided in an embodiment of the present application;
fig. 6a, fig. 6b, and fig. 6c are exemplary diagrams illustrating a flow of an image searching method according to an embodiment of the present application;
fig. 7 is a flowchart of a method for selecting a target original image feature with a matching degree meeting a preset matching condition according to an embodiment of the present application;
fig. 8a and fig. 8b are partial flowchart illustrations of an image searching method according to an embodiment of the present application;
fig. 9 is a schematic structural diagram of an image searching apparatus according to an embodiment of the present application;
fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In order to improve the searching accuracy of the image searching technology, in the embodiment of the application, various local original images and corresponding original image characteristics are pre-stored in a system, when an intelligent server intercepts corresponding target object compressed images from panoramic compressed images based on an image searching request sent by a client, and extracts the compressed image characteristics of the target object compressed images, an intelligent server compares the compressed image characteristics with the pre-stored original image characteristics, selects target original image characteristics meeting preset matching conditions from the compressed image characteristics, and then performs image searching based on the target original image characteristics; therefore, the accuracy of image searching is effectively improved by replacing the compressed image features with the original image features.
Referring to fig. 1, in the embodiment of the present application, the service system includes a plurality of clients 100, an intelligent server 110 and an optional image database 120, where the number of the intelligent servers 110 may be multiple, a network connection is established between each client 100 and each intelligent server 110, and each intelligent server 110 may provide an image search service to the client 100.
In the embodiment of the present application, the sample panoramic original images may be shared among the intelligent servers 110, for example, as shown in fig. 1, the still images and the video frame images in the same image database 120 may be shared, or the sample panoramic original images may be stored in the intelligent servers 110 in a distributed manner, and data sharing is implemented through local area network transmission or other manners.
Specifically, when setting the sample panoramic original image, the intelligent server may adopt, but is not limited to, at least one of the following modes:
mode 1: acquiring a specified static image, taking the obtained static image as a sample panoramic original image, intercepting a local original image from the sample panoramic original image, extracting original image characteristics of the local original image, and storing the original image characteristics in a storage area associated with the panoramic compressed image.
For example, referring to fig. 2a, taking an arbitrary still image (hereinafter referred to as a still image i) as an example, assuming that the smart server sets a sample panoramic original image based on the still image i, the method is as follows:
the intelligent server acquires a specified static image i containing a certain street scene at a certain moment from an image database, as shown in fig. 2b, the acquired static image i is taken as a corresponding sample panoramic original image, an automobile is intercepted from the static image i as a local original image according to an instruction, and the original image feature of the automobile is extracted and recorded as an original feature i; then, the intelligent server compresses the static image i to obtain a corresponding compressed image i, and then stores the original characteristic i and the compressed image i in a specified storage area in an associated manner, specifically, the original characteristic i and the compressed image i can be a user-defined area on the intelligent server or a user-specified area in an image database.
Further, the attribute information i of the local original image may also be stored together with the compressed image i and the original feature i, and the attribute information of the local original image may include, but is not limited to, at least one of the following: image address, image resolution, image size, image depth.
By adopting the method 1, the original image characteristics of the local original image can be extracted from each preselected static image, and the original image characteristics of each local original image and the corresponding panoramic compressed image are respectively stored in the storage area in a related manner.
Mode 2: obtaining a specified video data stream, carrying out YUV coding on the video data stream, taking a certain video frame image obtained after coding as a sample panoramic original image, intercepting a local original image from the sample panoramic original image, extracting original image characteristics of the local original image, and storing the original image characteristics in a storage area associated with the panoramic compressed image.
For example, referring to fig. 3a, taking an arbitrary segment of video data stream as an example (hereinafter referred to as a video data stream m), assuming that the intelligent server sets a sample panoramic original image based on the video data stream m, the method is as follows:
the intelligent server obtains a specified section of video data stream m containing a park scene in a certain time period from an image database, and as shown in fig. 3b and fig. 3c, YUV encoding is carried out on the video data stream m, a certain video frame image m obtained after encoding is used as a corresponding sample panoramic original image, a 'person' is intercepted from the video frame image m as a local original image according to an indication, and the original image feature of the 'person' is extracted and recorded as an original feature m; then, the intelligent server compresses the video frame coding image m to obtain a corresponding compressed image m, and then stores the original feature m in a storage area, wherein the storage area is associated with the original feature m and the compressed image m and is stored in a designated storage area, specifically, a user-defined area on the intelligent server or a user-designated area in an image database.
By adopting the mode 2, the original image characteristics of the local original images can be extracted from the pre-selected video data stream, and the original image characteristics of each local original image and the corresponding panoramic compressed image are respectively stored in the storage area in a related manner.
In both the method 1 and the method 2, the following optional methods can be adopted to extract the original image features of the local original image: the Sliding Window method (SWA) is adopted for the sample panoramic original image in the mode 1 and the mode 2, the position information of the specified target in the sample panoramic original image is extracted, and then the local original image with the preset size is intercepted from the sample panoramic original image according to the position information.
Furthermore, no matter the mode 1 or the mode 2 is adopted to obtain the original image characteristics of the local original images, the finally obtained original image characteristics of each local original image can be stored in each intelligent server in a distributed mode or stored in a shared database in a centralized mode; taking the former as an example, the storage form and transmission manner of the original image features of each local original image in the intelligent server are shown in fig. 4.
Based on the above preprocessing process, referring to fig. 5, in the embodiment of the present application, a detailed flow of the intelligent server for image search is as follows:
step 500: and the intelligent server sends the panoramic compressed image to the client.
In specific implementation, after the client logs in the intelligent server, the panoramic compressed image which can be used for image search can be obtained from various multimedia data provided by the intelligent server.
For example, referring to fig. 6a, assuming that the intelligent server is a video website server, after the client logs in the video website server, the client may obtain a specified video data stream from the video website server, and extract a panoramic compressed image from the obtained video data stream.
Step 510: and the client marks the marquee in the obtained panoramic compressed image, carries the panoramic compressed image marked with the marquee in an image search request and sends the image search request to the intelligent server.
In specific implementation, after the client presents the panoramic compressed image to the user, based on a click operation and a drag operation triggered by the user in an operation interface, relevant information of a marquee set by the user, including position information and size information, can be determined.
For example, referring to fig. 6b, a user may perform a click operation and a drag operation in an operation interface through an external device (a mouse, a touch pad, or the like), in this example, the user drags a marquee to a character a, and includes the character a by adjusting the size of the marquee, and after the user clicks and confirms, the client fixes the state of the marquee at this time, which is regarded as that the marquee of the object to be searched (the character a) is completed. And then, the client carries the panoramic compressed image marked with the marquee in an image search request and sends the panoramic compressed image to one or more intelligent servers.
Step 520: and the intelligent server receives an image search request sent by the client based on the panoramic compressed image, wherein the image search request carries the information of the marquee.
In the embodiment of the application, the image search request sent by the client based on the panoramic compressed image can be confirmed according to whether the intelligent server receives the panoramic compressed image.
Step 530: and the intelligent server intercepts the corresponding target object compressed image from the obtained panoramic compressed image based on the obtained marquee information, and extracts the compressed image characteristics of the target object compressed image.
Specifically, the intelligent server intercepts a compressed image of a target object according to the panoramic compressed image marked with the marquee and the position information and the size information of the marquee in the panoramic compressed image, and extracts the compressed image characteristics of the compressed image of the target object.
For example: as shown in fig. 6c, the extracted compressed image of the target object in the embodiment of the present application is a compressed image a of the corresponding person a, and the extracted compressed image feature in the compressed image a of the target object is a compressed image feature Ma。
Step 540: and the intelligent server respectively obtains the matching degrees between the compressed image features and the pre-stored original image features of each local original image, and selects target original image features with matching degrees meeting preset matching conditions based on the obtained matching degrees.
The original features of each pre-stored local original image are stored in association with the panoramic compressed image in a designated storage area in advance by the intelligent server before step 500, and the specific storage area may be a user-defined area on the intelligent server or a user-designated area in an image database.
Taking an original image feature of any local original image (hereinafter referred to as an original image feature i) as an example, the intelligent server may calculate a cosine value of an included angle between the original image feature i and a compressed image feature of the compressed image of the target object by using a cosine similarity method, and use the obtained cosine value as a matching degree between the original image feature i and the compressed image feature of the compressed image of the target object.
For example: the intelligent server reads the original image characteristics of the local original image as a vector N0：x1，x2，…，x5000The compressed image feature of the compressed image of the object is a vector M0：y1，y2，…，y5000Then, the cosine similarity method is adopted to calculate the vector N0And vector M0Cosine value of included angle between
At this time, the value of cos θ is taken as the original image feature N0Compressed image characteristic M of obtained compressed image of target object0The degree of match between them.
The above process only takes any one original image feature as an example, a calculation process of the matching degree is introduced, and the matching degrees of the other original image features can be obtained in the same manner, which is not described herein again.
Further, after obtaining the matching degrees between the compressed image features of the target object compressed image and the original image features of each local original image, the process of selecting the target original image features with the matching degrees meeting the preset matching conditions by the intelligent server is as follows1～NnAnd the original image characteristics N of each local original image obtained by the cosine similarity method1～NnAnd the matching degree with the compressed image characteristic M of the compressed image of the target object is recorded as A1～AnThe total number of n; then, referring to fig. 7, the specific matching process is as follows:
step 5400: the intelligent server reads NxAnd calculating NxThe degree of match between M and M is denoted AxWhere x is set to an initial value of 1.
In step 5401, if the matching degree a currently read by the smart server is not equal to the matching degree a currently read by the smart serverxIs A1Then, another matching degree A of the last reading is setx-1Is 0;
step 5402: the intelligent server side reads the matching degree A currentlyxCorrelated original image features NxAs a new target original image feature.
Step 5403: the intelligent server side continuously keeps matching degree Ax-1Correlated original image features Nx-1As the target original image feature.
Step 5404: the intelligent server reads the current value of x, compares the current value of x with the total matching degree n, and determines whether x is smaller than n, that is, x < n? If so, go to step 5405, otherwise, go to step 5406.
Step 5405: the smart server increases the value of x by 1, that is, x is x +1, and then step 5400 is performed.
Step 5406: and the intelligent server outputs the currently stored target original image characteristics Nx.
Step 550: and the intelligent server side carries out image search based on the obtained target original image characteristics to obtain a corresponding search result.
In the embodiment of the present application, as shown in fig. 8a and 8b, when step 540 is executed, the intelligent server may perform image search in, but is not limited to, the following manners:
mode 1: and searching in an image database by adopting a Scale-invariant feature transform (SIFT) matching method.
Mode 2: and searching in an image database based on the characteristics of the target original image by adopting a Fast Nearest Neighbor Search clustering (FLANN).
Step 560: and the intelligent server returns the obtained search result to the client.
In the embodiment of the present application, when step 550 is executed, the intelligent server executes different operations based on different scenarios, specifically as follows:
scene 1: and if the number of the obtained target images is 1, the intelligent server directly sends the obtained target images to the client.
Scene 2: and if the number of the obtained target images is more than 1, the intelligent server sorts the target images according to the initial letters of the names of the target images according to the alphabetical order, and sends the target images to the client according to the sorting result.
Fig. 9 is a schematic structural diagram of an image searching apparatus provided in an embodiment of the present application, and as shown in fig. 9, the image searching apparatus includes a communication module 901, an extraction module 902, a matching module 903, and a search module 904; wherein:
a communication module 901, configured to receive an image search request sent by a client based on a panoramic compressed image, where the image search request carries marquee information; the panoramic compressed image is sent to the client in advance by an intelligent server;
an extracting module 902, configured to intercept a corresponding target object compressed image from the panoramic compressed image based on the cull box information, and extract a compressed image feature of the target object compressed image;
a matching module 903, configured to obtain matching degrees between the compressed image features and original image features of each pre-stored local original image, and select, based on each obtained matching degree, a target original image feature whose matching degree meets a preset matching condition, where the local original image is captured from an appointed sample panoramic original image;
and a searching module 904, configured to perform image searching based on the target original image feature to obtain a corresponding search result.
In an optional embodiment, before receiving an image search request sent by a client based on a panoramic compressed image, the extracting module 902 is specifically configured to perform at least one of the following operations:
acquiring a specified static image, taking the static image as a sample panoramic original image, intercepting a local original image from the sample panoramic original image, extracting original image characteristics of the local original image, and storing the original image characteristics in a storage area associated with the panoramic compressed image.
Acquiring a specified video data stream, decoding the video data stream, taking an obtained video frame image as a sample panoramic original image, intercepting a local original image from the sample panoramic original image, extracting original image characteristics of the local original image, and storing the original image characteristics in a storage area associated with the panoramic compressed image.
In an optional embodiment, when the local original image is intercepted from the sample panoramic original image, the extracting module 902 is specifically configured to:
extracting the position information of a specified target in the sample panoramic original image by adopting a sliding window method SWA;
and according to the position information, intercepting a local original image with a preset size from the sample panoramic original image.
In an optional embodiment, when obtaining the matching degrees between the compressed image features and the pre-stored original image features of each local original image, the matching module 903 is specifically configured to:
respectively aiming at the original image characteristics of each local original image, executing the following operations:
calculating cosine values of included angles between original image features of a local original image and compressed image features of a compressed image of a target object by adopting a cosine similarity method;
and taking the cosine value as the matching degree between the original image characteristic of the local original image and the compressed image characteristic of the target object compressed image.
In an optional embodiment, when the target original image features with matching degrees meeting the preset matching conditions are selected based on the obtained matching degrees, the matching module 903 is specifically configured to:
sequentially reading the obtained matching degrees, executing the following operations every time when one matching degree is read, and outputting the target original image characteristics until all the matching degrees are read:
comparing the matching degree read currently with the matching degree read last time;
if the matching degree read currently is judged to be larger than the matching degree read last time, the original image features related to the matching degree read currently are used as new target original image features;
and if the matching degree read currently is judged to be smaller than the matching degree read last time, keeping the original image features associated with the matching degree as target original image features.
In an optional embodiment, when performing an image search based on the target original image feature and obtaining a corresponding search result, the search module 904 is specifically configured to:
and performing image search in an image database based on the target original image characteristics by adopting a scale invariant feature transform matching method SIFT or a fast nearest neighbor search algorithm FLANN, and taking the obtained target image as the search result.
In an alternative embodiment, after obtaining the corresponding search result, the communication module 901 is specifically configured to perform any one of the following operations:
and if the number of the target images is 1, directly sending the obtained target images to the client.
And if the number of the target images is more than 1, sorting the target images according to the alphabetical order according to the initial letters of the names of the target images, and sending the target images to the client according to a sorting result.
According to an aspect of the application, a computer program product or computer program is provided, comprising computer instructions, the computer instructions being stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform any of the image search methods in the above embodiments. The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
Based on the same inventive concept as the embodiment of the application, the embodiment of the application also provides electronic equipment which can be used for image searching. In one embodiment, the electronic device may be a server, a terminal device, or other electronic device. In this embodiment, the electronic device may be configured as shown in fig. 10, and include a memory 1001, a communication interface 1003, and one or more processors 1002.
A memory 1001 for storing computer programs executed by the processor 1002. The memory 1001 may mainly include a storage program area and a storage data area, where the storage program area may store an operating system, a program required for running an instant messaging function, and the like; the storage data area can store various instant messaging information, operation instruction sets and the like.
The processor 1002 may include one or more Central Processing Units (CPUs), a digital Processing Unit, and the like. The processor 1002 is configured to implement the image search method when a computer program stored in the memory 1001 is called.
The communication interface 1003 is used for communicating with a terminal device and other servers.
In the embodiment of the present application, the specific connection medium among the memory 1001, the communication interface 1003, and the processor 1002 is not limited. In the embodiment of the present application, the memory 1001 and the processor 1002 are connected by a bus 1004 in fig. 10, the bus 1004 is represented by a thick line in fig. 10, and the connection manner between other components is merely illustrative and is not limited thereto. The bus 1004 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 10, but this is not intended to represent only one bus or type of bus.
In the embodiment of the application, an intelligent server intercepts corresponding target object compressed images from panoramic compressed images and extracts compressed image features based on the information of a marquee carried by an image search request sent by a client, and performs image search based on target original features, of the original image features corresponding to each preset local original image, of which the matching degree with the compressed image features meets preset matching conditions, so as to obtain the target images; therefore, the target original image features which are most similar to the compressed image features are adopted to replace the compressed image features for image search, so that more accurate target images can be obtained, the image search accuracy is effectively improved on the premise of not increasing the system operation load, and meanwhile, the search efficiency is also ensured.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (10)
1. An image search method, comprising:
receiving an image search request sent by a client based on a panoramic compressed image, wherein the image search request carries marquee information; the panoramic compressed image is sent to the client in advance by an intelligent server;
based on the marquee information, intercepting a corresponding target object compressed image from the panoramic compressed image, and extracting the compressed image characteristics of the target object compressed image;
respectively obtaining the matching degrees between the compressed image features and the original image features of each pre-stored local original image, and selecting target original image features with matching degrees meeting preset matching conditions based on the obtained matching degrees, wherein the local original images are intercepted from the appointed sample panoramic original images;
and searching images based on the target original image characteristics to obtain corresponding search results.
2. The method of claim 1, wherein before receiving an image search request sent by a client based on the panoramic compressed image, at least one of the following operations is further performed:
acquiring a specified static image, taking the static image as a sample panoramic original image, intercepting a local original image from the sample panoramic original image, extracting original image characteristics of the local original image, and storing the original image characteristics in a storage area associated with the panoramic compressed image;
acquiring a specified video data stream, decoding the video data stream, taking an obtained video frame image as a sample panoramic original image, intercepting a local original image from the sample panoramic original image, extracting original image characteristics of the local original image, and storing the original image characteristics in a storage area associated with the panoramic compressed image.
3. The method of claim 2, wherein truncating a local original image from the sample panoramic original image comprises:
extracting the position information of a specified target in the sample panoramic original image by adopting a sliding window method SWA;
and according to the position information, intercepting a local original image with a preset size from the sample panoramic original image.
4. The method according to claim 1, wherein obtaining the matching degree between the compressed image features and the pre-stored original image features of the respective local original images comprises:
respectively aiming at the original image characteristics of each local original image, executing the following operations:
calculating cosine values of included angles between original image features of a local original image and compressed image features of a compressed image of a target object by adopting a cosine similarity method;
and taking the cosine value as the matching degree between the original image characteristic of the local original image and the compressed image characteristic of the target object compressed image.
5. The method as claimed in claim 1, 2 or 3, wherein selecting the target original image features with matching degrees meeting the preset matching conditions based on the obtained matching degrees comprises:
sequentially reading the obtained matching degrees, executing the following operations every time when one matching degree is read, and outputting the target original image characteristics until all the matching degrees are read:
comparing the matching degree read currently with the matching degree read last time;
if the matching degree read currently is judged to be larger than the matching degree read last time, the original image features related to the matching degree read currently are used as new target original image features;
and if the matching degree read currently is judged to be smaller than the matching degree read last time, keeping the original image characteristics associated with the matching degree read last time as target original image characteristics.
6. The method of claim 1, 2 or 3, wherein performing an image search based on the target original image features to obtain corresponding search results comprises:
and performing image search in an image database based on the target original image characteristics by adopting a scale invariant feature transform matching method SIFT or a fast nearest neighbor search algorithm FLANN, and taking the obtained target image as the search result.
7. The method of claim 5, wherein obtaining the corresponding search result further comprises any one of:
if the number of the target images is 1, directly sending the obtained target images to the client;
and if the number of the target images is more than 1, sorting the target images according to the alphabetical order according to the initial letters of the names of the target images, and sending the target images to the client according to a sorting result.
8. An image search apparatus characterized by comprising:
the communication module is used for receiving an image search request sent by a client based on the panoramic compressed image, wherein the image search request carries marquee information; the panoramic compressed image is sent to the client in advance by an intelligent server;
the extraction module is used for intercepting a corresponding target object compressed image from the panoramic compressed image based on the marquee information and extracting the compressed image characteristics of the target object compressed image;
the matching module is used for respectively obtaining the compressed image features and the matching degrees between the pre-stored original image features of each local original image, and selecting target original image features with the matching degrees meeting preset matching conditions based on the obtained matching degrees, wherein the local original images are intercepted from the appointed sample panoramic original images;
and the searching module is used for searching images based on the target original image characteristics to obtain corresponding searching results.
9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out an image search method according to any one of claims 1 to 7.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the image search method according to any one of claims 1 to 7 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110974274.0A CN113806577A (en) | 2021-08-24 | 2021-08-24 | Image searching method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110974274.0A CN113806577A (en) | 2021-08-24 | 2021-08-24 | Image searching method and device, storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113806577A true CN113806577A (en) | 2021-12-17 |
Family
ID=78941570
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110974274.0A Pending CN113806577A (en) | 2021-08-24 | 2021-08-24 | Image searching method and device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113806577A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005339075A (en) * | 2004-05-26 | 2005-12-08 | Nippon Telegr & Teleph Corp <Ntt> | Image searching device, image searching method, program for the method, and recording medium recorded with the program |
| CN106294798A (en) * | 2016-08-15 | 2017-01-04 | 华为技术有限公司 | A kind of images share method based on thumbnail and terminal |
| CN106445939A (en) * | 2015-08-06 | 2017-02-22 | 阿里巴巴集团控股有限公司 | Image retrieval, image information acquisition and image identification methods and apparatuses, and image identification system |
| CN110110113A (en) * | 2019-05-20 | 2019-08-09 | 重庆紫光华山智安科技有限公司 | Image search method, system and electronic device |
| CN110769262A (en) * | 2018-07-25 | 2020-02-07 | 清华大学 | Video image compression method and system |
| CN110766048A (en) * | 2019-09-18 | 2020-02-07 | 平安科技（深圳）有限公司 | Image content identification method and device, computer equipment and storage medium |
| CN112819683A (en) * | 2021-01-19 | 2021-05-18 | 武汉旷视金智科技有限公司 | Image processing method, image processing device, computer equipment and storage medium |
-
2021
- 2021-08-24 CN CN202110974274.0A patent/CN113806577A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005339075A (en) * | 2004-05-26 | 2005-12-08 | Nippon Telegr & Teleph Corp <Ntt> | Image searching device, image searching method, program for the method, and recording medium recorded with the program |
| CN106445939A (en) * | 2015-08-06 | 2017-02-22 | 阿里巴巴集团控股有限公司 | Image retrieval, image information acquisition and image identification methods and apparatuses, and image identification system |
| CN106294798A (en) * | 2016-08-15 | 2017-01-04 | 华为技术有限公司 | A kind of images share method based on thumbnail and terminal |
| CN110769262A (en) * | 2018-07-25 | 2020-02-07 | 清华大学 | Video image compression method and system |
| CN110110113A (en) * | 2019-05-20 | 2019-08-09 | 重庆紫光华山智安科技有限公司 | Image search method, system and electronic device |
| CN110766048A (en) * | 2019-09-18 | 2020-02-07 | 平安科技（深圳）有限公司 | Image content identification method and device, computer equipment and storage medium |
| CN112819683A (en) * | 2021-01-19 | 2021-05-18 | 武汉旷视金智科技有限公司 | Image processing method, image processing device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10885100B2 (en) | Thumbnail-based image sharing method and terminal | |
| CN109871490B (en) | Media resource matching method and device, storage medium and computer equipment | |
| CN110598014B (en) | Multimedia data processing method, device and storage medium | |
| KR102354692B1 (en) | Rule-based video importance analysis | |
| US11450027B2 (en) | Method and electronic device for processing videos | |
| CN111327945B (en) | Method and apparatus for segmenting video | |
| CN111651636B (en) | Video similar segment searching method and device | |
| CN112330685B (en) | Image segmentation model training method, image segmentation device and electronic equipment | |
| US20160371565A1 (en) | Ranking Based on Facial Image Analysis | |
| KR102222087B1 (en) | Image recognition method and apparatus based on augmented reality | |
| CN108513139B (en) | Virtual object identification method, device, storage medium and equipment in video live broadcast | |
| US20100250588A1 (en) | Image searching system and image searching method | |
| Costa et al. | Image phylogeny forests reconstruction | |
| US11714921B2 (en) | Image processing method with ash code on local feature vectors, image processing device and storage medium | |
| CN110163061B (en) | Method, apparatus, device and computer readable medium for extracting video fingerprint | |
| CN113762280A (en) | Image category identification method, device and medium | |
| US20130343618A1 (en) | Searching for Events by Attendants | |
| CN110505513A (en) | A kind of video interception method, apparatus, electronic equipment and storage medium | |
| CN111062426A (en) | Method, device, electronic equipment and medium for establishing training set | |
| JP2003030672A (en) | Document recognition device, method, program and storage medium | |
| CN109697240B (en) | Image retrieval method and device based on features | |
| US20050105798A1 (en) | Method and device for verifying multimedia entities and in particular for verifying digital images | |
| CN113297499A (en) | Information recommendation system, method, computer equipment and storage medium | |
| CN109213515B (en) | Multi-platform lower buried point normalization method and device and electronic equipment | |
| CN108446737B (en) | Method and device for identifying objects |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |