CN116010634A - Mutual information rearrangement method, device, equipment and medium for image retrieval - Google Patents
Mutual information rearrangement method, device, equipment and medium for image retrieval Download PDFInfo
- Publication number
- CN116010634A CN116010634A CN202211729146.0A CN202211729146A CN116010634A CN 116010634 A CN116010634 A CN 116010634A CN 202211729146 A CN202211729146 A CN 202211729146A CN 116010634 A CN116010634 A CN 116010634A
- Authority
- CN
- China
- Prior art keywords
- database
- image
- images
- similarity
- database images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Processing Or Creating Images (AREA)
Abstract
The present invention relates to the field of image retrieval technologies, and in particular, to a method, an apparatus, a device, and a medium for rearrangement of mutual information for image retrieval. The method comprises the steps of retrieving the first K database images which are most similar to an image to be retrieved from a database, arranging the first K database images into an initial sequence, screening associated images associated with the database images from the initial sequence according to the sequence identification numbers of the database images in the initial sequence, taking the average similarity value of all the associated images and the database images as the update similarity of the database images, rearranging the first K database images into a rearrangement sequence according to the update similarity, updating the similarity of each database image according to the mutual information among the database images, rearranging the initial sequence according to the update similarity, so that the rearrangement result is more orderly, the difference between the correct retrieval result and the error retrieval result is more obvious, and the accuracy of image retrieval is improved.
Description
Technical Field
The present invention relates to the field of image retrieval technologies, and in particular, to a method, an apparatus, a device, and a medium for rearrangement of mutual information for image retrieval.
Background
At present, with the development of artificial intelligence technology, image retrieval technology is widely applied to a plurality of application scenes, such as intelligent security, intelligent communities, intelligent campuses, intelligent traffic and the like, wherein the image retrieval includes face image retrieval, building image retrieval, vehicle image retrieval and the like. The existing image retrieval method is that the images to be retrieved are ranked according to the similarity between the images to be retrieved and all stored images in a retrieval database, and K most similar retrieval images with the largest similarity are returned.
However, because the collecting working conditions of the images are different, such as the influence of factors such as image blurring and angle difference, and the like, and the information in the images is blocked, wrong sequencing occurs, and for the K most similar searched images, the threshold value is difficult to determine to separate the correct search result and the wrong search result, so that the accuracy of image search is lower. Therefore, how to improve the accuracy of image retrieval is a problem to be solved.
Disclosure of Invention
In view of the above, the embodiments of the present invention provide a mutual information rearrangement method, apparatus, device, and medium for image retrieval, so as to solve the problem of improving the accuracy of image retrieval.
In a first aspect, an embodiment of the present invention provides a mutual information rearrangement method for image retrieval, where the mutual information rearrangement method includes:
according to the similarity between the images, searching from a preset database to obtain the top K database images which are most similar to the acquired images to be searched, wherein K is an integer larger than zero;
according to the similarity between the first K database images and the images to be searched, arranging the first K database images to obtain an initial sequence, wherein the initial sequence comprises the first K database images and the corresponding sequencing identification numbers;
for any database image, screening at least one associated image associated with the database image from the initial sequence according to the ordering identification number of the database image;
taking the average value of the similarity between each associated image and the database image as the updated similarity between the database image and the image to be searched;
And rearranging the first K database images according to the updated similarity between all database images and the images to be searched to obtain a rearranged sequence.
In a second aspect, an embodiment of the present invention provides a mutual information rearrangement apparatus for image retrieval, the mutual information rearrangement apparatus including:
the image retrieval module is used for retrieving the top K database images which are most similar to the acquired images to be retrieved from a preset database according to the similarity between the images, wherein K is an integer larger than zero;
the initial sequence module is used for arranging the first K database images according to the similarity between the first K database images and the images to be searched to obtain an initial sequence, wherein the initial sequence comprises the first K database images and the corresponding sequence identification numbers;
the image screening module is used for screening at least one associated image associated with the database images from the initial sequence according to the ordering identification number of the database images aiming at any database image;
the similarity updating module is used for taking the average value of the similarity between each associated image and the database image as the updated similarity between the database image and the image to be searched;
And the image rearrangement module is used for rearranging the first K database images according to the updated similarity between all database images and the images to be searched to obtain a rearrangement sequence.
In a third aspect, an embodiment of the present invention provides a computer device, the computer device comprising a processor, a memory and a computer program stored in the memory and executable on the processor, the processor implementing the mutual information rearrangement method for image retrieval as described in the first aspect when executing the computer program.
In a fourth aspect, an embodiment of the present invention provides a computer readable storage medium storing a computer program which, when executed by a processor, implements the mutual information rearrangement method for image retrieval according to the first aspect.
Compared with the prior art, the embodiment of the invention has the beneficial effects that:
according to the similarity between images, searching the first K database images which are most similar to the acquired images to be searched from a preset database, according to the similarity between the first K database images and the images to be searched, arranging the first K database images to obtain an initial sequence, wherein the initial sequence comprises the first K database images and the corresponding sorting identification numbers thereof, for any database image, at least one associated image associated with the database image is screened out from the initial sequence according to the sorting identification numbers of the database images, the average value of the similarity between each associated image and the database image is used as the update similarity between the database image and the images to be searched, the first K database images are rearranged according to the update similarity between all the database images and the images to be searched, so as to obtain a rearrangement sequence, the update similarity of each initial image is updated through the mutual information between the initial images in the initial sequence, and the initial sequence is rearranged according to the update similarity, so that the rearrangement results are more ordered, and the difference between correct retrieval results and wrong retrieval results is more obvious, and the accuracy of the rearrangement results is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic view of an application environment of a mutual information rearrangement method for image retrieval according to a first embodiment of the present invention;
fig. 2 is a flow chart of a mutual information rearrangement method for image retrieval according to a first embodiment of the present invention;
fig. 3 is a schematic flow chart of batch acceleration processing of a mutual information rearrangement method for image retrieval according to a second embodiment of the present invention;
fig. 4 is a schematic structural diagram of a mutual information rearrangement device for image retrieval according to a third embodiment of the present invention;
fig. 5 is a schematic structural diagram of a computer device according to a fourth embodiment of the present invention.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, techniques, etc., in order to provide a thorough understanding of the embodiments of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
As used in the present description and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".
Furthermore, the terms "first," "second," "third," and the like in the description of the present specification and in the appended claims, are used for distinguishing between descriptions and not necessarily for indicating or implying a relative importance.
Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the invention. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.
It should be understood that the sequence numbers of the steps in the following embodiments do not mean the order of execution, and the execution order of the processes should be determined by the functions and the internal logic, and should not be construed as limiting the implementation process of the embodiments of the present invention.
In order to illustrate the technical scheme of the invention, the following description is made by specific examples.
The mutual information rearrangement method for image retrieval provided by the embodiment of the invention can be applied to an application environment as shown in fig. 1, wherein a client communicates with a server. The client includes, but is not limited to, a palm top computer, a desktop computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cloud terminal device, a personal digital assistant (personal digital assistant, PDA), and other computer devices. The server may be an independent server, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.
The server may communicate with at least one image capturing device to obtain an image to be retrieved from the image capturing device, where the image capturing device includes, but is not limited to, a camera, a video recorder, a handheld camera device, etc., and the image capturing device may be deployed in a variety of application scenarios, such as an intelligent security scenario, an intelligent community scenario, an intelligent campus scenario, etc.
It will be appreciated that in the specific embodiments of the present application, related data such as facial images, personal identification, etc. are referred to, and when the embodiments of the present application are applied to specific products or technologies, user permissions or consents need to be obtained, and the collection, use, and processing of related data need to comply with relevant laws and regulations and standards of relevant countries and regions.
Referring to fig. 2, a flow chart of a mutual information rearrangement method for image retrieval according to a first embodiment of the present invention is provided, where the mutual information rearrangement method for image retrieval may be applied to a client in fig. 1, a computer device corresponding to the client is connected to a server to obtain an image to be retrieved, a database is deployed in the server to be used as image retrieval, after the computer device corresponding to the client obtains the image to be retrieved, a retrieval instruction is generated and sent to the server, the image to be retrieved is retrieved from the database deployed in the server, and a retrieval result is returned to the computer device corresponding to the client. As shown in fig. 2, the mutual information rearrangement method for image retrieval may include the steps of:
Step S201, searching from a preset database to obtain the top K database images most similar to the acquired images to be searched according to the similarity between the images.
The similarity can be represented by Euclidean distance, cosine similarity and Manhattan distance equidistant measurement results, a preset database can be deployed in a server, a plurality of database images are stored in the database, the images to be searched can be images required to be searched, the database images can be images stored in the database, and K is an integer larger than zero.
Specifically, the embodiment may be applied to a face image retrieval task, where both the image to be retrieved and the database image are face images, and the object of the face image retrieval task is to retrieve the database image belonging to the same person as the image to be retrieved from the database.
When the Top-K mode is adopted to acquire the Top K database images most similar to the acquired images to be searched, the number of face images stored in the database by different people is possibly different, so that the fixed K value is difficult to be effectively applied to different people, the database images belonging to the same person are searched for the images to be searched of different people respectively, if the number of face images stored in the database by one person is larger than the K value, the search result is missed, namely, the database images belonging to the same person as the images to be searched exist in the database, and if the number of face images stored in the database by one person is smaller than the K value, the search result is misdetected, namely, the database images not belonging to the same person as the images to be searched in the database are also used as the search result. The Top-K mode arranges the Top K database images most similar to the images to be searched according to the similarity, so that the database images with the higher rank are generally considered to have higher probability of belonging to the same person with the images to be searched, and the accuracy of the search result can be improved if the arrangement result is truncated by determining a proper interception point according to the similarity change trend corresponding to the arrangement result.
However, there is often diversity in the face image when acquiring, for example, the face area in the face image is blocked by objects such as mask, cap, glasses, or the angle difference, image blurring, illumination condition, and other external factors during face image acquisition may cause that the similarity between multiple face images of the same person is smaller, or the similarity between face images of different persons is larger, so that the retrieval result acquired by using the Top-K method has a case of disordered sequence, that is, the sequence of the database image of the same person as the image to be retrieved is earlier than that of the database image of the same person as the image to be retrieved. And because the sequence of the search results is disordered, the implementer is more difficult to determine the proper cutoff point to cut off the search results so as to acquire all database images belonging to the same person as the image to be searched, on the other hand, even if the obtained search results are not disordered, if the similarity of the last database image belonging to the same person as the image to be searched in the sequence of the search results and the first database image not belonging to the same person as the image to be searched in the sequence of the search results are relatively close, the implementer is still difficult to determine the proper cutoff point from the relatively smooth similarity change trend.
Aiming at the problems, in the embodiment, top-K searching is adopted to obtain the Top K database images most similar to the images to be searched, and the Top K database images are rearranged subsequently, so that the condition of disordered searching result ordering is reduced, the cut-off point is easier to determine, and the accuracy of the searching result is improved.
Optionally, the database comprises N database images;
according to the similarity between the images, retrieving the top K database images which are most similar to the acquired images to be retrieved from the preset database comprises the following steps:
calculating the similarity of each database image and the image to be searched to obtain first similarity of N corresponding database images;
and screening the first K first similarity with the largest first similarity value from the N first similarity values, and determining the database images corresponding to the first K first similarity values as the first K database images which are most similar to the acquired image to be searched.
Where N is an integer greater than K, the first similarity may be used to characterize difference information between the database image and the image to be retrieved.
Specifically, a first similarity d 1 The calculation method of (2) can be expressed as:
wherein, the liquid crystal display device comprises a liquid crystal display device,
may refer to a first similarity, x, between the nth database image and the image to be retrieved n Can refer to the nth database image, y can refer to the image to be retrieved, cosine (x n Y) may refer to performing cosine similarity calculation between the nth database image and the image to be retrieved, where the cosine similarity has a value range of [ -1,1]The more similar the cosine similarity is to-1, the less likely the nth database image and the image to be retrieved are, i.e., the less likely the nth database image and the image to be retrieved belong to the same person, and the more similar the cosine similarity is to-1, the more likely the nth database image and the image to be retrieved are, i.e., the nth database image and the image to be retrieved belong to the same person.
Since the image can be regarded as a two-dimensional vector, the similarity can be calculated by using a cosine value calculation mode of the vector, if the cosine value is-1, the pointing directions of the two vectors used for calculating the cosine value are completely opposite, the two vectors are completely dissimilar, if the cosine value is 1, the pointing directions of the two vectors used for calculating the cosine value are completely consistent, the two vectors are completely similar, and the cosine similarity calculation between the nth database image and the image to be searched can be expressed as follows:
wherein, |x n The I can represent the corresponding module of the nth database image, the Y can represent the corresponding module of the image to be searched, and the X can represent the corresponding module of the image to be searched n Y may refer to the point-wise multiplication and addition of the nth database image and the image to be retrieved, and the modulus may be calculated by the sum of squares of all points in the image.
In an embodiment, feature extraction can be performed on the database image and the image to be searched respectively, and then cosine similarity calculation is performed on the feature vectors extracted respectively, so that the method of feature vector representation is adopted, the calculation parameter quantity can be reduced while the influence of image noise on the accuracy of cosine similarity calculation is avoided, and the calculation efficiency is improved.
In this embodiment, cosine similarity is used to calculate the similarity between the database images and the images to be searched, and the range of the first similarity is limited to a fixed range, so that the similarity between the database images can be compared conveniently, and the database images can be ordered conveniently according to the sequence of the similarity.
According to the similarity between the images, the first K database images which are most similar to the acquired images to be searched are searched from the preset database, the database is searched in a Top-K searching mode through the similarity between the database images and the images to be searched, a preliminary searching result is obtained, a basic sequence is provided for rearrangement of the sequence of the subsequent images, database images which are obviously lower in similarity with the images to be searched are eliminated, the calculation burden of the subsequent rearrangement process is reduced, and the overall image searching efficiency and accuracy are improved.
Step S202, according to the similarity between the top K database images and the images to be searched, the top K database images are arranged to obtain an initial sequence.
The arrangement may be that the database images are combined into a sequence according to a preset sequence, where the initial sequence includes the first K database images and their corresponding sorting identification numbers, and the sorting identification numbers may be used to mark the positions of the database images in the initial sequence.
Specifically, the preset sequence may be a descending sequence or an ascending sequence of the similarity, in general, after the initial sequence is obtained, the initial sequence is directly presented to a searcher or a terminal as a search result, and the searcher or the terminal performs subsequent analysis according to the search result, for example, in this embodiment, a face image search task is taken as an example, and the search result obtained by searching can be used for various application scenes such as intelligent security, an intelligent community, an intelligent campus and the like, so as to quickly identify the identity of a target person in the image to be searched, for example, in the intelligent campus scene, the identity of the target person can be a student, a teacher, a school and other people, and since in the normal school scene, other people are not allowed to flow in and out, when the target person in the image to be searched is identified as other people, reminding information can be generated and sent to a manager.
However, if the initial sequence is manually analyzed, the accurate judgment according to the initial sequence can be ensured, but the human resource cost is high at this time and is difficult to implement in an application scene, if the analysis is performed by a computer terminal, the accurate judgment may not be obtained, for example, the target person in the previous example is identified, the preset rule in the computer terminal may be to assign the same person category to the image to be searched according to the person category to which the database image in the initial sequence belongs, and if the condition of disordered sequencing exists, the target person is identified as a non-other person category, so that the misjudgment occurs in the computer terminal. Therefore, in this embodiment, after the initial sequence is obtained, a subsequent rearrangement operation is also required for the initial sequence to ensure the accuracy of the search result.
Optionally, arranging the first K database images to obtain an initial sequence includes:
the first K database images are arranged according to descending order of the first similarity corresponding to the first K database images respectively;
and determining a sequencing identification number corresponding to each database image according to the position of each database image in the sequencing result, wherein an initial sequence is formed by the first K database images and the sequencing identification numbers corresponding to the first K database images.
Wherein, a database image corresponds to a first similarity, the descending order may refer to the order from big to small, and the arrangement may refer to the sequential combination of the first K database images into a sequence.
The position may be a sequential position of the database-image in the initial sequence, which may be represented by a ranking identification number.
In particular, the initial sequence may be expressed as [ x ] 1 ,x 2 ,…x k ,…,x K ]Wherein x is k Database image, x, which may represent the kth position from left to right in the initial sequence k I.e. its ranking identification number.
In the embodiment, the database images are arranged in a descending order, so that the similarity relation between all database images and the images to be searched is more intuitively represented, the follow-up steps are also convenient to directly take the initial sequence as a processing object, and the processing efficiency is improved.
According to the similarity between the first K database images and the images to be searched, the first K database images are arranged to obtain the initial sequence, the initial search results are arranged to provide position information for the mutual information between the database images in the initial sequence, so that the mutual information extraction between the database images in the initial sequence is facilitated, and the rearrangement efficiency of the search results is improved.
Step S203, for any database image, at least one associated image associated with the database image is screened from the initial sequence according to the sorting identification number of the database image.
Wherein the associated image may refer to an image that may be used to optimize the similarity between the database image and the image to be retrieved.
Specifically, according to the prior condition that the database image with higher similarity with the image to be searched is more likely to belong to the same person as the image to be searched, the database image before the database image is used as the related image of the image to be searched, for example, for the database images with the sorting identification number of k, the database images with the sorting identification numbers of 1 to the sorting identification number of k-1 are used as the related images of the database images with the sorting identification number of k, and at this time, the related image is provided for each database image by the global information of the search result.
Optionally, the selecting at least one associated image associated with the database image from the initial sequence according to the ranking identification number of the database image includes:
determining a sequencing identification range according to the sequencing identification number of the database image and a preset value;
in the initial sequence, all database images corresponding to the sorting identification numbers conforming to the sorting identification range are determined as associated images.
The range of the preset value can be [1, K ], and K can represent the maximum value of the sorting identification number. All ranking identification numbers that fit into the ranking identification range may refer to all ranking identification numbers that are within the ranking identification range.
In the embodiment, the sequencing identification range is determined through the preset value, the sequencing identification number of the database image and the preset value, and the associated image associated with the database image is rapidly acquired based on the prior, so that the efficiency of rearranging the image retrieval result is improved.
Optionally, determining the ordering identifier range according to the ordering identifier number of the database image and the preset value includes:
taking a preset value as a starting value of the sequencing identification range;
taking the ordering identification number of the database image as the ending value of the ordering identification range;
and determining the ordering identifier range according to the start value and the end value of the ordering identifier range.
The start value may refer to a left boundary value of the ranking identification range, and the end value may refer to a right boundary value of the ranking identification range.
Specifically, in this embodiment, taking a face image retrieval task as an example, since a database image with a larger similarity to an image to be retrieved is more likely to belong to a person with the same person as the image to be retrieved as a priori condition, a database image with a smaller ranking identification number is more likely to belong to a person with the image to be retrieved, and thus the database image with a smaller ranking identification number can be regarded as the face image of the target person as the image to be retrieved, and therefore, whether the database image with a larger ranking identification number belongs to the face image of the target person can be further determined according to the similarity between the database image with a smaller ranking identification number and the database image with a larger ranking identification number, so that the similarity corresponding to the database image with a larger ranking identification number can be updated.
Because each database image corresponding to the sorting identification number smaller than the sorting identification number corresponding to the database image is more likely to belong to the same person as the image to be searched than the database image, all database images with the sorting identification number before the sorting identification number corresponding to the database image are used as the associated images of the database images, and the similarity between the database image and the image to be searched is updated.
In this embodiment, if the preset value is set to 1, the database images x with k identification numbers are sorted k The associated image is x 1 To x k-1 。
In the embodiment, the associated image of each database image is determined according to the ordering position information, so that mutual information extraction between the database images is convenient to carry out subsequently, the similarity between the database images and the images to be searched is updated in an auxiliary mode, and the accuracy of image search is improved.
The step of screening at least one associated image associated with the database images from the initial sequence according to the sorting identification number of the database images, determining the associated image of each database image according to the position relation of the database images in the initial sequence, providing additional associated information for the database images, assisting in updating the similarity of the database images, and improving the accuracy of image retrieval.
Step S204, taking the average value of the similarity between each associated image and the database image as the updated similarity between the database image and the image to be retrieved.
The similarity between the associated image and the database image can still be calculated by adopting cosine similarity, and the average value can be used for representing the similarity between the database image and the image to be searched after combining mutual information, namely updating the similarity.
In particular, the similarity between the database image and the image to be retrieved, which essentially represents the likelihood that the database image and the image to be retrieved belong to the same person.
Optionally, taking the average value of the similarity between each associated image and the database image as the updated similarity between the database image and the image to be retrieved includes:
calculating the similarity between each associated image and the database image respectively to obtain a second similarity of the corresponding associated image;
and calculating the average value of all the second similarity to obtain the updated similarity between the database image and the image to be retrieved.
Wherein the second similarity may be used to characterize the difference information between the associated image and the database image, and the updated similarity may be used to characterize the likelihood that the database image and the image to be retrieved belong to the same person.
Specifically, the second similarity is calculated by cosine similarity, and when the calculation object is a database image and an associated image, the update similarity D may be expressed as:
wherein D is k Can represent the update similarity between the kth database image and the image to be retrieved, x k Can represent a database image with a ranking identification number k, x i Can be represented as a ranking identification number of [1, k-1 ]]Database images within range, i.e. associated images of the kth database image.
In the embodiment, the similarity calculation between the associated image and the database image is performed through cosine similarity, so that the difference information between the associated image and the database image can be effectively represented, normalization operation is not needed, the calculation is convenient and fast, and the image rearrangement efficiency can be improved.
And the step of updating the similarity between the database image and the image to be searched by taking the average value of the similarity between each associated image and the database image as the updated similarity between the database image and the image to be searched, so that additional associated information is provided for the database image, the database image is assisted in similarity updating, and the accuracy of image searching is improved.
Step S205, rearranging the first K database images according to the updated similarity between all the database images and the images to be searched to obtain a rearranged sequence.
The rearrangement sequence may refer to a search result after the rearrangement processing, and the search result may refer to the first K database images included in the initial sequence.
Specifically, the rearrangement may refer to combining the database images into a rearranged sequence in a preset order, which may be in a descending order or an ascending order of updating the similarity.
Optionally, rearranging the first K database images according to the updated similarity between all database images and the image to be retrieved, to obtain a rearranged sequence includes:
and rearranging the first K database images according to the descending order of the update similarity corresponding to the first K database images respectively to obtain a rearranged sequence.
In this embodiment, a database image corresponds to an updated similarity, and the descending order may refer to the order from large to small, and the arrangement may refer to the sequential combination of the first K database images into a sequence.
In particular, the initial sequence may be expressed as [ x ] 1 ′ ,x 2 ′ ,…x ′ k ,…,x ′ K ]Wherein x is ′ k The database image at the kth position from left to right in the reorder sequence may be represented. For example, under the task of searching face images, the rearrangement sequence can display the database images belonging to the same person as the image to be searched in front, so that And the subsequent analysis processing is carried out on the search result, and the updated similarity enables the change amplitude of the updated similarity between different database images and the images to be searched to be more obvious, and the distinguishing limit of the same person and the non-same person can be conveniently determined according to the updated similarity.
In this embodiment, the database images are rearranged in descending order, so that the updated similarity relationship between all the database images and the image to be searched is more intuitively represented.
According to the updating similarity between all the database images and the images to be searched, the first K database images are rearranged to obtain a rearrangement sequence, the chaotic region existing in the search result is ordered, the image searching precision is improved, the database images belonging to the same target with the images to be searched are enabled to be more forward in the rearrangement sequence, the database images not belonging to the same target with the images to be searched are enabled to be more backward in the rearrangement sequence, and meanwhile the database images belonging to the same target with the images to be searched and the database images not belonging to the same target with the images to be searched are enabled to be more remarkable on the distinguishing limit of the similarity.
According to the method, the updating similarity of each initial image is updated through mutual information among the initial images in the initial sequence, the initial sequence is rearranged according to the updating similarity, so that the rearrangement results are more orderly, meanwhile, the difference between the correct retrieval result and the error retrieval result is more obvious, and therefore the accuracy of image retrieval is improved.
Referring to fig. 3, a flow chart of batch acceleration processing of a mutual information rearrangement method for image retrieval according to a second embodiment of the present invention is shown, where in the mutual information rearrangement method for image retrieval, similarity update of database images may be performed in a manner of updating one by one, or in a manner of batch update, that is, in a manner of batch acceleration processing.
When the similarity update of the database images adopts a mode of updating one by one, refer to the first embodiment, and the description thereof is omitted.
When the similarity update of the database images adopts a batch update mode, in the mutual information rearrangement method for image retrieval, the process of obtaining a rearrangement sequence by batch update of the similarity after searching the front K database images which are most similar to the acquired images to be retrieved from the preset database and arranging the front database images according to the similarity between the front K database images and the images to be retrieved comprises the following steps:
step S301, calculating the similarity between any two database images to obtain a similarity matrix of the first K database images;
step S302, taking an upper triangular region of the similarity matrix, and carrying out mean value calculation on each column of the upper triangular region to obtain a mean value vector;
Step S303, rearranging the elements in the mean vector, and determining a rearranging sequence according to the rearranging result.
Wherein a single element in the similarity matrix may represent the similarity between two database images, the upper triangular region may represent the similarity between each database image and its associated image, the mean vector may represent the updated similarity of all database images, and the rearrangement result may represent the result of the rearrangement according to the updated similarity of all database images.
Specifically, for each group of database images, that is, between every two database images, similarity calculation is performed, where the similarity calculation still uses cosine similarity in embodiment one to obtain a similarity value, where the size of the similarity matrix is preset to k×k, that is, a similarity matrix of K rows and K columns, where the K column represents a database image in the initial sequence and is sequentially K, and similarity between each database image and each database image, for example, for an element value at (3, K) in the matrix, a similarity value between a database image in the initial sequence and sequentially 3 may be represented.
When the upper triangle area of the similarity matrix is taken, the upper triangle area does not comprise a cutting line, namely the upper triangle area does not comprise elements with the same number of rows and columns, and the number of rows corresponding to each element in the kth column is 1 to k-1, so that the database images with the sequence identification numbers of 1 to k-1 are used as the associated images of the database images with the sequence identification numbers of k in the first embodiment.
And (3) carrying out average value calculation on each column of the upper triangular region, namely calculating the average value of the similarity between the database images with the sequence k and each associated image, wherein the obtained average value vector can represent the update similarity of all the database images, determining that each update similarity is in the average value vector according to the sequence corresponding to the left-right direction, and determining the corresponding database image according to the sequence of the update similarity, namely establishing the corresponding relation between the update similarity and the database images.
Rearranging elements in the mean value vector, namely, the update similarity in the mean value vector according to a preset sequence to obtain a rearranging result, wherein the preset sequence can adopt descending or ascending order, and the corresponding relation between the update similarity and the database image is combined according to the rearranging result to obtain a rearranging sequence.
In this embodiment, the similarity between the database images is converted into a matrix form for representation, and the batch calculation is performed on the retrieved initial sequence rearrangement process through the matrix form, so that the time of the image retrieval rearrangement process can be greatly saved, and the efficiency of the image retrieval rearrangement is improved.
Fig. 4 shows a block diagram of a mutual information rearrangement device for image retrieval according to a third embodiment of the present invention, where the mutual information rearrangement device for image retrieval is applied to a client, a computer device corresponding to the client is connected to a server to obtain an image to be retrieved, a database is disposed in the server to be used as image retrieval, after the computer device corresponding to the client obtains the image to be retrieved, a retrieval instruction is generated and sent to the server, the image to be retrieved is retrieved from the database disposed in the server, and a retrieval result is returned to the computer device corresponding to the client. For convenience of explanation, only portions relevant to the embodiments of the present invention are shown.
Referring to fig. 4, the mutual information rearrangement apparatus for image retrieval includes:
the image retrieval module 41 is configured to retrieve, from a preset database, the first K database images most similar to the acquired image to be retrieved according to the similarity between the images, where K is an integer greater than zero;
the initial sorting module 42 is configured to sort the first K database images according to the similarity between the first K database images and the image to be retrieved, so as to obtain an initial sequence, where the initial sequence includes the first K database images and the sorting identification numbers corresponding to the first K database images;
An image screening module 43, configured to screen, for any database image, at least one associated image associated with the database image from the initial sequence according to the ranking identification number of the database image;
a similarity updating module 44, configured to take an average value of similarity between each associated image and the database image as updated similarity between the database image and the image to be retrieved;
the image rearrangement module 45 is configured to rearrange the first K database images according to the updated similarity between all database images and the images to be retrieved, so as to obtain a rearrangement sequence.
Optionally, the database comprises N database images, N being an integer greater than K;
the image retrieval module 41 includes:
the first similarity sub-module is used for calculating the similarity of each database image and the image to be searched to obtain the first similarity of N corresponding database images;
and the similarity screening sub-module is used for screening the first K first similarities with the maximum first similarity value from the N first similarities, and determining the database images corresponding to the first K first similarities as the first K database images which are the most similar to the acquired image to be searched.
Optionally, the initial sorting module 42 includes:
the first sequencing submodule is used for sequencing the first K database images according to the descending order of the first similarity corresponding to the first K database images respectively;
the identification determination submodule is used for determining a sequencing identification number corresponding to each database image according to the position of each database image in the arrangement result, and an initial sequence is formed by the first K database images and the sequencing identification numbers corresponding to the database images.
Optionally, the image filtering module 43 includes:
the range determination submodule is used for determining a sequencing identification range according to the sequencing identification number of the database image and a preset value;
and the association determination submodule is used for determining all database images corresponding to the sorting identification numbers conforming to the sorting identification range as association images in the initial sequence.
Optionally, the range determining submodule includes:
the starting value determining unit is used for taking a preset value as a starting value of the sequencing identification range;
a termination value determining unit for taking the sequence identification number of the database image as the termination value of the sequence identification range;
and the range determining unit is used for determining the ordering identifier range according to the starting value and the ending value of the ordering identifier range.
Optionally, the similarity update module 44 includes:
the second similarity sub-module is used for calculating the similarity between each associated image and the database image respectively to obtain the second similarity of the corresponding associated image;
and the average value calculation sub-module is used for calculating the average value of all the second similarity to obtain the updated similarity between the database image and the image to be retrieved.
Optionally, the image rearrangement module 45 includes:
and the second sequencing submodule rearranges the first K database images according to the descending order of the update similarity corresponding to the first K database images respectively to obtain a rearrangement sequence.
It should be noted that, because the content of the information interaction and the execution process between the modules, the sub-modules, and the units is based on the same concept as the method embodiment of the present invention, specific functions and technical effects thereof may be referred to in the method embodiment section, and will not be described herein.
Fig. 5 is a schematic structural diagram of a computer device according to a fourth embodiment of the present invention. As shown in fig. 5, the computer device of this embodiment includes: at least one processor (only one shown in fig. 5), a memory, and a computer program stored in the memory and executable on the at least one processor, the processor executing the computer program to perform the steps of any of the various embodiments of the mutual information rearrangement method for image retrieval described above.
The computer device may include, but is not limited to, a processor, a memory. It will be appreciated by those skilled in the art that fig. 5 is merely an example of a computer device and is not intended to limit the computer device, and that a computer device may include more or fewer components than shown, or may combine certain components, or different components, such as may also include a network interface, a display screen, an input device, and the like.
The processor may be a CPU, but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory includes a readable storage medium, an internal memory, etc., where the internal memory may be the memory of the computer device, the internal memory providing an environment for the execution of an operating system and computer-readable instructions in the readable storage medium. The readable storage medium may be a hard disk of a computer device, and in other embodiments may be an external storage device of the computer device, for example, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), etc. that are provided on the computer device. Further, the memory may also include both internal storage units and external storage devices of the computer device. The memory is used to store an operating system, application programs, boot loader (BootLoader), data, and other programs such as program codes of computer programs, and the like. The memory may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, the specific names of the functional units and modules are only for distinguishing from each other, and are not used for limiting the protection scope of the present invention. The specific working process of the units and modules in the above device may refer to the corresponding process in the foregoing method embodiment, which is not described herein again. The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present invention may implement all or part of the flow of the method of the above-described embodiment, and may be implemented by a computer program to instruct related hardware, and the computer program may be stored in a computer readable storage medium, where the computer program, when executed by a processor, may implement the steps of the method embodiment described above. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, executable files or in some intermediate form, etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code, a recording medium, a computer Memory, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium. Such as a U-disk, removable hard disk, magnetic or optical disk, etc. In some jurisdictions, computer readable media may not be electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.
The present invention may also be implemented as a computer program product for implementing all or part of the steps of the method embodiments described above, when the computer program product is run on a computer device, causing the computer device to execute the steps of the method embodiments described above.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided by the present invention, it should be understood that the disclosed apparatus/computer device and method may be implemented in other manners. For example, the apparatus/computer device embodiments described above are merely illustrative, e.g., the division of modules or units is merely a logical functional division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
The above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.
Claims (10)
1. A mutual information rearrangement method for image retrieval, characterized in that the mutual information rearrangement method comprises:
according to the similarity between the images, searching from a preset database to obtain the top K database images which are most similar to the acquired images to be searched, wherein K is an integer larger than zero;
According to the similarity between the first K database images and the images to be searched, arranging the first K database images to obtain an initial sequence, wherein the initial sequence comprises the first K database images and the corresponding sequencing identification numbers;
for any database image, screening at least one associated image associated with the database image from the initial sequence according to the ordering identification number of the database image;
taking the average value of the similarity between each associated image and the database image as the updated similarity between the database image and the image to be searched;
and rearranging the first K database images according to the updated similarity between all database images and the images to be searched to obtain a rearranged sequence.
2. The mutual information rearrangement method as claimed in claim 1, wherein the database contains N database images, N being an integer greater than K;
according to the similarity between the images, retrieving the top K database images which are most similar to the acquired images to be retrieved from the preset database comprises the following steps:
calculating the similarity of each database image and the image to be searched to obtain first similarity of N corresponding database images;
And screening the first K first similarity with the maximum first similarity value from the N first similarity values, and determining the database images corresponding to the first K first similarity values as the first K database images which are the most similar to the acquired image to be searched.
3. The mutual information rearrangement method as claimed in claim 2, wherein the arranging the first K database images to obtain an initial sequence includes:
arranging the first K database images according to descending order of the first similarity corresponding to the first K database images respectively;
and determining a sequencing identification number corresponding to each database image according to the position of each database image in the sequencing result, wherein the initial sequence is formed by the first K database images and the sequencing identification numbers corresponding to the first K database images.
4. The mutual information rearrangement method as recited in claim 1, wherein the screening at least one associated image associated with the database image from the initial sequence based on the ordering identification number of the database image includes:
determining a sequencing identification range according to the sequencing identification number of the database image and a preset value;
and in the initial sequence, determining all database images corresponding to the sorting identification numbers conforming to the sorting identification range as the associated images.
5. The mutual information rearrangement method of claim 4, wherein determining the ordering identifier range based on the ordering identifier number of the database image and a preset value includes:
taking the preset value as a starting value of the sequencing identification range;
taking the ordering identification number of the database image as the termination value of the ordering identification range;
and determining the ordering identifier range according to the starting value and the ending value of the ordering identifier range.
6. The mutual information rearrangement method according to claim 1, wherein the taking an average value of the similarity between each associated image and the database image as the updated similarity between the database image and the image to be retrieved includes:
calculating the similarity between each associated image and the database image respectively to obtain a second similarity of the corresponding associated image;
and calculating the average value of all the second similarity to obtain the updated similarity between the database image and the image to be retrieved.
7. The mutual information rearrangement method as claimed in any one of claims 1 to 6, wherein rearranging the first K database images according to updated similarity between all database images and the image to be retrieved to obtain a rearranged sequence includes:
And rearranging the first K database images according to the descending order of the update similarity corresponding to the first K database images respectively to obtain the rearrangement sequence.
8. A mutual information rearrangement apparatus for image retrieval, characterized in that the mutual information rearrangement apparatus comprises:
the image retrieval module is used for retrieving the top K database images which are most similar to the acquired images to be retrieved from a preset database according to the similarity between the images, wherein K is an integer larger than zero;
the initial sequence module is used for arranging the first K database images according to the similarity between the first K database images and the images to be searched to obtain an initial sequence, wherein the initial sequence comprises the first K database images and the corresponding sequence identification numbers;
the image screening module is used for screening at least one associated image associated with the database images from the initial sequence according to the ordering identification number of the database images aiming at any database image;
the similarity updating module is used for taking the average value of the similarity between each associated image and the database image as the updated similarity between the database image and the image to be searched;
And the image rearrangement module is used for rearranging the first K database images according to the updated similarity between all database images and the images to be searched to obtain a rearrangement sequence.
9. A computer device, characterized in that it comprises a processor, a memory and a computer program stored in the memory and executable on the processor, which processor implements the mutual information rearrangement method according to any of claims 1-7 when executing the computer program.
10. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the mutual information rearrangement method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211729146.0A CN116010634A (en) | 2022-12-30 | 2022-12-30 | Mutual information rearrangement method, device, equipment and medium for image retrieval |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211729146.0A CN116010634A (en) | 2022-12-30 | 2022-12-30 | Mutual information rearrangement method, device, equipment and medium for image retrieval |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116010634A true CN116010634A (en) | 2023-04-25 |
Family
ID=86026412
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211729146.0A Pending CN116010634A (en) | 2022-12-30 | 2022-12-30 | Mutual information rearrangement method, device, equipment and medium for image retrieval |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116010634A (en) |
-
2022
- 2022-12-30 CN CN202211729146.0A patent/CN116010634A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109376596B (en) | Face matching method, device, equipment and storage medium | |
| CN108228792B (en) | Picture retrieval method, electronic device and storage medium | |
| CN110019876B (en) | Data query method, electronic device and storage medium | |
| CN107729935B (en) | The recognition methods of similar pictures and device, server, storage medium | |
| CN110287346B (en) | Data storage method, device, server and storage medium | |
| CN111581423B (en) | Target retrieval method and device | |
| CN110941978B (en) | Face clustering method and device for unidentified personnel and storage medium | |
| CN111914775A (en) | Living body detection method, living body detection device, electronic apparatus, and storage medium | |
| CN115424053B (en) | Small sample image recognition method, device, equipment and storage medium | |
| CN110704677A (en) | Program recommendation method and device, readable storage medium and terminal equipment | |
| CN110888981A (en) | Title-based document clustering method and device, terminal equipment and medium | |
| CN115063589A (en) | Knowledge distillation-based vehicle component segmentation method and related equipment | |
| CN112232203A (en) | Pedestrian recognition method and device, electronic equipment and storage medium | |
| CN111104915B (en) | Method, device, equipment and medium for peer analysis | |
| CN116383423A (en) | Image retrieval result rearrangement method, device, computer equipment and medium | |
| CN116010634A (en) | Mutual information rearrangement method, device, equipment and medium for image retrieval | |
| CN110895555B (en) | Data retrieval method and device, storage medium and electronic device | |
| CN111522969A (en) | Image retrieval method, image retrieval device, computer equipment and storage medium | |
| CN112085788B (en) | Loop detection method and device, computer readable storage medium and mobile device | |
| CN112800258B (en) | Image retrieval method, image retrieval device, electronic equipment and computer readable storage medium | |
| US11386340B2 (en) | Method and apparatus for performing block retrieval on block to be processed of urine sediment image | |
| CN113032610B (en) | File management method, device, equipment and computer readable storage medium | |
| CN114780780A (en) | Image retrieval method, image retrieval device, electronic equipment and storage medium | |
| CN113902030A (en) | Behavior identification method and apparatus, terminal device and storage medium | |
| CN110287943B (en) | Image object recognition method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |