CN116089641A - Image retrieval method, device, equipment and medium based on self-adaptive threshold - Google Patents

Abstract

The present invention relates to the field of data retrieval technologies, and in particular, to an image retrieval method, apparatus, device, and medium based on an adaptive threshold. The method comprises the steps of searching front K reference images which are most similar to an image to be searched from a database, calculating to obtain a first-order differential curve according to a reference curve formed by the similarity corresponding to the front K reference images, sliding and intercepting a curve segment on the first-order differential curve by using a preset window, taking the curve segment corresponding to the maximum standardized value as a target curve segment according to a standardized value obtained by standardized calculation of the curve segment, determining the similarity of the reference images corresponding to the central point of the target curve segment as a similarity threshold value, screening the front K reference images by using the similarity threshold value to obtain an image search result, standardizing the reference curve to represent the distribution characteristic of the similarity, adaptively determining the similarity threshold value, further screening the front K reference images, reducing the conditions of missing detection and false detection, and improving the accuracy of image search.

Description

Image retrieval method, device, equipment and medium based on self-adaptive threshold

Technical Field

The present invention relates to the field of data retrieval technologies, and in particular, to an image retrieval method, apparatus, device, and medium based on an adaptive threshold.

Background

Currently, with the development of artificial intelligence technology, image retrieval is widely applied in many application scenarios, such as intelligent security, intelligent communities, intelligent campuses, and the like. The existing image retrieval method is usually to sort according to the similarity between the image to be retrieved and all the stored images in the retrieval database, return K most similar retrieval images with the largest similarity, or retrieve according to a similarity threshold, and return retrieval images with the similarity larger than a preset similarity threshold to the image to be retrieved.

However, in practical applications, the image retrieval generally needs to return a retrieval result which belongs to the same object as the image to be retrieved, and because the number of images stored in the retrieval database is different for different objects, the retrieval is difficult to be performed by setting a K value, and the environment conditions of the same object are different when the stored images are acquired, and the retrieval is performed by adopting a fixed similarity threshold, the condition of missed detection or false detection can be caused, and further, the accuracy of the image retrieval is not high, so how to improve the accuracy of the image retrieval becomes a problem to be solved urgently.

Disclosure of Invention

In view of the above, the embodiments of the present invention provide an image retrieval method, apparatus, device and medium based on an adaptive threshold, so as to solve the problem of low accuracy of image retrieval.

In a first aspect, an embodiment of the present invention provides an image retrieval method based on an adaptive threshold, where the image retrieval method includes:

acquiring images to be searched, and searching the first K reference images which are most similar to the images to be searched from a preset image database according to the similarity between the images, wherein K is an integer larger than zero;

forming reference curves according to the similarity corresponding to the first K reference images in sequence from large to small, and respectively performing first-order difference calculation on each similarity in the reference curves to obtain first-order difference curves;

sliding on the first-order differential curve according to a preset step length by using a preset window to obtain curve segments in the preset window after each sliding stop, and performing standardized calculation on the curve segments to obtain standardized values of corresponding curve segments;

taking the largest standardized value in all standardized values as a target value, determining a curve segment used when the target value performs standardized calculation as a target curve segment, and determining the center point of the target curve segment;

And acquiring the reference images corresponding to the center points, taking the similarity between the reference images corresponding to the center points and the images to be searched as a similarity threshold, and screening the first K reference images by using the similarity threshold to obtain an image search result.

In a second aspect, an embodiment of the present invention provides an image retrieval device based on an adaptive threshold, the image retrieval device including:

the primary retrieval module is used for acquiring images to be retrieved, and retrieving the first K reference images which are most similar to the images to be retrieved from a preset image database according to the similarity between the images, wherein K is an integer larger than zero;

the difference calculation module is used for forming reference curves according to the similarity corresponding to the first K reference images respectively in sequence from large to small, and performing first-order difference calculation on each similarity in the reference curves respectively to obtain first-order difference curves;

the curve standardization module is used for sliding on the first-order differential curve according to a preset step length by using a preset window to obtain curve segments in the preset window after each sliding stop, and carrying out standardization calculation on the curve segments to obtain standardized values of corresponding curve segments;

The center point determining module is used for taking the maximum standardized value in all the standardized values as a target value, determining a curve segment used when the target value is subjected to standardized calculation as a target curve segment, and determining the center point of the target curve segment;

and the image screening module is used for acquiring the reference images corresponding to the center points, taking the similarity between the reference images corresponding to the center points and the images to be searched as a similarity threshold, and screening the first K reference images by using the similarity threshold to obtain an image search result.

In a third aspect, an embodiment of the present invention provides a computer device, the computer device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, the processor implementing the image retrieval method according to the first aspect when executing the computer program.

In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the image retrieval method according to the first aspect.

Compared with the prior art, the embodiment of the invention has the beneficial effects that:

The method comprises the steps of obtaining images to be searched, searching front K reference images which are most similar to the images to be searched from a preset image database according to the similarity among the images, forming reference curves according to the sequence from big to small by the similarity corresponding to the front K reference images respectively, carrying out first-order difference calculation on each similarity in the reference curves to obtain first-order difference curves, sliding on the first-order difference curves according to preset step length by using a preset window to obtain curve segments in the preset window after each sliding stop, carrying out standardization calculation on the curve segments to obtain standardization values of the corresponding curve segments, taking the largest standardization value in all standardization values as a target value, determining the curve segments used when the standardization calculation is carried out on the target value as a target curve segment, determining the center point of the target curve segments, obtaining the reference images corresponding to the center point, taking the similarity between the reference images corresponding to the center point and the images to be searched as similarity threshold, screening image retrieval results by using the similarity threshold, carrying out the screening on the front K reference images to obtain image retrieval results, carrying out standardization calculation on the similarity corresponding to the first K reference images after each sliding stop, carrying out the standardization calculation on the reference images, and further carrying out the self-adaption of the similarity to the reference images according to the characteristics of the similarity distribution characteristics of the similarity, thereby improving the self-adaption of the similarity of the reference images.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic view of an application environment of an image retrieval method based on an adaptive threshold according to a first embodiment of the present invention;

fig. 2 is a flowchart of an image retrieval method based on an adaptive threshold according to a first embodiment of the present invention;

fig. 3 is a flowchart of an image retrieval method based on an adaptive threshold according to a second embodiment of the present invention;

fig. 4 is a schematic structural diagram of an image retrieval device based on an adaptive threshold according to a third embodiment of the present invention;

fig. 5 is a schematic structural diagram of a computer device according to a fourth embodiment of the present invention.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, techniques, etc., in order to provide a thorough understanding of the embodiments of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It should also be understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.

As used in the present description and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".

Furthermore, the terms "first," "second," "third," and the like in the description of the present specification and in the appended claims, are used for distinguishing between descriptions and not necessarily for indicating or implying a relative importance.

Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the invention. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.

It should be understood that the sequence numbers of the steps in the following embodiments do not mean the order of execution, and the execution order of the processes should be determined by the functions and the internal logic, and should not be construed as limiting the implementation process of the embodiments of the present invention.

In order to illustrate the technical scheme of the invention, the following description is made by specific examples.

The image retrieval method based on the adaptive threshold provided by the embodiment of the invention can be applied to an application environment as shown in fig. 1, wherein a client communicates with a server. The client includes, but is not limited to, a palm top computer, a desktop computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cloud terminal device, a personal digital assistant (personal digital assistant, PDA), and other computer devices. The server may be an independent server, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

The image acquisition equipment can be deployed under various application scenes, such as an intelligent security scene, an intelligent community scene, an intelligent campus scene and the like, the client can be used for searching images to be searched in a preset image database, the images similar to the images to be searched are searched in the image database, the image search can be used for carrying out real-time monitoring on the flows of people in the community to realize the function of identifying abnormal people, at the moment, the image search mainly aims at face images, the face images of people in the community are stored in the client deployed in the community, the image acquisition equipment deployed in the community acquires real-time images of the people in the community and transmits the real-time images to the server, and as the objects of the image search are the face images, the client needs to carry out preprocessing operation on the images to obtain images of the face areas of the real-time images, and if the images of the people in the community are similar to the face images of the people in the community, the people in the community cannot be identified in real time, and the face images of the community cannot be obtained by the people in the community, and the people in the community cannot be identified in real time. It will be appreciated that in the specific embodiments of the present application, related privacy data such as identity information, address information, face image information, etc. of a user are referred to, and when the embodiments of the present application are applied to specific products or technologies, user permission or consent is required to be obtained, and collection, use, and processing of related privacy data are required to comply with related laws and regulations and standards of related countries and regions.

Referring to fig. 2, a flow chart of an image retrieval method based on an adaptive threshold according to an embodiment of the present invention is shown, where the image retrieval method based on an adaptive threshold may be applied to a client in fig. 1, a computer device corresponding to the client is connected to a server to obtain an image to be retrieved, the client has a storage function, an image database is stored in the computer device corresponding to the client, and the image database may be used for performing image retrieval on the image to be retrieved. As shown in fig. 2, the adaptive threshold-based image retrieval method may include the steps of:

step S201, obtaining images to be searched, and searching the first K reference images which are most similar to the images to be searched from a preset image database according to the similarity between the images.

The image to be searched can refer to an image to be searched in an image database, the similarity can be used for representing the difference between two images, the image database can be used for providing search data for image search, K is an integer larger than zero, and the reference image can refer to an image with the similarity between the image database and the image to be searched ranked at the front.

Specifically, in this embodiment, both the image to be searched and the image stored in the image database may be a face image, that is, the image searching task in this embodiment is face image searching, after the default image collecting device collects the real-time image, the real-time image is preprocessed to obtain the image to be searched only including the face region image information, the preprocessing process may use a target detection model, a semantic segmentation model, and the like, in this embodiment, the preprocessing may use the target detection model to perform preprocessing, the detection object is a face of a person, the real-time image is input into the target detection model, output as a detection bounding box of the face of the person, the real-time image is cut according to the detection bounding box, and the size of the cutting result is normalized, so as to obtain the image to be searched.

In an embodiment, the semantic segmentation model may be further adopted to perform preprocessing, the segmentation categories of the pixel points may include a face category of a person and other categories, the real-time image is input into the target detection model and output as a semantic segmentation map, a minimum external rectangle of all the pixel points belonging to the face category of the person in the semantic segmentation map is obtained, the real-time image is cut by the minimum external rectangle, and the cutting result is subjected to size normalization, so that the image to be retrieved may be obtained.

The method for obtaining the similarity between each storage image in the preset image database and the image to be searched respectively can adopt a twin network to calculate, specifically, the input of each calculation of the twin network is any storage image and any image to be searched in the preset image database, the storage image and the image to be searched are respectively subjected to feature extraction through an encoder in the twin network, a first feature vector corresponding to the storage image and a second feature vector corresponding to the image to be searched are obtained, euclidean distance calculation is carried out on the first feature vector and the second feature vector, the similarity between the storage image and the image to be searched is represented by the Euclidean distance obtained through calculation, the similarity between the storage image and the image to be searched is represented to be smaller when the Euclidean distance is smaller, and the similarity between the storage image and the image to be searched is represented to be larger.

The twin network adopted in the similarity calculation is a trained twin network, namely model parameters of the twin network are fixed, at the moment, a first feature vector output after each stored image is input into the trained twin network is also fixed, and the output quantity is only related to the input quantity, so that the first feature vector corresponding to each stored image can be stored in advance, after the image to be searched is obtained, the image to be searched is input into the trained twin network to obtain a second feature vector, the second feature vector is directly used for carrying out Euclidean distance calculation with the first feature vector corresponding to each stored image, and the whole similarity calculation process only needs to carry out feature extraction on the image to be searched through the trained twin network once, so that the similarity calculation of the image to be searched and each stored image in the image database on the feature level can be realized.

It should be noted that, the trained twin network may only include a single trained encoder, that is, only feature extraction needs to be performed, in the training process of the twin network, the training architecture of the twin network may include two encoders and comparators sharing parameters, an implementer may acquire a training set from an actual application scene in this embodiment, train the twin network, so that the trained encoder may more accurately extract feature information in the application scene, for example, in a smart community scene, the training sample includes two images, the two images may be face images of the same person, or face images of different persons, the labels of the training samples may be two types, one type is the same person, the other type is a non-same person, and a loss function during training may use a contrast loss function.

The step of obtaining the images to be searched, namely searching the first K reference images which are most similar to the images to be searched from a preset image database according to the similarity among the images, and performing preliminary searching on the images to be searched from the preset image database, so that stored images which are irrelevant to the images to be searched in the image database are eliminated, and meanwhile, the subsequent optimization processing on the preliminary searching results is facilitated, so that the searching results with higher accuracy are obtained.

Step S202, forming reference curves from the similarities corresponding to the first K reference images according to the sequence from large to small, and performing first-order difference calculation on each similarity in the reference curves to obtain first-order difference curves.

The reference curve may be used to represent a change trend of the similarity corresponding to the first K reference images, the coordinate system in which the reference curve is located may be a first coordinate system, the horizontal axis of the first coordinate system may be a sequential identification axis, the vertical axis of the first coordinate system may be a similarity axis, the first-order difference may refer to a similarity difference value corresponding to adjacent sequential identification in the reference curve, and the first-order difference curve may be used to represent a change trend of the first-order difference value.

Specifically, the similarity corresponding to the first K reference images is arranged in order from large to small, and according to the arrangement result, the sequence identifier of each reference image is determined, where the value range of the sequence identifier may be an integer in [1, K ], for example, the sequence identifier corresponds to the reference image with the largest similarity, the sequence identifier corresponds to 1, and the sequence identifier corresponds to the reference image with the smallest similarity, and the sequence identifier is K. For any reference image, taking the sequential marks as the abscissa and the corresponding similarity as the ordinate, one coordinate point on a preset first coordinate system related to the sequential marks and the similarity can be obtained, traversing each reference image, obtaining K coordinate points on the first coordinate system, and performing curve fitting on the K coordinate points on the first coordinate system to obtain a reference curve.

Optionally, the reference curve contains K similarities;

performing first-order differential calculation on each similarity in the reference curve respectively to obtain a first-order differential curve, wherein the first-order differential curve comprises:

for any one of the similarities in the reference curve, determining associated similarities adjacent to the similarities in the reference curve;

calculating a difference value of the similarity and the associated similarity, and taking the difference value as a first-order difference value corresponding to the similarity in a first-order difference curve;

traversing all the similarities to obtain K first-order differential values corresponding to the similarities, and determining the K first-order differential values to form a first-order differential curve.

The sequence identifier corresponding to the associated similarity may be adjacent to the sequence identifier corresponding to the aimed similarity, and the first-order difference value may be used to characterize a difference between the similarities of the two reference images and the image to be retrieved, respectively.

Specifically, for any one of the similarities in the reference curves, a sequential identifier corresponding to the similarity is obtained, that is, in the first coordinate system, for the ordinate of any coordinate point, the abscissa of the coordinate point is obtained, and since the value range of the sequential identifier is an integer in [1, k ] in the embodiment, the horizontal coordinate difference value between adjacent coordinate points is 1, the sequential identifier corresponding to the similarity is set to be j, the similarity of the sequential identifier of j+1 is determined to be the associated similarity of the similarity, and the first-order difference value corresponding to the similarity of the sequential identifier of j can be expressed as:

Δp _j ＝p _j -p _j+1

Wherein p is _j May refer to similarity, p, of order identified as j _j+1 May refer to the similarity, Δp, of the order identified as j+1 _j Can be a first-order differential value corresponding to similarity with the sequence marked as j, and the value range of j is [1, K-1 ]]。

Constructing a second coordinate system about sequence identifications and first-order differential values, regarding any reference image, taking the sequence identifications as abscissa, taking the corresponding first-order differential values as ordinate, obtaining a coordinate point on the second coordinate system, traversing each reference image to obtain K coordinate points on the second coordinate system, and performing curve fitting on the K coordinate points on the second coordinate system to obtain a first-order differential curve.

It should be noted that, in this embodiment, since the first K reference images that are initially retrieved need to be screened again, K may be set to a larger value to avoid omission caused by different numbers of face images stored in the image database 5 by each person, for example, in this embodiment, K may be set to 250, and in general, the number of face images of the same person in the image database will not exceed the K value, when the similarity sorting is performed, the similarity corresponding to the last bit may default to 0, and when the first-order differential value is calculated, the reference images that are sequentially identified as K may be directly set to 0.

0 in this embodiment, the sequence identifier is used to determine the associated similarity, so as to avoid that when the same similarity exists,

the first-order difference values between the calculated similarities are disordered, so that a reliable first-order difference curve cannot be provided for subsequent standardized processing, and the accuracy of the whole image retrieval process is improved.

The similarity corresponding to the first K reference images respectively forms a reference curve according to the sequence from big to small

5 steps of obtaining a first-order differential curve by performing first-order differential calculation on each similarity in the reference curve, and performing differential calculation on each similarity in the reference curve to highlight differences between the reference images

And the similarity threshold value is convenient to determine subsequently, so that more accurate image retrieval is realized.

And step S203, sliding on the first-order differential curve according to a preset step length by using a preset window to obtain curve segments in the preset window after each sliding stop, and performing standardized calculation on the curve segments to obtain standardized values of the corresponding curve segments.

0, wherein the preset window may be a sliding window under the coordinate system of the first-order differential curve, and the preset step length

The method can be characterized in that the distance of each movement of the preset window can be referred to, the sliding can be referred to as the movement of the preset window according to the preset direction, the curve section can be referred to as a part of a first-order differential curve, and the standardized calculation can be used for converting multiple groups of data into a value without units, so that the data standards are unified, the data comparability is improved, and the weakening is achieved

The data interpretation, normalized value may refer to the result of the normalization process, i.e., the above-described unitless score. 5 in particular, the preset window may be sized by width and height in order to ensure that the preset window can cover

All points on the first order difference curve, the height h of the preset window should satisfy h.gtoreq.max (Δp _j ) The value range of j is [1, K-1 ]]. The bottom frame of the preset window moves on a transverse axis in a coordinate system where the first-order differential curve is located, the moving direction is the positive direction of the transverse axis, and the left frame of the preset window is initially overlapped with the vertical axis in the coordinate system where the first-order differential curve is located.

0 in this embodiment, the width w of the preset window is set to 10, and the step length is preset to 1, so that each time the window slides, a curve segment containing 10 first-order difference values can be extracted from the preset window, and the size of the preset window and the preset step length can be automatically adjusted by an implementer according to actual situations.

Optionally, the curve segments include a window inner curve segment and a window tail curve segment;

sliding on the first-order differential curve according to a preset step length by using a preset window, and obtaining curve segments in the preset window after each sliding stop comprises the following steps:

sliding on the first-order differential curve by using a preset window according to a preset step length, and acquiring a starting position and a terminating position of the preset window after each sliding stop;

Intercepting the first-order differential curve through the starting position and the ending position to obtain a curve section in the window;

intercepting the first-order differential curve through the starting position and the ending position of the first-order differential curve to obtain a curve section at the tail of the window;

accordingly, performing normalized calculation on the curve segment to obtain a normalized value of the corresponding curve segment includes:

and carrying out standardized calculation on the curve section in the window according to the curve section in the window and the curve section at the tail of the window to obtain a standardized value corresponding to the curve section in the window.

The curve segments in the window can be curve segments extracted in a preset window, and the curve segments at the tail of the window can be curve segments extracted in the preset window and the right side of the preset window. The starting position may refer to an abscissa corresponding to a left frame of the preset window when sliding is stopped, the ending position may refer to an abscissa corresponding to a right frame of the preset window when sliding is stopped, and the ending position of the first-order differential curve may refer to an abscissa corresponding to a rightmost point in the first-order differential curve.

Specifically, after the starting position and the ending position of the preset window are obtained, discarding the part of the abscissa in the first-order differential curve, which is smaller than the starting position of the preset window or larger than the ending position of the preset window, wherein the rest is the curve section in the window, and similarly discarding the part of the abscissa in the first-order differential curve, which is smaller than the starting position of the preset window, and the rest is the curve section at the tail of the window.

If the starting position of the preset window is set to be i, the ending position can be represented as i+w, and the ending position of the first-order differential curve can be represented as K, namely, the curve section inside the window is extracted through a section [ i, i+w ], and the curve section at the tail of the window is extracted through a section [ i, K ].

In this embodiment, the window inner curve segment and the window tail curve segment are extracted through the preset window, so that in the process of normalizing the window inner curve segment, the information of the window tail curve segment can be used as a reference, and the relevance between data can be effectively explored, thereby improving the effect of normalizing.

Optionally, performing normalized calculation on the curve section inside the window according to the curve section inside the window and the curve section at the tail of the window, and obtaining the normalized value of the curve section inside the corresponding window includes:

calculating first average values of all first-order differential values in curve segments in the window, and calculating second average values and standard deviations of all first-order differential values in curve segments at the tail of the window;

and carrying out standardized calculation on the curve segments in the window according to the first mean value, the second mean value and the standard deviation to obtain standardized values corresponding to the curve segments in the window.

The first mean value may refer to a mean value of all first-order differential values in the curve section inside the window, the second mean value may refer to a mean value of all first-order differential values in the curve section at the tail of the window, and the standard deviation may be used for characterizing numerical stability of all first-order differential values in the curve section at the tail of the window.

Specifically, a first average value μ ₁ Can be expressed as

Wherein w is the width of the preset window, i.e. the difference between the end position and the start position of the preset window, Δp _n Can be the first-order differential value corresponding to the nth sequence identifier, and the value range of n is [ i, i+w ]]Second mean mu ₂ Can be expressed as +.>

Δp _m Can be the first-order differential value corresponding to the mth sequence identifier, and the value range of m is [ i, K]The standard deviation sigma can be expressed as

The embodiment adopts the mean value and the standard deviation to perform standardized calculation, is convenient and quick to calculate, and can effectively improve the efficiency of curve segment standardized processing, thereby improving the efficiency of integral image retrieval.

Optionally, performing normalized calculation on the curve segment inside the window according to the first mean value, the second mean value and the standard deviation, and obtaining the normalized value of the curve segment inside the corresponding window includes:

calculating a difference between the first average value and the second average value;

and comparing the absolute value of the calculated result with the standard deviation, and taking the obtained ratio as a standardized value of the curve section in the corresponding window.

The difference between the first mean value and the second mean value can be used for representing the difference between the overall similarity change degree of the curve section in the window and the overall similarity change degree of the curve section at the tail of the window, and the ratio of the absolute value of the calculated result to the standard deviation can be used for representing the normalized score value.

Specifically, the calculation formula of the normalized value z can be expressed as:

in the embodiment, the standard processing is performed on the curve segments in the window by the z-score method, so that the standard values which are convenient to compare can be obtained, the target curve segments are more accurately determined by mutual comparison in the determination of the follow-up target values, and the accuracy of image retrieval is further improved.

And the step of performing sliding on the first-order differential curve according to the preset step length by using the preset window to obtain curve segments in the preset window after each sliding stop, performing standardized calculation on the curve segments to obtain standardized values of the corresponding curve segments, and taking the curve segments as standardized processing objects, so that the information of a plurality of reference images can be measured simultaneously when the target curve segments are determined later, the accuracy of determining the target curve segments is improved, and the accuracy of image retrieval is also improved.

And S204, taking the maximum standardized value in all the standardized values as a target value, determining a curve segment used for standardized calculation of the target value as a target curve segment, and determining the center point of the target curve segment.

The normalized value may be used to characterize the intensity of the similarity change, and the target value may be used to identify a curve segment with the greatest similarity change, that is, a target curve segment, where the center point may be determined by the average of the starting position and the ending position of the target curve segment.

Specifically, in this embodiment, the target curve segment is determined according to the standardized value, where the standardized value corresponding to each curve segment may represent the intensity of the similarity change in the curve segment, and the larger the standardized value, the larger the amplitude of the similarity change in the curve segment is, the faster the frequency, and similarly, the smaller the standardized value, the smaller the amplitude of the similarity change in the curve segment is, and the slower the frequency is.

When the similarity in a curve segment changes greatly, the curve segment can be considered to contain turning points of image retrieval, for example, for a face image retrieval task, the turning points can mean that the retrieved image is changed from the image belonging to the same person to the image not belonging to the same person, at the moment, the retrieval result can be screened again according to the turning points, so that only the retrieved image belonging to the same person to be retrieved is reserved, and the image is the target of face image retrieval, but when the previous K images which are most similar to the image to be retrieved are adopted as the retrieval result, as the number of face images stored in an image database of each person is different, for face image retrieval of different persons, the stored face images of the person possibly exceed K values, thereby causing omission in the image retrieval process, and the stored face images are possibly smaller than K values, thereby causing the image retrieval result to contain the face images of other persons, namely false detection, causing the face image retrieval accuracy to be not high, and if a mode of blocking the face threshold value is adopted, the face images are adopted, or the two face images are influenced by the same environment, the similarity threshold value is difficult to be determined, and the importance of the similarity threshold value is particularly difficult to be fixed for the person because the two face images are affected by the same.

In this embodiment, the center point of the target curve segment is taken as the turning point, and the center point can be determined according to the average value of the starting position and the ending position of the target curve segment, and it should be noted that, for the calculation result of the average value, the modulus taking calculation needs to be performed, and the modulus taking calculation result is taken as the center point.

In an embodiment, the determination may be further performed according to the starting position of the target curve segment, the curve segment width and the preset coefficient, that is, the curve segment width and the preset coefficient are multiplied, the multiplication result and the starting position of the target curve segment are added, and then the addition result is taken as the turning point, because in this embodiment, the central point of the target curve segment needs to be obtained, the preset coefficient may be set to 0.5, and for the addition result, the modulo calculation is also required, and the modulo calculation result is taken as the central point.

It should be noted that, the practitioner may adjust the value of the preset coefficient according to the actual situation, so as to adjust the position of the turning point in the target curve segment, for example, the preset coefficient may be set to 1/3, and the value range of the preset coefficient is [0,1].

And determining turning points by taking the largest standardized value in all the standardized values as a target value, determining a curve segment used when the target value is subjected to standardized calculation as a target curve segment and determining the center point of the target curve segment, so as to conveniently screen the preliminary retrieval results again according to the turning points, and improve the accuracy of the image retrieval results.

Step S205, obtaining a reference image corresponding to the center point, taking the similarity between the reference image corresponding to the center point and the image to be searched as a similarity threshold, and screening the first K reference images by using the similarity threshold to obtain an image search result.

The similarity threshold may be used to determine whether the reference image is sufficiently similar to the image to be retrieved, and the image retrieval result may refer to the reference image sufficiently similar to the image to be retrieved.

Specifically, the practitioner may evaluate and verify the method of this embodiment by using a common face image dataset, for example, a CASIA Webface dataset, define an accuracy rate and a recall rate as evaluation criteria, where the accuracy rate may be represented by a ratio of the number of images belonging to the person to be searched in the image search result to the number of all images in the image search result, and the recall rate may be represented by a ratio of the number of images belonging to the person to be searched in the image search result to the number of images belonging to the person to be searched in the image database. When the accuracy rate and the recall rate both meet a preset threshold, the image retrieval method of the embodiment is applied to a specific application scene.

Optionally, filtering the first K reference images by using a similarity threshold, and obtaining an image retrieval result includes:

Comparing the similarity between each reference image in the first K reference images and the image to be searched with a similarity threshold value;

and reserving the reference images corresponding to the similarity greater than the similarity threshold, and arranging all the reserved reference images according to the descending order of the similarity to obtain an image retrieval result.

In the present embodiment, the reference image corresponding to the similarity larger than the similarity threshold may be regarded as an image sufficiently similar to the image to be retrieved, and the reference image corresponding to the similarity smaller than or equal to the similarity threshold may be regarded as an image insufficiently similar to the image to be retrieved.

According to the embodiment, the reference images corresponding to the similarity larger than the similarity threshold are reserved, and the ordering retrieval and the threshold retrieval are effectively fused by the dynamically determined similarity threshold, so that the accuracy of image retrieval is improved.

And the step of obtaining the image retrieval result by screening the first K reference images by taking the similarity between the reference images corresponding to the center points and the images to be retrieved as a similarity threshold and screening the preliminary retrieval result again by the similarity threshold, so that the retrieved images belonging to the same object as the images to be retrieved are maintained, and the accuracy of image retrieval is improved.

According to the embodiment, the reference curve formed by the similarity corresponding to the first K reference images which are initially searched is standardized to represent the distribution characteristic of the similarity, a similarity threshold value is adaptively determined according to the distribution characteristic of the similarity, and the first K reference images are further screened, so that the search image belonging to the same object is obtained, the conditions of missing detection and false detection are reduced, and the accuracy of image search is improved.

Referring to fig. 3, a flow chart of an image searching method based on an adaptive threshold according to a second embodiment of the present invention is shown, where when first K reference images most similar to an image to be searched are searched from a preset image database, similarity between images needs to be calculated, and a mode of calculating similarity between images may be calculated by using a model or may also be calculated directly.

When the model is adopted for calculation, the similarity between the images can be compared from the characteristic level, the interference of factors such as image noise and the like is not easy, the robustness of the similarity calculation is high, and the specific process of carrying out the similarity calculation by adopting the model can be referred to as embodiment one and is not repeated here.

When the similarity between the images is directly calculated, the similarity calculation process comprises the following steps:

step S301, calculating the similarity between each stored image and the image to be retrieved in the image database;

step S302, the top K similarity is determined from all the similarities, and the reference images corresponding to the top K similarity are determined to be the top K reference images most similar to the image to be searched.

The similarity calculation method can adopt measurement methods such as cosine similarity, euclidean distance, manhattan distance and the like, and the similarity can be used for representing difference information between the stored image and the image to be searched.

Specifically, in this embodiment, the cosine similarity is used to calculate the similarity, where the value range of the cosine similarity is [0,1], and the closer the cosine similarity is to 0, the less similar the stored image used for the cosine similarity calculation is to the image to be searched, the closer the cosine similarity is to 1, and the more similar the stored image used for the cosine similarity calculation is to the image to be searched.

The method is characterized in that a model is adopted for similarity calculation, the model for similarity calculation is required to be additionally deployed in a client, the method is not suitable for a scene with limited storage space of the client, and if the method for storing the feature vector in the first embodiment is adopted for calculation acceleration, a certain storage space is required to be additionally occupied, so that the method for similarity calculation by the model is difficult to be well applied to the scene with lower deployment cost, for example, for an intelligent security scene of a residential building, because security work of the residential building is generally responsible for a property and is not a special security company, the deployment cost is lower, calculation equipment with larger storage space cannot be provided, at the moment, the method for directly performing similarity calculation can be used for performing cosine similarity calculation on the images to be searched respectively after the images to be searched are acquired, and therefore the first K reference images which are most similar to the images to be searched in an image database are obtained under the condition that the load of the storage space is not increased.

In the embodiment, a mode of directly performing similarity calculation is provided for a scene with low deployment cost, so that the application difficulty of an image retrieval method is reduced, and the universality of the application scene of the image retrieval method is improved.

Fig. 4 shows a block diagram of an image retrieval device based on an adaptive threshold according to a third embodiment of the present invention, where the image retrieval device based on an adaptive threshold is applied to a client, a computer device corresponding to the client is connected to a server to obtain an image to be retrieved, the client has a storage function, an image database is stored in the computer device corresponding to the client, and the image database can be used for image retrieval of the image to be retrieved. For convenience of explanation, only portions relevant to the embodiments of the present invention are shown.

Referring to fig. 4, the adaptive threshold-based image retrieval apparatus includes:

the preliminary retrieval module 41 is configured to obtain images to be retrieved, retrieve first K reference images most similar to the images to be retrieved from a preset image database according to the similarity between the images, where K is an integer greater than zero;

The difference calculation module 42 is configured to form reference curves from the similarities corresponding to the first K reference images in the order from large to small, and perform first-order difference calculation on each similarity in the reference curves to obtain first-order difference curves;

the curve normalization module 43 is configured to slide on the first-order differential curve according to a preset step size by using a preset window, obtain a curve segment in the preset window after each sliding stop, and perform normalization calculation on the curve segment to obtain a normalized value of the corresponding curve segment;

the center point determining module 44 is configured to determine, using a maximum normalized value of all normalized values as a target value, a curve segment used when the target value performs normalized calculation as a target curve segment, and determine a center point of the target curve segment;

the image screening module 45 is configured to obtain a reference image corresponding to the center point, take a similarity between the reference image corresponding to the center point and the image to be retrieved as a similarity threshold, and screen the first K reference images by using the similarity threshold to obtain an image retrieval result.

Optionally, the preliminary retrieval module 41 includes:

the similarity calculation submodule is used for calculating the similarity between each stored image and the image to be retrieved in the image database;

The image determining sub-module is used for determining the top K similarity degrees with the maximum value from all the similarity degrees, and determining the reference images corresponding to the top K similarity degrees as the top K reference images which are the most similar to the image to be searched.

Optionally, the reference curve contains K similarities;

the differential calculation module 42 includes:

the correlation determination submodule is used for determining correlation similarity adjacent to the similarity in the reference curve aiming at any similarity in the reference curve;

the difference calculation sub-module is used for calculating the difference value between the similarity and the associated similarity, and taking the difference value as a first-order difference value corresponding to the similarity in a first-order difference curve;

the curve composition submodule is used for traversing all the similarities to obtain K first-order differential values corresponding to the similarities, and determining the K first-order differential values to form a first-order differential curve.

Optionally, the curve segments include a window inner curve segment and a window tail curve segment;

the curve normalization module 43 includes:

the position acquisition sub-module is used for sliding on the first-order differential curve by using a preset window according to a preset step length, and acquiring the starting position and the ending position of the preset window after each sliding stop;

the first intercepting submodule is used for intercepting the first-order differential curve through the starting position and the ending position to obtain a curve section inside the window;

The second intercepting submodule is used for intercepting the first-order differential curve through the starting position and the ending position of the first-order differential curve to obtain a curve section at the tail of the window;

accordingly, the curve normalization module 43 includes:

and the standardized calculation sub-module is used for carrying out standardized calculation on the curve section in the window according to the curve section in the window and the curve section at the tail of the window to obtain a standardized value corresponding to the curve section in the window.

Optionally, the standardized computing submodule includes:

the first calculation unit is used for calculating first average values of all first-order differential values in the curve section in the window and calculating second average values and standard deviations of all first-order differential values in the curve section at the tail of the window;

and the second calculation unit is used for carrying out standardized calculation on the 5 sections of the window inner curve according to the first mean value, the second mean value and the standard deviation to obtain standardized values of the corresponding window inner curve sections.

Optionally, the second computing unit includes:

a difference value calculating subunit, configured to calculate a difference value between the first average value and the second average value;

and the ratio calculating subunit is used for comparing the absolute value of the calculated result with the standard deviation to obtain a ratio serving as a standardized value of the curve section in the corresponding window.

0 optionally, the image filtering module 45 includes:

the threshold comparison sub-module is used for comparing the similarity between each reference image in the first K reference images and the image to be searched with a similarity threshold;

and the result determining submodule is used for reserving reference images corresponding to the similarity greater than the similarity threshold value, and arranging all reserved reference images according to the descending order of the similarity to obtain an image retrieval result.

5 it should be noted that the above modules, sub-modules, units, and information interaction between sub-units, and execution

The details of the process and the like are based on the same conception as the method embodiment of the present invention, and specific functions and technical effects thereof can be found in the method embodiment section, and are not described herein.

Fig. 5 is a schematic structural diagram of a computer device according to a fourth embodiment of the present invention. As shown in fig. 5, the computer device of the embodiment 0 includes: at least one processor (only one is shown in FIG. 5), a memory

And a computer program stored in the memory and executable on at least one processor, the processor executing the computer program to perform the steps of any of the various adaptive threshold-based image retrieval method embodiments described above.

The computer device may include, but is not limited to, a processor, a memory. Those skilled in the art will appreciate that fig. 5 is merely an example of a computer device and is not meant to be limiting, and that a computer device may include more or fewer components than shown, or may combine certain components, or different components, such as may also include a network interface, a display screen, an input device, and the like.

The processor may be CPU, other general purpose processor, digital signal

A processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific0Integrated Circuit, ASIC), an off-the-shelf Programmable gate array (FPGA) or other Programmable logic device, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory includes a readable storage medium, an internal memory, etc., where the internal memory may be the memory of the computer device 5, and provides an environment for the execution of an operating system and computer readable instructions in the readable storage medium. The readable storage medium may be a hard disk of a computer device, or in other embodiments may be an external storage device of a computer device, for example, a plug-in hard disk, smart Media Card (SMC), secure Digital (SD) Card, flash memory, etc. provided on a computer device

Card (Flash Card), etc. Further, the memory may also include both internal storage 0 units of the computer device and external storage devices. The memory is used to store an operating system, application programs, boot loader (BootLoader), data, and other programs such as program codes of computer programs, and the like. The memory may also be used to temporarily store data that has been output or is to be output.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above

The division of the functional units and modules is illustrated, and in practical application, the above 5 functional allocations may be performed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules, so as to perform all or part of the functions described above. The functional units and modules in the embodiment can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit, and the integrated units can be hardware

The implementation can also be in the form of software functional units. In addition, the specific names of 0 of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention. Above-mentioned dress

The specific working process of the centering unit and the centering module may refer to the corresponding process in the foregoing method embodiment, and will not be described herein. The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present invention

The invention implements all or part of the flow of the method of the above embodiment, and may be implemented by a computer program to instruct related hardware 5, where the computer program may be stored in a computer readable storage medium, where the computer program, when executed by a processor, may implement the steps of the method embodiment described above. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, executable files or in some intermediate form, etc. The computer readable medium may include at least: can carry a computer program

Any entity or device of code, recording medium, computer Memory, read-Only0Memory (ROM), random access Memory (Random Access Memory, RAM), electrical carrier signals, telecommunications signals, and software distribution media. Such as a U-disk, removable hard disk, magnetic or optical disk, etc. In some jurisdictions, computer readable media may not be electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.

The present invention may also be implemented as a computer program product for implementing all or part of the steps of the method embodiments described above, when the computer program product is run on a computer device, causing the computer device to execute the steps of the method embodiments described above.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

In the embodiments provided by the present invention, it should be understood that the disclosed apparatus/computer device and method may be implemented in other manners. For example, the apparatus/computer device embodiments described above are merely illustrative, e.g., the division of modules or units is merely a logical functional division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

The above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.

Claims (10)

1. An image retrieval method based on an adaptive threshold, the image retrieval method comprising:

acquiring images to be searched, and searching the first K reference images which are most similar to the images to be searched from a preset image database according to the similarity between the images, wherein K is an integer larger than zero;

Forming reference curves by the similarity corresponding to the first K reference images according to the sequence similarity from large to small, and respectively performing first-order difference calculation on each similarity in the similarity reference curves to obtain first-order difference curves;

sliding on the first-order differential curve according to a preset step length by using a preset window to obtain curve segments in the preset window after each sliding stop, and performing standardized calculation on the curve segments to obtain standardized values of corresponding curve segments;

taking the largest standardized value in all standardized values as a target value, determining a curve segment used when the target value performs standardized calculation as a target curve segment, and determining the center point of the target curve segment;

and acquiring the reference images corresponding to the center points, taking the similarity between the reference images corresponding to the center points and the images to be searched as a similarity threshold, and screening the first K reference images by using the similarity threshold to obtain an image search result.

2. The image retrieval method according to claim 1, wherein retrieving the first K reference images most similar to the image to be retrieved from a preset image database according to the similarity between the images includes:

Calculating the similarity between each stored image in the image database and the image to be searched;

and determining the top K similarity of the maximum from all the similarities, and determining the reference images corresponding to the top K similarity as the top K reference images most similar to the image to be searched.

3. The image retrieval method according to claim 1, wherein the similarity curve contains K similarities;

performing first-order differential calculation on each similarity in the reference curve to obtain a first-order differential curve, wherein the first-order differential calculation comprises the following steps of:

determining, for any one of the similarities in the reference curve, an associated similarity adjacent to the similarity in the reference curve;

calculating a difference value of the similarity and the associated similarity, and taking the difference value as a first-order difference value corresponding to the similarity in the first-order difference curve;

traversing all the similarities to obtain K first-order differential values corresponding to the similarities, and determining the K first-order differential values to form the first-order differential curve.

4. The image retrieval method as recited in claim 1, wherein the curve segments include an intra-window curve segment and a tail window curve segment;

Sliding on the first-order differential curve by using a preset window according to a preset step length, and obtaining curve segments in the preset window after each sliding stop comprises the following steps:

sliding on the first-order differential curve by using a preset window according to a preset step length, and acquiring a starting position and a terminating position of the preset window after each sliding stop;

intercepting the first-order differential curve through the starting position and the ending position to obtain a curve section inside the window;

intercepting the first-order differential curve through the starting position and the ending position of the first-order differential curve to obtain a curve section at the tail of the window;

correspondingly, the performing normalized calculation on the curve segment to obtain a normalized value of the corresponding curve segment includes:

and carrying out standardized calculation on the curve section in the window according to the curve section in the window and the curve section at the tail of the window to obtain a standardized value corresponding to the curve section in the window.

5. The method of claim 4, wherein said performing a normalization calculation on the window interior curve segment according to the window interior curve segment and the window tail curve segment to obtain a normalized value corresponding to the window interior curve segment comprises:

Calculating first average values of all first-order differential values in the curve section in the window, and calculating second average values and standard deviations of all first-order differential values in the curve section at the tail of the window;

and carrying out standardized calculation on the curve section in the window according to the first mean value, the second mean value and the standard deviation to obtain a standardized value corresponding to the curve section in the window.

6. The method of claim 5, wherein the performing a normalized calculation on the curve segment inside the window according to the first mean, the second mean, and the standard deviation to obtain a normalized value corresponding to the curve segment inside the window comprises:

calculating a difference between the first mean value and the second mean value;

and comparing the absolute value of the calculated result with the standard deviation to obtain a ratio which is used as a standardized value of the curve section in the corresponding window.

7. The image retrieval method according to any one of claims 1 to 6, wherein the filtering the first K reference images using the similarity threshold value to obtain an image retrieval result includes:

comparing the similarity between each of the first K reference images and the image to be retrieved with the similarity threshold;

And reserving reference images corresponding to the similarity larger than the similarity threshold, and arranging all reserved reference images in a descending order of the similarity to obtain the image retrieval result.

8. An image retrieval device based on an adaptive threshold, the image retrieval device comprising:

the primary retrieval module is used for acquiring images to be retrieved, and retrieving the first K reference images which are most similar to the images to be retrieved from a preset image database according to the similarity between the images, wherein K is an integer larger than zero;

the difference calculation module is used for forming reference curves according to the similarity corresponding to the first K reference images respectively in sequence from large to small, and performing first-order difference calculation on each similarity in the reference curves respectively to obtain first-order difference curves;

the curve standardization module is used for sliding on the first-order differential curve according to a preset step length by using a preset window to obtain curve segments in the preset window after each sliding stop, and carrying out standardization calculation on the curve segments to obtain standardized values of corresponding curve segments;

the center point determining module is used for taking the maximum standardized value in all the standardized values as a target value, determining a curve segment used when the target value is subjected to standardized calculation as a target curve segment, and determining the center point of the target curve segment;

And the image screening module is used for acquiring the reference images corresponding to the center points, taking the similarity between the reference images corresponding to the center points and the images to be searched as a similarity threshold, and screening the first K reference images by using the similarity threshold to obtain an image search result.

9. A computer device, characterized in that it comprises a processor, a memory and a computer program stored in the memory and executable on the processor, which processor implements the image retrieval method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the image retrieval method according to any one of claims 1 to 7.

Images