CN114549475A

CN114549475A - Human body real-time archive retrieval method and device and storage medium

Info

Publication number: CN114549475A
Application number: CN202210171372.5A
Authority: CN
Inventors: 柯辛玥; 陈立力; 周明伟
Original assignee: Zhejiang Dahua Technology Co Ltd
Current assignee: Zhejiang Dahua Technology Co Ltd
Priority date: 2022-02-24
Filing date: 2022-02-24
Publication date: 2022-05-27

Abstract

The invention relates to the technical field of image processing, and discloses a method, a device and a storage medium for searching a human body real-time archive, wherein the method comprises the following steps: based on the first edge relation between the human body characteristic included by each target real-time picture in the target real-time picture subset and the centroid characteristic corresponding to the target file, and a second edge relation between the human body characteristics of any two target real-time pictures in the target real-time picture subset in different airspaces, determining a first retrieval file, and determining a second search profile based on a third edge relationship between the remaining profiles and the pictures in the remaining subset of real-time pictures, the target archive, the residual archive, the target real-time picture subset and the residual real-time picture subset are obtained based on the picture to be inquired, the alternative archive picture and the alternative real-time picture, the first retrieval archive and the second retrieval archive are combined, and the combined archive serves as a human body retrieval result of the picture to be inquired, so that the calculation amount of retrieval is reduced, and the retrieval efficiency is improved.

Description

Human body real-time archive retrieval method and device and storage medium

Technical Field

The disclosure relates to the technical field of image processing, and provides a method and a device for searching a human body real-time archive and a storage medium.

Background

At present, in the field of intelligent security and particularly in the field of image detection, image searching and file searching by images are very important basic functions.

In a common human body real-time searching process, the characteristic data included in the human body picture is not like human face characteristic data, and a relatively accurate clustering effect can be achieved only by characteristic values. The reason is that the face data mainly characterizes the face of a person, and the repeatability among different people is small. When a human body searches for files in real time, the characteristic data included in the human body picture depends on the information such as the physique, the clothing and the like of the human body, and errors are easy to occur. Therefore, at present, human clustering does not only perform simple similarity comparison, but generally performs clustering under certain time and space constraints. That is, when a human body is searched by a picture, a user is required to upload a human body picture and also required to fill some space-time information for auxiliary search, and obviously, the above process introduces a large amount of calculation, and the response speed of search is poor.

Disclosure of Invention

The embodiment of the disclosure provides a human body real-time archive retrieval method, a human body real-time archive retrieval device and a storage medium, which are used for reducing the calculation amount of retrieving an image to be queried and improving the retrieval efficiency.

The specific technical scheme provided by the disclosure is as follows:

in a first aspect, an embodiment of the present disclosure provides a method for retrieving a human body real-time archive, where the method is applied to an intelligent terminal, and includes:

determining a first retrieval file based on a first edge relation between human body features included by each target real-time picture in the target real-time picture subset and centroid features corresponding to the target file, and a second edge relation between the human body features of any two target real-time pictures in the target real-time picture subset at different time in a space domain, wherein the space domain is determined based on a sliding window and a geographical position mark carried by the target real-time pictures;

determining a second search profile based on a third edge relationship between the remaining profiles and the pictures in the remaining subset of real-time pictures; the target archive and the residual archive are obtained by dividing the alternative archives according to a first similarity between the picture to be queried and any one of alternative archive pictures in the alternative archives, the target real-time picture subset and the residual real-time picture subset are obtained by dividing the alternative real-time picture set according to a second similarity, the second similarity is used for representing the similarity between the picture to be queried and any one of the alternative real-time pictures, and the alternative real-time picture set comprises all the alternative real-time pictures which do not reach the archiving condition of the alternative archives;

and merging the first retrieval file and the second retrieval file, and taking the merged file as a human body retrieval result of the picture to be queried.

Optionally, the target profile and the remaining profiles are obtained by:

extracting human body features from the picture to be inquired, and determining the centroid feature corresponding to each alternative archive picture in the alternative archives;

respectively calculating cosine similarity between the human body features and each mass center feature, and taking the cosine similarity between the human body features and each mass center feature as a first similarity;

and determining the alternative archive pictures corresponding to the first similarity greater than the first preset threshold as target archives, and determining the alternative archive pictures corresponding to the first similarity not greater than the first preset threshold as residual archives.

Optionally, the target real-time picture subset and the remaining real-time picture subsets are obtained by:

determining the real-time human body characteristics of any one optional real-time picture in the optional real-time picture set;

respectively calculating cosine similarity between the human body features and each real-time human body feature, and taking the cosine similarity between the human body features and each real-time human body feature as a second similarity;

and determining the candidate real-time pictures corresponding to the second similarity greater than a second preset threshold as a target real-time picture subset, and filing the candidate real-time pictures corresponding to the second similarity not greater than the second preset threshold as the rest real-time picture subset, wherein the second preset threshold is greater than the first preset threshold.

Optionally, the first edge relationship is determined by:

calculating cosine similarity between human body features included in each target real-time picture in the target real-time picture subset and centroid features corresponding to the target files;

judging the target real-time picture and the alternative archive picture corresponding to the cosine similarity larger than a third preset threshold value as the same person, wherein the third preset threshold value is larger than the second preset threshold value;

and establishing a first edge relation between the target real-time picture and the alternative archive picture based on the judgment of the same person.

Optionally, the second edge relationship is determined by:

sequentially dividing the target real-time picture subset into a plurality of time domains according to the window length corresponding to the sliding window, dividing the target real-time picture subset corresponding to any one time domain into a plurality of space domains according to the geographical position identification carried by the target real-time picture, and obtaining the target real-time picture subsets in different space domains based on the time domains and the space domains;

respectively calculating cosine similarity between human body characteristics of any two target real-time pictures in target real-time picture subsets in different time-space domains;

for each space-time domain, judging two target real-time pictures corresponding to cosine similarity larger than a tidal threshold of the space-time domain as the same person, wherein the tidal thresholds corresponding to different space-time domains are different, and each tidal threshold is used for representing that the two target real-time pictures are judged as the lowest probability value of the same person in the corresponding space-time domain;

and establishing a second edge relation based on the two target real-time pictures judged as the same person.

Optionally, the first search profile is determined by:

the first edge relation and the second edge relation are summarized to obtain a first search file.

Optionally, determining a second retrieval profile based on a third edge relationship between the remaining profile and the pictures in the remaining subset of real-time pictures, comprising:

respectively calculating cosine similarity between the human body characteristics corresponding to any one alternative archive picture in the residual archives and the human body characteristics corresponding to any one alternative real-time picture in the residual real-time picture subset;

judging the alternative archive picture and the alternative real-time picture corresponding to the cosine similarity larger than a fourth preset threshold value as the same person, wherein the fourth preset threshold value is larger than the third preset threshold value;

and establishing a third edge relation based on the alternative archive picture and the alternative real-time picture which are judged as the same person, and determining the third edge relation as a second retrieval archive.

In a second aspect, an embodiment of the present disclosure further provides a device for retrieving a human body real-time archive, including:

the first retrieval file generation unit is used for determining a first retrieval file based on a first edge relation between the human body features of each target real-time picture in the target real-time picture subset and the centroid features corresponding to the target file and a second edge relation between the human body features of any two target real-time pictures in the target real-time picture subset in different airspaces, wherein the airspaces are determined based on the sliding window and the geographic position marks carried by the target real-time pictures;

a second search archive generation unit for determining a second search archive based on a third edge relationship between the remaining archive and the pictures in the remaining real-time picture subset; the target archive and the residual archive are obtained by dividing the alternative archives according to a first similarity between the picture to be queried and any one of alternative archive pictures in the alternative archives, the target real-time picture subset and the residual real-time picture subset are obtained by dividing the alternative real-time picture set according to a second similarity, the second similarity is used for representing the similarity between the picture to be queried and any one of the alternative real-time pictures, and the alternative real-time picture set comprises all the alternative real-time pictures which do not reach the archiving condition of the alternative archives;

and the file merging unit is used for merging the first retrieval file and the second retrieval file and taking the merged file as a human body retrieval result of the picture to be inquired.

Optionally, the target profile and the remaining profiles are obtained by:

Optionally, the first edge relationship is determined by:

Optionally, the second edge relationship is determined by:

sequentially dividing the target real-time picture subset into a plurality of time domains according to the window length corresponding to the sliding window, dividing the target real-time picture subset corresponding to any one time domain into a plurality of space domains according to the geographical position identification carried by the target real-time picture, and obtaining the target real-time picture subsets in different airspaces based on the time domains and the space domains;

Optionally, the first search profile generation unit is configured to:

Optionally, a second search profile is determined based on a third edge relationship between the remaining profile and the pictures in the remaining subset of real-time pictures, the second search profile generation unit is configured to:

In a third aspect, a smart terminal includes:

a memory for storing executable instructions;

a processor for reading and executing executable instructions stored in the memory to implement a method as in any one of the first aspect.

In a fourth aspect, a computer-readable storage medium, wherein instructions, when executed by a processor, enable the processor to perform the method of any of the first aspect.

The beneficial effects of this disclosure are as follows:

in summary, in the embodiment of the present disclosure, a method, an apparatus, and a storage medium for retrieving a human body real-time archive are provided, where the method is applied to an intelligent terminal, and includes: determining a first retrieval file based on a first edge relation between a human body feature included by each target real-time picture in the target real-time picture subset and a centroid feature corresponding to the target file, and a second edge relation between the human body features of any two target real-time pictures in the target real-time picture subset in different airspaces, wherein the airspace is determined based on a sliding window and a geographic position mark carried by the target real-time pictures, after the first retrieval file is determined, determining a second retrieval file based on a third edge relation between the remaining files and pictures in the remaining real-time picture subset, wherein the target file and the remaining files are obtained by dividing the alternative files according to a first similarity between the picture to be queried and any one of the alternative file pictures in the alternative files, and the target real-time picture subset and the remaining real-time picture subset are obtained by dividing the alternative real-time picture set according to a second similarity, the second similarity is used for representing the similarity between the picture to be queried and any one of the alternative real-time pictures, the alternative real-time picture set comprises all the alternative real-time pictures which do not reach the filing condition of the alternative archives, the first retrieval archive and the second retrieval archive are merged, the merged archive serves as a human body retrieval result of the picture to be queried, the alternative archive pictures are divided into a target archive and a residual archive, the target real-time picture is divided into a target real-time picture subset and a residual real-time picture subset, and a processing mode of comparing the similarities is carried out respectively, so that the calculation amount of the picture to be queried in the retrieval process is effectively reduced, and the retrieval efficiency is improved.

Additional features and advantages of the disclosure will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the disclosure. The objectives and other advantages of the disclosure may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Drawings

The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this disclosure, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure and not to limit the disclosure. In the drawings:

FIG. 1 is a schematic flow chart illustrating the acquisition of a target archive and a remaining archive in an embodiment of the present disclosure;

fig. 2 is a schematic flow chart illustrating the process of obtaining a target real-time picture subset and remaining real-time picture subsets according to an embodiment of the present disclosure;

FIG. 3 is a schematic diagram illustrating a process of retrieving a human body real-time file according to an embodiment of the present disclosure;

FIG. 4 is a schematic flow chart illustrating establishment of a first edge relationship according to an embodiment of the present disclosure;

FIG. 5 is a schematic flow chart illustrating establishment of a second edge relationship according to an embodiment of the present disclosure;

FIG. 6 is a flowchart illustrating the process of determining a second search file according to an embodiment of the present disclosure;

FIG. 7 is a schematic diagram illustrating a logical structure of a human body real-time file retrieval apparatus according to an embodiment of the present disclosure;

fig. 8 is a schematic entity architecture diagram of an intelligent terminal in an embodiment of the present disclosure.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are some embodiments, but not all embodiments, of the technical solutions of the present disclosure. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments described in the present disclosure without any creative effort belong to the protection scope of the technical solution of the present disclosure.

The terms "first," "second," and the like in the description and claims of the present invention and in the preceding drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein.

Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

In the embodiment of the present disclosure, the implementation of the method for retrieving the human body real-time archive is mainly performed at the intelligent terminal side, that is, the intelligent terminal side: determining a first retrieval file based on the first edge relationship and the second edge relationship, determining a second retrieval file based on the third edge relationship, merging the first retrieval file and the second retrieval file, and using the merged file as a human body retrieval result of the picture to be queried, which is described in detail below.

In the embodiment of the application, the target archive and the residual archive are obtained by dividing the alternative archives according to a first similarity between the picture to be queried and any one of alternative archive pictures in the alternative archives, the target real-time picture subset and the residual real-time picture subset are obtained by dividing the alternative real-time picture set according to a second similarity, the second similarity is used for representing the similarity between the picture to be queried and any one of the alternative real-time pictures, and the alternative real-time picture set comprises all the alternative real-time pictures which do not reach the archiving condition of the alternative archives.

Referring to fig. 1 and fig. 2, the following target files, remaining files, target real-time picture subsets and the manner of acquiring the remaining real-time picture subsets will be described.

(1) The target profile and the remaining profiles are obtained by:

step 101: extracting human body features from the picture to be inquired, and determining the centroid feature corresponding to each alternative archive picture in the alternative archives.

The picture to be inquired can be a picture directly shot by the intelligent terminal or a picture which is input into the intelligent terminal in advance by the user terminal. The above-mentioned alternative file is usually a portrait archive gathering service, in which a front-end camera captures a picture, an intelligent terminal (e.g., a computer, etc.) performs an archive gathering algorithm on the picture to obtain a result in the form of one file for one person, and the result is stored in a database or a memory as a file, which is collectively referred to as a history base file.

In the prior art, when human body real-time archive retrieval is carried out, all pictures belonging to the same person as a picture to be inquired need to be screened from a historical bottom file, but the screening process is complicated, and the corresponding calculation amount is large.

In the embodiment of the present application, in order to reduce the calculation amount of the retrieval, each alternative archive picture in the alternative archives is classified, that is, divided into the target archive and the remaining archives. In addition, the to-be-queried pictures for human body real-time archive retrieval in the embodiment of the application are human body pictures, and the to-be-queried pictures usually carry information such as human faces, human bodies, shooting time and places. Considering that human body pictures are different from human face pictures, retrieval cannot be performed according to obvious five sense organ features, for this reason, in the implementation process, human body features are extracted from the picture to be queried through a human body analytic model, for example, the human body features are converted into 1 x 512 dimensions, and each dimension is an array of float 64.

Further, in order to calculate the first similarity between each candidate archive picture in the candidate archive and the picture to be queried, a centroid feature corresponding to each candidate archive picture in the candidate archive needs to be determined, where the centroid feature is usually a picture feature with the highest archive quality score, for example, a length-width ratio of a height, and the like, and is used as an archive index.

Step 102: and respectively calculating cosine similarity between the human body features and each centroid feature, and taking the cosine similarity between the human body features and each centroid feature as a first similarity.

After the human body features of the picture to be inquired and the mass center features of the alternative archive picture are determined, the cosine similarity between the human body features of the picture to be inquired and each mass center feature is calculated, so that a plurality of cosine similarities are obtained, and in the implementation process, the cosine similarities are all used as the first similarity.

Step 103: and determining the alternative archive pictures corresponding to the first similarity greater than the first preset threshold as target archives, and determining the alternative archive pictures corresponding to the first similarity not greater than the first preset threshold as residual archives.

Considering that the numerical value of the first similarity is uncertain, namely when the numerical value of the first similarity is larger, the picture to be inquired is similar to the alternative archive picture, and the corresponding alternative archive picture can be used as a retrieval object of the human body real-time archive; when the value of the first similarity is smaller, the picture to be inquired is not similar to the alternative archive picture, and the corresponding alternative archive picture cannot be used as a retrieval object of the human body real-time archive.

In the implementation process, in order to screen out the alternative archive pictures with higher similarity, the alternative archive pictures corresponding to the first similarity greater than a first preset threshold are determined as target archives, the alternative archive pictures corresponding to the first similarity not greater than the first preset threshold are determined as residual archives, namely, the alternative archives are divided according to the first preset threshold, and the first preset threshold can be preconfigured according to the actual application scene.

(2) The target real-time picture subset and the remaining real-time picture subsets are obtained by:

because the generation of the human body real-time file is dynamic, the generation of the historical file generally adopts two modes, namely, the clustering is started when the pictures entering the file clustering process reach a certain data volume, and the file clustering is performed at regular time, for example, the file clustering is performed once every 12 hours. No matter which way of gathering files is adopted, some newly acquired pictures are not classified into the historical files, and the pictures are called as alternative real-time pictures, that is, the alternative real-time pictures are pictures which are not determined as alternative files.

Step 201: and determining the real-time human body characteristics of any one optional real-time picture in the optional real-time picture set.

Considering that retrieval objects with high similarity to the picture to be queried also exist in the alternative real-time pictures, in order to screen the pictures, the real-time human body characteristics of each alternative real-time picture can be determined through a human body analysis model aiming at any one of the alternative real-time pictures in the alternative real-time picture set.

Step 202: and respectively calculating cosine similarity between the human body features and each real-time human body feature, and taking the cosine similarity between the human body features and each real-time human body feature as a second similarity.

After the real-time human body features of each alternative real-time picture are determined, the cosine similarity between the human body features of the picture to be inquired and each real-time human body feature is calculated, so that a plurality of cosine similarities are obtained, and in the implementation process, the cosine similarities between the human body features and the real-time human body features are used as second similarities.

Step 203: and determining the candidate real-time pictures corresponding to the second similarity greater than a second preset threshold as a target real-time picture subset, and filing the candidate real-time pictures corresponding to the second similarity not greater than the second preset threshold as the rest real-time picture subset, wherein the second preset threshold is greater than the first preset threshold.

Considering that the numerical value of the second similarity is uncertain, namely when the numerical value of the second similarity is larger, the image to be inquired is similar to the alternative real-time image, and the corresponding alternative real-time image can be used as a retrieval object of the human body real-time archive; when the numerical value of the second similarity is smaller, the picture to be inquired is not similar to the alternative real-time picture, and the corresponding alternative real-time picture cannot be used as a retrieval object of the human body real-time archive.

In the implementation process, in order to screen out the alternative real-time pictures with higher similarity, the alternative real-time pictures corresponding to the second similarity larger than a second preset threshold are determined as the pictures in the target real-time picture subset, the alternative real-time pictures corresponding to the second similarity not larger than the second preset threshold are determined as the pictures in the remaining real-time picture subset, that is, the alternative real-time pictures are divided according to the second preset threshold, and the second preset threshold can be preconfigured according to the actual application scene. Moreover, since the uncertainty of the candidate real-time picture is large, in order to reduce the calculation amount, the second preset threshold is larger than the first preset threshold.

The above processing method of dividing the alternative archive picture into the target archive and the residual archive, dividing the target real-time picture into the target real-time picture subset and the residual real-time picture subset, and performing similarity comparison respectively may be, for example, performed simultaneously through a plurality of threads, which is intended to limit the computational complexity and further improve the response speed of retrieval.

After determining the target archive, the remaining archives, the target real-time picture subset, and the remaining real-time picture subset, referring to fig. 3, in the embodiment of the present disclosure, a specific execution flow of the search of the human body real-time archive is as follows:

step 301: and determining a first retrieval file based on a first edge relation between the human body features of each target real-time picture in the target real-time picture subset and the centroid features corresponding to the target files and a second edge relation between the human body features of any two target real-time pictures in the target real-time picture subset in different airspaces, wherein the airspaces are determined based on the sliding window and the geographic position marks carried by the target real-time pictures.

In order to make the retrieval of the human body real-time archive simpler and faster, a first edge relation and a second edge relation can be simultaneously obtained, wherein the first edge relation is established under the condition that the target real-time pictures in the target real-time picture subset are highly similar to the alternative archive pictures in the target archive, and the second edge relation is established under the condition that any two target real-time pictures in the target real-time picture subset are highly similar. The first search profile is determined to be part of the emphasis calculation in the embodiment of the present application based on the first edge relationship and the second edge relationship.

Specifically, referring to fig. 4, the first edge relationship is determined by:

step 401: and calculating the cosine similarity between the human body characteristics included in each target real-time picture in the target real-time picture subset and the centroid characteristics corresponding to the target archive.

After the human body features corresponding to each target real-time picture and the centroid features corresponding to the alternative archive pictures in the target archive are determined, the similarity between the target archive and the target real-time picture subset is further calculated according to the alternative archive picture in the target archive with higher similarity to the picture to be inquired and the target real-time pictures in the target real-time picture subset with higher similarity to the picture to be inquired in the implementation process, and specifically, the cosine similarity between the human body features included in each target real-time picture in the target real-time picture subset and the centroid features corresponding to the target archive is calculated, so that the target real-time picture and the alternative archive pictures which can be aggregated into the same person can be further screened.

Step 402: and judging the target real-time picture corresponding to the cosine similarity larger than a third preset threshold and the alternative archive picture as the same person, wherein the third preset threshold is larger than the second preset threshold.

In the further screening process, a third preset threshold which is judged to be the same person is set to be larger than the second preset threshold, namely the target real-time picture and the alternative archive picture which can be gathered into the same person can be screened out, and the target real-time picture and the alternative archive picture which correspond to the cosine similarity larger than the third preset threshold are judged to be the same person.

Step 403: and establishing a first edge relation between the target real-time picture and the alternative archive picture based on the judgment of the same person.

Because the similarity between the pictures is represented by the edge relationship in the field of image aggregation, correspondingly, after the target real-time picture and the alternative archive picture of the same person are determined in step 402, the first edge relationship is established according to the target real-time picture and the alternative archive picture which are determined as the same person. Therefore, in the subsequent processing process, the target real-time picture and the alternative archive picture with higher association can be directly determined according to the first edge relation.

Specifically, referring to fig. 5, the second edge relationship is determined by:

step 501: the method comprises the steps of sequentially dividing a target real-time picture subset into a plurality of time domains according to the window length corresponding to a sliding window, dividing the target real-time picture subset corresponding to any one time domain into a plurality of space domains according to a geographical position mark carried by a target real-time picture, and obtaining the target real-time picture subsets in different airspaces based on the time domains and the space domains.

In consideration of ambiguity of human body features, and in order to further reduce the amount of computation, in the embodiment of the present application, the target real-time picture subset is further divided by a sliding window and a geographic location identifier, so that the target real-time picture subsets in different time-space domains can be processed simultaneously by a plurality of parallel threads, and the like.

Specifically, in the time dimension, the window length corresponding to the sliding window may be set according to the actual application scenario, and meanwhile, after sequencing each target real-time picture in the target real-time picture subset according to the time sequence, each sequenced target real-time picture is sequentially divided into a plurality of time domains according to the window length, so that the obtained result is that each time domain corresponds to N target real-time pictures.

In the spatial dimension, in view of that each target real-time picture carries a geographical location identifier, the geographical location identifier needs to be extracted according to each target real-time picture, and on this basis, the target real-time pictures with the same geographical location identifier, that is, belonging to the same geographical range, are divided, specifically, the target real-time picture subset corresponding to each time domain is divided into a plurality of spatial domains according to the geographical location identifier, and it needs to be additionally described that the division of the geographical range can be flexibly set according to the situation.

After the time domain and the space domain are obtained, the target real-time pictures in the same time domain and the same space domain are divided into the same time-space domain, so that the target real-time picture subset is divided into a plurality of small picture sets in different time-space domains.

Step 502: and respectively calculating the cosine similarity between the human body characteristics of any two target real-time pictures in the target real-time picture subsets in different time-space domains.

In the implementation process, after the target real-time picture subset is divided into different time-space domains, threads can be respectively created based on the different time-space domains to perform synchronous processing, so that the retrieval efficiency is improved. Specifically, the cosine similarity between the human body features of any two target real-time pictures in the target real-time picture subset is respectively calculated, so that the cosine similarity between the human body features of the target real-time pictures in each time-space domain is obtained.

Step 503: and aiming at each space-time domain, judging two target real-time pictures corresponding to cosine similarity greater than the tidal threshold of the space-time domain as the same person, wherein the tidal thresholds corresponding to different space-time domains are different, and each tidal threshold is used for representing that the two target real-time pictures are judged as the lowest probability value of the same person in the corresponding space-time domain.

Considering that human body characteristics are easily affected by clothes, environment and the like in the actual monitoring process, for example, two women who wear white gowns in different spaces (pharmacy a and hospital B) can be easily identified as the same person in the document gathering algorithm. In addition, the nature of the spatial domain may also affect the determination of the gathering document, for example, if a person with similar clothing appears in the dish market, the accuracy of the gathering document may be affected if the same determination threshold value as that of a certain landing is adopted. For this reason, the tidal threshold is set for different time-space domains, that is, the similarity threshold of the same person is determined to be different for each time-space domain setting.

In the specific implementation process, a tide threshold corresponding to the current space-time domain is determined, namely the tide threshold represents the lowest probability value of determining two target real-time pictures as the same person in the current space-time domain. Then, it is determined whether the cosine similarity calculated in the step 502 is greater than the corresponding tide threshold, and when it is determined that the cosine similarity is greater than the corresponding tide threshold, the two corresponding target real-time pictures are determined as the same person.

Step 504: and establishing a second edge relation based on the two target real-time pictures judged as the same person. Similarly, in the image clustering field, the similarity between the images is represented by the edge relationship, and after the two target real-time images of the same person are determined in step 503, the second edge relationship is established according to the two target real-time images determined as the same person. Therefore, in the subsequent processing process, the target real-time picture with higher association degree in the target real-time picture subset can be directly determined according to the second edge relation.

After obtaining the first edge relationship and the second edge relationship, the first search profile is determined by: the first edge relation and the second edge relation are summarized to obtain a first search file.

In the implementation process, after the first edge relationship and the second edge relationship are determined, the edge relationships are summarized, including but not limited to, removing repeated parts in the edge relationships, and sorting pictures related to the first edge relationship and the second edge relationship together to obtain a first search file.

In the implementation process, similarity calculation is carried out on the basis of the target archive and the target real-time picture subset to respectively obtain a first edge relation and a second edge relation, and after a first retrieval archive is obtained through summarizing, similarity calculation is carried out on the residual archive and the residual real-time picture subset correspondingly, so that a second retrieval archive is determined.

Step 302: determining a second search profile based on a third edge relationship between the remaining profile and the pictures in the remaining subset of real-time pictures.

Because some pictures of the archives possibly belonging to the same person may be omitted in the actual document gathering process only by means of acquiring the real-time human body archives by means of the target archives and the target real-time picture subsets, the pictures of the archives belonging to the same person need to be determined according to the remaining archives and the remaining real-time picture subsets, and the parts belong to the rough calculation part in the application.

Specifically, determining the second search profile based on the third edge relationship between the remaining profile and the pictures in the remaining real-time picture subset is shown in fig. 6, and includes:

step 601: and respectively calculating the cosine similarity between the human body characteristic corresponding to any one alternative archive picture in the residual archive and the human body characteristic corresponding to any one alternative real-time picture in the residual real-time picture subset.

Similarly, in order to determine whether the remaining archive and the pictures in the remaining real-time picture subset are the archive of the same person, in the implementation process, corresponding human body features need to be extracted from the alternative archive pictures and corresponding human body features need to be extracted from the alternative real-time pictures.

Further, the cosine similarity between the human body features of each alternative archive picture in the residual archive and the human body features corresponding to each alternative real-time picture in the residual real-time picture subset is calculated, so as to judge whether the picture is the same person as the picture to be inquired.

Step 602: and judging the alternative archive picture and the alternative real-time picture corresponding to the cosine similarity larger than a fourth preset threshold as the same person, wherein the fourth preset threshold is larger than the third preset threshold.

In order to improve the retrieval efficiency, a fourth preset threshold needs to be set in advance for the selection of the alternative archive picture and the alternative real-time picture, wherein the fourth preset threshold is larger than the third preset threshold, that is, the requirement for distinguishing the same person from the remaining archive and the remaining real-time picture subset is higher.

In the implementation process, if the cosine similarity between the alternative archive picture and the alternative real-time picture is greater than a fourth preset threshold, the alternative archive picture and the alternative real-time picture are determined as the same person.

Step 603: and establishing a third edge relation based on the alternative archive picture and the alternative real-time picture which are judged as the same person, and determining the third edge relation as a second retrieval archive.

Similarly, after the alternative archive picture and the alternative real-time picture of the same person are determined in step 602, a third edge relationship is established according to the alternative archive picture and the alternative real-time picture determined as the same person, so as to directly determine the alternative archive picture and the alternative real-time picture with higher association according to the third edge relationship, in which case, the third edge relationship may be determined as the second retrieval archive.

Step 303: and merging the first retrieval file and the second retrieval file, and taking the merged file as a human body retrieval result of the picture to be queried.

In the implementation process, after the first retrieval file and the second retrieval file are obtained, the first retrieval file and the second retrieval file are merged, wherein merging can be that pictures corresponding to the archives gathering results of the target archives and the target real-time picture subsets are gathered into pictures corresponding to the archives gathering results of the remaining archives and the remaining real-time picture subsets, or that pictures corresponding to the archives gathering results of the remaining archives and the remaining real-time picture subsets are gathered into pictures corresponding to the archives gathering results of the target archives and the target real-time picture subsets, so that the merged archives serve as human body retrieval results of the pictures to be inquired.

Based on the same inventive concept, referring to fig. 7, an embodiment of the present disclosure provides a human body real-time file retrieval device, including:

a first search archive generating unit 701, configured to determine a first search archive based on a first edge relationship between a human body feature included in each target real-time picture in the target real-time picture subset and a centroid feature corresponding to the target archive, and a second edge relationship between human body features of any two target real-time pictures in the target real-time picture subset in a different time space domain, where the time space domain is determined based on a sliding window and a geographic location identifier carried by the target real-time pictures;

a second search profile generation unit 702, configured to determine a second search profile based on a third edge relationship between the remaining profile and the pictures in the remaining real-time picture subset; the target archive and the residual archive are obtained by dividing the alternative archives according to a first similarity between the picture to be queried and any one of alternative archive pictures in the alternative archives, the target real-time picture subset and the residual real-time picture subset are obtained by dividing the alternative real-time picture set according to a second similarity, the second similarity is used for representing the similarity between the picture to be queried and any one of the alternative real-time pictures, and the alternative real-time picture set comprises all the alternative real-time pictures which do not reach the archiving condition of the alternative archives;

the archive merging unit 703 is configured to merge the first retrieval archive and the second retrieval archive, and use the merged archive as a human body retrieval result of the picture to be queried.

Optionally, the target profile and the remaining profiles are obtained by:

Optionally, the first edge relationship is determined by:

Optionally, the second edge relationship is determined by:

and establishing a second edge relation based on the two target real-time pictures which are judged to be the same person.

Optionally, the first search profile generation unit 701 is configured to:

Optionally, a second search profile is determined based on a third edge relationship between the remaining profile and the pictures in the remaining subset of real-time pictures, and the second search profile generation unit 702 is configured to:

Based on the same inventive concept, referring to fig. 8, an embodiment of the present disclosure provides a server, including: a memory 801 for storing executable instructions; a processor 802 for reading and executing executable instructions stored in the memory, and performing any one of the methods of the first aspect described above.

Based on the same inventive concept, the disclosed embodiments provide a computer-readable storage medium, wherein instructions that, when executed by a processor, enable the processor to perform the method of any of the above first aspects.

In summary, in the embodiment of the present disclosure, a method, an apparatus, and a storage medium for retrieving a human body real-time archive are provided, where the method is applied to an intelligent terminal, and includes: determining a first retrieval file based on a first edge relation between a human body characteristic included by each target real-time picture in the target real-time picture subset and a centroid characteristic corresponding to the target file, and a second edge relation between the human body characteristics of any two target real-time pictures in the target real-time picture subset in different airspaces, wherein the airspace is determined based on a sliding window and a geographic position mark carried by the target real-time pictures, after the first retrieval file is determined, determining a second retrieval file based on a third edge relation between the remaining files and pictures in the remaining real-time picture subset, wherein the target file and the remaining files are obtained by dividing the alternative files according to a first similarity between the picture to be queried and any one of the alternative file pictures in the alternative files, and the target real-time picture subset and the remaining real-time picture subset are obtained by dividing the alternative real-time picture set according to a second similarity, the second similarity is used for representing the similarity between the picture to be queried and any one of the alternative real-time pictures, the alternative real-time picture set comprises all the alternative real-time pictures which do not reach the filing condition of the alternative archives, the first retrieval archive and the second retrieval archive are merged, the merged archive serves as a human body retrieval result of the picture to be queried, the alternative archive pictures are divided into a target archive and a residual archive, the target real-time picture is divided into a target real-time picture subset and a residual real-time picture subset, and a processing mode of comparing the similarities is carried out respectively, so that the calculation amount of the picture to be queried in the retrieval process is effectively reduced, and the retrieval efficiency is improved.

As will be appreciated by one skilled in the art, embodiments of the present disclosure may be provided as a method, system, or computer program product system. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product system embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program product systems according to the present disclosure. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

It will be apparent to those skilled in the art that various changes and modifications can be made in the present disclosure without departing from the spirit and scope of the disclosure. Thus, if such modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include such modifications and variations as well.

Claims

1. A human body real-time archive retrieval method is applied to an intelligent terminal and comprises the following steps:

determining a first retrieval file based on a first edge relation between human body features included by each target real-time picture in a target real-time picture subset and centroid features corresponding to a target file and a second edge relation between the human body features of any two target real-time pictures in the target real-time picture subset at different time airspaces, wherein the airspaces are determined based on a sliding window and geographical position marks carried by the target real-time pictures;

determining a second search profile based on a third edge relationship between the remaining profiles and the pictures in the remaining subset of real-time pictures; the target archive and the residual archive are obtained by dividing alternative archives according to a first similarity between a picture to be queried and any one alternative archive picture in the alternative archives, the target real-time picture subset and the residual real-time picture subset are obtained by dividing the alternative real-time picture set according to a second similarity, the second similarity is used for representing the similarity between the picture to be queried and any one alternative real-time picture, and the alternative real-time picture set comprises all alternative real-time pictures which do not reach the archiving condition of the alternative archives;

and merging the first retrieval file and the second retrieval file, and taking the merged file as a human body retrieval result of the picture to be inquired.

2. The method of claim 1, wherein the target profile and the remaining profiles are obtained by:

respectively calculating cosine similarity between the human body features and each mass center feature, and taking the cosine similarity between the human body features and each mass center feature as the first similarity;

and determining the alternative archive pictures corresponding to the first similarity greater than a first preset threshold as target archives, and determining the alternative archive pictures corresponding to the first similarity not greater than the first preset threshold as residual archives.

3. The method of claim 2, wherein the target real-time picture subset and the remaining real-time picture subsets are obtained by:

respectively calculating cosine similarity between the human body features and each real-time human body feature, and taking the cosine similarity between the human body features and each real-time human body feature as the second similarity;

and determining the candidate real-time pictures corresponding to the second similarity which is greater than a second preset threshold as a target real-time picture subset, and filing the candidate real-time pictures corresponding to the second similarity which is not greater than the second preset threshold as a residual real-time picture subset, wherein the second preset threshold is greater than the first preset threshold.

4. The method of claim 3, wherein the first edge relationship is determined by:

judging the target real-time picture and the alternative archive picture corresponding to the cosine similarity larger than a third preset threshold as the same person, wherein the third preset threshold is larger than the second preset threshold;

and establishing the first edge relation based on the target real-time picture and the alternative archive picture which are judged to be the same person.

5. The method of claim 1, wherein the second edge relationship is determined by:

respectively calculating cosine similarity between human body characteristics of any two target real-time pictures in the target real-time picture subsets in different time-space domains;

for each space-time domain, judging two target real-time pictures corresponding to the cosine similarity greater than the tidal threshold of the space-time domain as the same person, wherein the tidal thresholds corresponding to different space-time domains are different, and each tidal threshold is used for representing the lowest probability value of judging the two target real-time pictures as the same person in the corresponding space-time domain;

and establishing the second edge relation based on the two target real-time pictures which are judged as the same person.

6. The method of claim 1, wherein the first search profile is determined by:

and summarizing the first edge relation and the second edge relation to obtain the first retrieval file.

7. The method of claim 4, wherein determining a second search profile based on a third edge relationship between the remaining profile and the pictures in the remaining subset of real-time pictures comprises:

respectively calculating cosine similarity between the human body feature corresponding to any one alternative archive picture in the residual archive and the human body feature corresponding to any one alternative real-time picture in the residual real-time picture subset;

judging the alternative archive picture and the alternative real-time picture corresponding to the cosine similarity larger than a fourth preset threshold to be the same person, wherein the fourth preset threshold is larger than the third preset threshold;

and establishing a third edge relation based on the alternative archive picture and the alternative real-time picture which are judged as the same person, and determining the third edge relation as the second retrieval archive.

8. A human body real-time archive retrieval device is characterized by comprising:

the first retrieval file generation unit is used for determining a first retrieval file based on a first edge relation between human body features included by each target real-time picture in a target real-time picture subset and centroid features corresponding to the target file and a second edge relation between the human body features of any two target real-time pictures in the target real-time picture subset in different time airspaces, wherein the time airspaces are determined based on a sliding window and geographical position marks carried by the target real-time pictures;

a second search archive generation unit for determining a second search archive based on a third edge relationship between the remaining archive and the pictures in the remaining real-time picture subset; the target archive and the residual archive are obtained by dividing alternative archives according to a first similarity between a picture to be queried and any one alternative archive picture in the alternative archives, the target real-time picture subset and the residual real-time picture subset are obtained by dividing the alternative real-time picture set according to a second similarity, the second similarity is used for representing the similarity between the picture to be queried and any one alternative real-time picture, and the alternative real-time picture set comprises all alternative real-time pictures which do not reach the archiving condition of the alternative archives;

9. An intelligent terminal, comprising:

a memory for storing executable instructions;

a processor for reading and executing executable instructions stored in the memory to implement the method of any one of claims 1-7.

10. A computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor, enable the processor to perform the method of any of claims 1-7.