CN117690172A

CN117690172A - Associated target identification method and device, electronic equipment and storage medium

Info

Publication number: CN117690172A
Application number: CN202311699591.1A
Authority: CN
Inventors: 陈益新
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2020-10-19
Filing date: 2020-10-19
Publication date: 2024-03-12
Also published as: CN112149627A

Abstract

The embodiment of the invention provides a method and a device for identifying an associated target, electronic equipment and a storage medium. The scheme is as follows: acquiring a target time period, at least one target shooting point and a first characteristic value of a target person; acquiring at least one first shooting record from prestored shooting records, wherein the shooting time point included in the first shooting record is within a target time period, the shooting point included in the first shooting record is one of at least one target shooting point, and the first similarity between the first characteristic value and the second characteristic value included in the first shooting record is larger than a first similarity threshold; and determining an associated target of the target person according to the shooting time point and the shooting point included in the at least one first shooting record. By the technical scheme provided by the embodiment of the invention, GPU resources or CPU resources which are required to be consumed by clustering processing are saved, so that GPU resources or CPU resources consumed by associated target identification are reduced.

Description

Associated target identification method and device, electronic equipment and storage medium

The application is a divisional application of China patent application which is submitted by the national intellectual property office, the application number is 202011120049.2 and the invention name is 'a peer identification method, device, electronic equipment and storage medium' on the day of 10 months and 19 in 2020.

Technical Field

The present invention relates to the field of big data analysis technologies, and in particular, to a method and apparatus for identifying an associated target, an electronic device, and a storage medium.

Background

With the continuous development of video acquisition technology, associated target recognition is becoming one of the important links in the video analysis process.

In the related art related object recognition method, a plurality of shooting records are recorded in a shooting library, and each shooting record comprises information such as Identification (ID) of a face image, a feature value of the face image and the like; based on the tag ID of the face image in the photographing library, the associated targets of different persons are identified.

The tag ID of the face image is obtained through clustering, specifically, the feature value of the face image is compared with the feature value of each face image included in a preset face image library one by one, and then the tag ID of the face image is determined.

However, the number of shooting records recorded in the shooting library is very large. This makes the clustering process described above require consuming a large amount of graphics processor (Graphics Processing Unit, GPU) or central processor (Central Processing Unit, CPU) resources, i.e., makes the associated object recognition require consuming a large amount of GPU or CPU resources.

Disclosure of Invention

The embodiment of the invention aims to provide a method, a device, electronic equipment and a storage medium for identifying an associated target, so as to reduce GPU (graphics processing unit) resources or CPU (Central processing Unit) resources consumed by the identification of the associated target. The specific technical scheme is as follows:

the embodiment of the invention also provides a related target identification method, which comprises the following steps:

acquiring a target time period, at least one target shooting point and a first characteristic value of a target person;

acquiring at least one first shooting record from prestored shooting records, wherein the shooting time point included in the first shooting record is within the target time period, the shooting point included in the first shooting record is one of the at least one target shooting point, and the first similarity between the first characteristic value and the second characteristic value included in the first shooting record is larger than a first similarity threshold;

and determining the associated target of the target person according to the shooting time point and the shooting point included in the at least one first shooting record.

Optionally, the step of determining the associated target of the target person according to the shooting time point and the shooting point included in the at least one first shooting record includes:

For each first shooting record, acquiring a second shooting record from candidate shooting records, wherein the candidate shooting records are shooting records except for the first shooting record in the prestored shooting records, the time interval between the shooting time point included in the second shooting record and the shooting time point included in the first shooting record is smaller than or equal to a first preset time threshold, and the shooting point included in the second shooting record is identical to the shooting point included in the first shooting record;

and determining the associated target of the target person according to the second characteristic values included in all the acquired second shooting records.

Optionally, the step of determining the associated target of the target person according to the second feature values included in all the acquired second shooting records includes:

calculating a second similarity between second characteristic values included in every two second shooting records according to the second characteristic values included in all the acquired second shooting records;

dividing the acquired second shooting records into at least one shooting record group according to the second similarity and a second similarity threshold, wherein the second similarity between second characteristic values included in every two second shooting records in each shooting record group is larger than the second similarity threshold;

And determining the personnel represented by the shooting record group with the number of the second shooting records larger than the preset number threshold as the associated targets of the target personnel.

Optionally, the method further comprises:

outputting a group identifier representing a corresponding shooting record group of each associated target, the number of shooting records in the shooting record group and a second shooting record in the shooting record group;

and when a query request containing a group identifier to be queried is received, outputting all second shooting records in a shooting record group corresponding to the group identifier to be queried.

Optionally, when the number of the obtained first shooting records is greater than or equal to 2, a time interval of shooting time points included in each two first shooting records is greater than or equal to a second preset time threshold.

Optionally, the method further comprises:

and displaying the recognition progress of the associated target recognition of the target person in the process of determining the associated target of the target person.

The embodiment of the invention also provides a device for identifying the associated targets, which comprises the following steps:

the first acquisition module is used for acquiring a target time period, at least one target shooting point and a first characteristic value of a target person;

A second obtaining module, configured to obtain at least one first shooting record from pre-stored shooting records, where a shooting time point included in the first shooting record is within the target time period, a shooting point included in the first shooting record is one of the at least one target shooting point, and a first similarity between the first feature value and a second feature value included in the first shooting record is greater than a first similarity threshold;

and the determining module is used for determining the associated target of the target person according to the shooting time point and the shooting point included in the at least one first shooting record.

Optionally, the determining module includes:

a first obtaining sub-module, configured to obtain, for each of the first shooting records, a second shooting record from candidate shooting records, where the candidate shooting records are shooting records other than the first shooting record in the prestored shooting records, a time interval between a shooting time point included in the second shooting record and a shooting time point included in the first shooting record is less than or equal to a first preset time threshold, and a shooting point included in the second shooting record is the same as a shooting point included in the first shooting record;

And the determining submodule is used for determining the associated target of the target person according to the second characteristic values included in all the acquired second shooting records.

Optionally, the determining submodule is specifically configured to calculate a second similarity between second feature values included in each two second shooting records according to second feature values included in all acquired second shooting records;

dividing the acquired second shooting records into at least one shooting record group according to the second similarity and a second similarity threshold, wherein the second similarity between second characteristic values included in every two second shooting records in the shooting record group is larger than the second similarity threshold;

Optionally, the apparatus further includes:

the output module is used for outputting group identification representing the corresponding shooting record group of each associated target, the number of shooting records in the shooting record group and a second shooting record in the shooting record group;

and the feedback module is used for outputting all second shooting records in the shooting record group corresponding to the group identifier to be queried when receiving the query request containing the group identifier to be queried.

Optionally, the apparatus further includes:

and the display module is used for displaying the recognition progress of the associated target recognition of the target personnel in the process of determining the associated target of the target personnel.

The embodiment of the invention also provides electronic equipment, which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface, and the memory are communicated with each other through the communication bus;

a memory for storing a computer program;

and the processor is used for realizing any of the above related target recognition method steps when executing the program stored in the memory.

The embodiment of the invention also provides a computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and the computer program realizes the steps of any of the associated target recognition methods when being executed by a processor.

The embodiment of the invention also provides a computer program which, when run on a computer, causes the computer to execute any one of the associated target recognition methods.

The embodiment of the invention has the beneficial effects that:

according to the associated target identification method, the associated target identification device, the electronic equipment and the storage medium, at least one first shooting record is acquired from the pre-stored shooting records according to the acquired target time period, the at least one target shooting point and the first characteristic value of the target personnel, and then the associated target of the target personnel is determined. In the embodiment of the invention, the associated targets are identified based on the shooting time points, the shooting points and the characteristic values, namely, the associated targets are not required to be identified by depending on the tag IDs of the face images, so that the face images in the shooting library are not required to be clustered to obtain the tag IDs of the face images, GPU resources or CPU resources required to be consumed by the clustering process are saved, and GPU resources or CPU resources consumed by the associated target identification are reduced.

Of course, it is not necessary for any one product or method of practicing the invention to achieve all of the advantages set forth above at the same time.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are necessary for the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention and that other embodiments may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic flow chart of a first method for identifying an associated target according to an embodiment of the present invention;

fig. 2 is a second flowchart of an associated object recognition method according to an embodiment of the present invention;

FIG. 3 is a third flowchart of an associated object recognition method according to an embodiment of the present invention;

fig. 4 is a fourth flowchart of an associated object recognition method according to an embodiment of the present invention;

fig. 5 is a fifth flowchart of an associated object recognition method according to an embodiment of the present invention;

FIG. 6 is a sixth flowchart of a related object recognition method according to an embodiment of the present invention;

fig. 7 is a seventh flowchart of an associated object recognition method according to an embodiment of the present invention;

FIG. 8 is a signaling diagram of an associated target recognition procedure according to an embodiment of the present invention;

fig. 9 is a schematic structural diagram of an associated object recognition device according to an embodiment of the present invention;

fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

For ease of understanding, the words appearing in the embodiments of the invention are explained below.

Shooting and recording: the photographing records photographing information for recording a photographed face image (hereinafter referred to as a face image), and the photographing information may include, but is not limited to, a face image, a tag ID of the face image (which may also be referred to as a face record ID), a feature value of the face image, a time point at which the face image is photographed (i.e., a photographing time point), a photographing point at which the face image is photographed (i.e., a photographing point), face attribute information of the face image, and the like. The face attribute information may include attribute information of a plurality of dimensions of the face image, for example, attribute information of dimensions of facial expression dimensions, facial feature dimensions, and the like. For each shot record, the shot record includes only one face image and does not include a plurality of face images.

For example, the image captured by the camera at a certain moment includes person a and person B. However, when the shot records are generated, the shot records corresponding to the person a and the person B are generated, respectively, and the face image in the shot record corresponding to the person a does not include the face image of the person B nor the face image in the shot record corresponding to the person B.

Shooting library: the shooting library is a database storing shooting records of face images shot by the camera. Each shot record has a unique shot record ID.

Facial image library: the face image library is a database storing face records. Each face record has a unique face record ID. The facial images may include real name libraries and stranger libraries, in particular. Each face record in the real name library may include a face image of a real name person, a feature value of the face image, and person information of the real name person. The personnel information of real name personnel can include name, native place, sex, date of departure, etc. Each face record of the stranger library may include a face image of the stranger, a feature value of the face image, and person information of the stranger. The person information of strangers may include gender, facial attribute information, and the like.

In the related art, when the clustering process is performed in the related art related object recognition process, the feature value of the face image may be compared with the feature value in the face image library, and when the feature value of the face image is the same as the feature value of a certain face image in the face image library, the face record ID of the face image in the face image library is determined as the tag ID of the face image, that is, the tag ID. Therefore, according to the tag ID of the face image and the tag ID of the target person included in each shooting record in the shooting library, the shooting record of the target person is determined, and then the associated target of the target person is identified. The association target may be represented as a person having an association relationship with the target person, and the association target may also be a device carried by the target person, and the like.

However, in an application scenario with a large number of people, such as a safe city, that is, an extra-large and very comprehensive smart city management system, a large number of shooting records will be stored at each moment in the shooting store. At this time, a large amount of GPU resources or CPU resources are consumed in clustering each of the face images, resulting in the need for consuming a large amount of GPU resources or CPU resources for associated object recognition.

In order to solve the problem that related targets are required to consume a large amount of GPU resources or CPU resources in related technology, the embodiment of the invention provides a related target identification method. The method can be applied to any electronic device, such as an electronic device comprising the shooting library.

Fig. 1 is a schematic flow chart of a first method for identifying an associated object according to an embodiment of the present invention. The method comprises the following steps.

Step S101, acquiring a target time period, at least one target shooting point, and a first characteristic value of a target person.

Step S102, at least one first shooting record is obtained from the pre-stored shooting records, where the shooting time point included in the first shooting record is within the target time period, the shooting point included in the first shooting record is one of the at least one target shooting point, and a first similarity between the first feature value and a second feature value included in the first shooting record is greater than a first similarity threshold.

Step S103, determining an associated target of the target person according to the shooting time point and the shooting point included in the at least one first shooting record.

According to the method provided by the embodiment of the invention, at least one first shooting record is obtained from the pre-stored shooting records according to the obtained target time period, at least one target shooting point and the first characteristic value of the target personnel, so that the associated target of the target personnel is determined. In the embodiment of the invention, the associated targets are identified based on the shooting time points, the shooting points and the characteristic values, namely, the associated targets are not required to be identified by depending on the tag IDs of the face images, so that the face images in the shooting library are not required to be clustered to obtain the tag IDs of the face images, GPU resources or CPU resources required to be consumed by the clustering process are saved, and GPU resources or CPU resources consumed by the associated target identification are reduced.

The following describes embodiments of the present invention by way of specific examples. For convenience of description, the electronic device is described below as an execution body, and is not limited in any way.

For the above step S101, a target period, at least one target shooting point, and a first feature value of a target person are acquired. The first feature value of the target person may be a feature value extracted from a face image of the target person, or may be a feature value extracted from a human body image of the target person, or the like. The face image and the human body image are hereinafter collectively referred to as a person image.

In an alternative embodiment, the target person may be any person of the persons photographed by the camera. The electronic device can acquire a target time period, at least one target shooting point and a first characteristic value of the target person aiming at each target person, and generate an identification task of an associated target of the target person.

In another alternative embodiment, the target person may be a user-designated person. For example, the user inputs a person image of a person, i.e., a person image of a target person, on the face application platform, and selects a time period to be queried (i.e., the target time period described above), and a shooting point to be queried (i.e., the at least one target shooting point described above). At this time, the electronic device extracts a first feature value of the target person from the person image of the target person, and generates an identification task of the associated target of the target person based on the first feature value of the target person, the target time period and at least one target shooting point input by the user.

In an alternative embodiment, the identification request may include the target time period, at least one target shooting point, and a feature value of the person image of the target person (i.e., the first feature value of the target person). The electronic device may obtain the first characteristic value, the target time period, and the at least one target shooting point directly from the identification request.

In another optional embodiment, the identification request may carry a target time period, at least one target shooting point, and a person image of the target person, and the electronic device may acquire the target time period, the at least one target shooting point, and the person image of the target person from the identification request, and acquire the first feature value of the target person based on the acquired person image.

The first feature value of the target person may be obtained by extracting features from a person image of the target person using a deep learning network. For example, a convolutional neural network (Convolutional Neural Networks, CNN) may be used to perform feature extraction on a person image of the target person, or a deep neural network (Deep Neural Networks, DNN) may be used to perform feature extraction on a person image of the target person, and a feature vector obtained by the feature extraction may be determined as a first feature value of the target person. The first characteristic value of the target person can also be extracted by other modes. Here, the deep learning network used for extracting the first feature value is not particularly limited.

The target time period may be one or more, for example, the target time period may be one time period of 11:00-12:00, or may be two time periods of 11:00-11:20 and 11:30-11:50. The number of the target shooting points may be one or a plurality of. Here, the above-described target period and the number of target shooting points are not particularly limited.

In an alternative embodiment, each identification task may have a corresponding priority, where the identification request carries a priority identifier, where the priority identifier may be used to indicate the priority of the identification task corresponding to the identification request, and the priority representation corresponding to each device task may be set according to the user class, the user requirement, and the like of the user that triggers the identification task. When the electronic device needs to execute a plurality of recognition tasks, each recognition task can be executed according to the priority of each recognition task, for example, each recognition task is added in a preset task queue, each recognition task in the preset task queue is arranged according to the order of the priority from high to low, and the electronic device can execute the recognition task with the highest priority first.

In an alternative embodiment, each of the above identified tasks may have a corresponding task identifier. The electronic equipment can also send the task identification of each recognition task to the equipment provided with the face application platform, so that a later user can conveniently and timely inquire the recognition progress of the recognition task.

In an alternative embodiment, the user may set the criteria of the associated target of the target person, for example, when triggering the identification task, the user may set parameters such as the number of times of the target person and the associated target are in the same line, the duration of the same line, and the like, in addition to the target time point and the target shooting point. That is, parameters such as the number of the same lines, the duration of the same lines, etc. can be also included in the identification request.

For the step S102, at least one first shooting record is obtained from the pre-stored shooting records, where the shooting time point included in the first shooting record is within the target time period, the shooting point included in the first shooting record is one of the at least one target shooting point, and the first similarity between the first feature value and the second feature value included in the first shooting record is greater than the first similarity threshold. The first characteristic value refers to the characteristic value of the target person, and the second characteristic value refers to the characteristic value included in the pre-stored shooting record. The first and second characteristic values are merely for ease of understanding of distinction and do not have special meanings.

For ease of understanding, the acquisition of the first photographing record is illustrated. The target time period is assumed to be 11:00-12:00. The number of the target shooting points is 1, and is denoted as shooting point 1.

The electronic device may acquire, from the pre-stored shooting records, a shooting record in which the shooting time point is in a period of 11:00-12:00, the shooting point is shooting point 1, and the similarity between the second feature value and the first feature value is greater than a first similarity threshold, where the acquired shooting record is the first shooting record. The manner of acquiring the first photographing record may be described below, and will not be described in detail.

In the embodiment of the present invention, since the first similarity between the second feature value included in the first photographing record and the first feature value is greater than the first similarity threshold, the photographed person image included in the first photographing record is very similar to the person image of the target person. At this time, the electronic device may determine the person image included in the first photographing record and the person image of the target person as the person image of the same person. That is, the above-described first shooting record is a shooting record in which the target person is shot at the target shooting point and the shooting time point within the target period.

In the embodiment of the present invention, the first similarity may be calculated in various manners. Taking the first feature value and the second feature value as feature vectors as examples, the electronic device may calculate the euclidean distance or the manhattan distance between the first feature value and the second feature value, and determine the inverse or negative number of the distance as the first similarity between the first feature value and the second feature value. Here, the manner of calculating the first similarity is not particularly limited. The first similarity threshold may be set according to a calculation method of the similarity, a user demand, and the like, and the first similarity threshold is not particularly limited.

In the embodiment of the present invention, the pre-stored shooting record may be a shooting record stored in the shooting library. The shooting library may be stored in the storage space of the electronic device, or may be stored in the storage space of another device. Here, the storage location of the shooting record stored in advance is not particularly limited.

The second characteristic value included in the pre-stored shooting record is a characteristic value corresponding to each shot personnel image included in the pre-stored shooting record. The extraction method of the second feature value may refer to the extraction method of the first feature value, and will not be described in detail herein.

In an alternative embodiment, in the case where the number of the acquired first shooting records is equal to or greater than 2, the time interval of the shooting time points included in each two first shooting records is equal to or greater than a second preset time threshold.

In the embodiment of the invention, the situation that the stay time of the target personnel at a certain target shooting point is longer may occur in the actual application scene. At this time, among the photographing records stored in advance, photographing records in which a large number of target persons are located at the target photographing point will be included. When the associated target is identified, other people appearing at the target shooting point may be mistakenly identified as the associated target due to the fact that the target person stays at the target shooting point for a long time. Therefore, the electronic device can enable the time interval of the shooting time points included in each two first shooting records in the obtained first shooting records to be larger than or equal to the second preset time threshold, so that the influence of longer stay time of a target person at a certain target shooting point on the recognition of the associated target is reduced, and the accuracy of the recognition of the associated target is improved.

In an alternative embodiment, the time interval between each two first shooting records includes a shooting time point greater than or equal to the second preset time threshold, and may be performed when the number of shooting records of a certain shooting point in the obtained first shooting records is far greater than the number of shooting records of other shooting points, for example, the number of shooting records of the certain shooting point is 3 times or more than the number of shooting records of other shooting points.

With respect to the above step S103, the associated target of the target person is determined according to the photographing time point and the photographing point included in the at least one first photographing record. The determination of the associated targets for the target person may be found in the following description and is not specifically described herein.

In an alternative embodiment, according to the method shown in fig. 1, an associated object identifying method is further provided in an embodiment of the present invention. Fig. 2 is a schematic flow chart of a second method for identifying an associated object according to an embodiment of the present invention. In the flowchart shown in fig. 2, the above step S102 is subdivided into steps S10211 to S10213, which are specifically as follows.

Step S10211, acquiring at least one third photographing record from the photographing records stored in advance, wherein the third photographing record includes a photographing time point within the target period, and the photographing point included in the third photographing record is one photographing point of the at least one target photographing point.

For ease of understanding, the above description will be given of the acquisition of the third shooting record taking the target time period of 11:00 to 12:00 as an example, and the target shooting point as shooting point 1.

The electronic device may select the shooting point as the shooting point 1 according to the shooting time point and the shooting point included in each of the pre-stored shooting records, and obtain all the third shooting records when the shooting time points are all shooting records in the 11:00-12:00 period.

In step S10212, a first similarity between the first feature value and the second feature value included in each of the acquired third photographing records is calculated.

In this step, for each obtained third shooting record, the electronic device may calculate a similarity between the above-mentioned first feature value and a second feature value included in the third shooting record, that is, a first similarity. That is, the similarity between the person image of the target person and the photographed person image included in the third photographing record is calculated.

Step S10213, selecting a third shooting record corresponding to the second characteristic value with the first similarity larger than the first similarity threshold as the first shooting record.

In this step, for each third shooting record, when the first similarity between the first feature value and the third shooting record including the second feature value is greater than the first similarity threshold, the electronic device may determine that the person image of the target person is similar to the person image included in the third shooting record. At this time, the electronic apparatus may determine the third photographing record as one photographing record, i.e., the first photographing record, in which the target person appears at the target photographing point within the target period of time.

By adopting the method shown in the steps S10211-S10213, the electronic equipment can accurately acquire at least one first shooting record of the target person from the pre-stored shooting records, the calculation amount of the first similarity is reduced, the accuracy of acquiring the first shooting record is improved, the acquiring time of the first shooting record is shortened, and the acquiring efficiency of the first shooting record is improved.

In another alternative embodiment, according to the method shown in fig. 1, the embodiment of the present invention further provides a related object identifying method. Fig. 3 is a schematic diagram of a third flow chart of an associated object recognition method according to an embodiment of the present invention. In the flowchart shown in fig. 3, the above step S102 is subdivided into steps S10221 to S10223, which are specifically as follows.

In step S10221, a first similarity between the first feature value and a second feature value included in each of the photographing records stored in advance is calculated.

In this step, for each of the photographing records stored in advance, the electronic device may calculate a similarity between the above-described first feature value and a second feature value included in the photographing record, that is, a first similarity.

In step S10222, a shooting record corresponding to the second feature value with the first similarity greater than the first similarity threshold is selected as the fourth shooting record.

In this step, for each of the photographing records stored in advance, when the first similarity between the first feature value and the second feature value included in the photographing record is greater than the first similarity threshold, the electronic device may determine that the person image of the target person is similar to the person image included in the photographing record. At this time, the electronic apparatus may determine the photographing record as the photographing record of the target person, i.e., the fourth photographing record.

Step S10223, acquiring at least one first shooting record from all the acquired fourth shooting records, where the first shooting record includes a shooting time point within the target time period, and the shooting point included in the first shooting record is one of the at least one target shooting point.

The acquisition of the first photographing record in the above step S10223 may refer to the acquisition of the third photographing record in the above step S10211, which is not described in detail herein.

By adopting the method shown in the steps S10221-S10223, the electronic equipment can accurately acquire at least one first shooting record of the target person from the pre-stored shooting records, so that the accuracy of acquiring the first shooting record is improved.

In an alternative embodiment, according to the method shown in fig. 1, an associated object identifying method is further provided in an embodiment of the present invention. Fig. 4 is a schematic flow chart of a fourth method for identifying an associated object according to an embodiment of the present invention. The method comprises the following steps.

Step S401, acquiring a target time period, at least one target shooting point and a first characteristic value of a target person.

Step S402, at least one first shooting record is obtained from pre-stored shooting records, where the shooting time point included in the first shooting record is within the target time period, the shooting point included in the first shooting record is one of the at least one target shooting point, and a first similarity between the first feature value and a second feature value included in the first shooting record is greater than a first similarity threshold.

The steps S401 to S402 are the same as the steps S101 to S102.

Step S403, for each first shooting record, acquiring a second shooting record from the candidate shooting records, where the candidate shooting records are shooting records other than the first shooting record in the pre-stored shooting records, and a time interval between a shooting time point included in the second shooting record and a shooting time point included in the first shooting record is less than or equal to a first preset time threshold, and a shooting point included in the second shooting record is the same as a shooting point included in the first shooting record.

For the sake of understanding, taking a certain first shooting record, for example, shooting point 1 included in shooting record a, and 11:30 capturing time points included in the first shooting record as an example, the above-mentioned second shooting record will be described.

The first preset time threshold is assumed to be 1 minute. 11:30-1 min=11:29, 11:30+1 min=11:31. The electronic apparatus may acquire, from the candidate shooting records, all shooting records whose shooting point is shooting point 1 and whose shooting time points are in the period 11:29 to 11:31 as all second shooting records corresponding to shooting record a. And by analogy, all the second shooting records corresponding to each first shooting record are acquired.

In the above embodiment, the associated target of the target person is represented as a person who appears at the same shooting point as the target person within the preset time range. That is, a person who appears at the same shooting point as the target person within a preset time range is determined as an associated target of the target person. The minimum value of the preset time range is a time point obtained by subtracting the first preset time threshold from the shooting time point of shooting the target person, and the maximum value of the preset time range is a time point obtained by adding the first preset time threshold to the shooting time point of shooting the target person.

In the embodiment of the invention, considering that the target person and the associated target can be in a motion state, the associated target and the target person may not necessarily be simultaneously present at the same shooting point in an actual scene. Therefore, in order to ensure the integrity of the acquired shooting records of the associated targets, the electronic device can accurately determine at least two persons appearing at the same shooting point in the preset time range as the associated targets by the method, so that the selection range of the associated targets is enlarged, the integrity of the acquired shooting records of the associated targets is improved, and the accuracy of determining the associated targets is improved.

In the embodiment of the present invention, the first preset time threshold may be set by a user according to a specific requirement, for example, a peer duration carried in the identification request may also be a default duration threshold. Here, the first preset time threshold is not particularly limited.

Step S404, determining the associated target of the target person according to the second characteristic values included in all the acquired second shooting records.

The determination of the above-mentioned association targets can be found in the following description, and is not specifically described herein.

The steps S403 to S404 are steps of refining the step S103. Through the steps S403 to S404, the shooting records of the associated targets of the target personnel can be accurately obtained from the candidate shooting records, so that the associated targets of the target personnel are determined, and the accuracy of the associated target identification is improved.

In another optional embodiment, the obtaining of the second shooting record may also be obtained according to a second feature value, a shooting time point and a shooting point included in the at least one first shooting record. The method specifically comprises the following steps:

for each first shooting record, a second shooting record is obtained from the shooting records stored in advance, the time interval between the shooting time point included in the second shooting record and the shooting time point included in the first shooting record is smaller than or equal to a first preset time threshold, the shooting point included in the second shooting record is identical to the shooting point included in the first shooting record, and the second similarity between the second characteristic value included in the second shooting record and the second characteristic value included in the first shooting record is smaller than a third similarity threshold.

The method for calculating the second similarity may refer to the method for calculating the first similarity, which is not described herein.

The third similarity threshold may be the first similarity threshold or may be smaller than the first similarity threshold. Here, the third similarity threshold is not particularly limited.

In an alternative embodiment, based on the method shown in fig. 4, the embodiment of the invention further provides a related object identification method. Fig. 5 is a schematic diagram of a fifth flowchart of an associated object recognition method according to an embodiment of the present invention, as shown in fig. 5. In the method shown in fig. 5, the above step S404 is specifically subdivided into the following steps S4041 to S4043.

Step S4041, calculating a second similarity between the second feature values included in each two second shooting records according to the second feature values included in all the obtained second shooting records.

In this step, the electronic device may calculate, for every two second shooting records among all the acquired second shooting records, a similarity between second feature values included in the two second shooting records, that is, a second similarity.

In step S4042, the acquired second shooting records are divided into at least one shooting record group according to the second similarity and the second similarity threshold, and the second similarity between the second feature values included in each two second shooting records in each shooting record group is greater than the second similarity threshold.

For easy understanding, record a with 3 second shots ₁ -a ₃ An example is described. A is calculated by the step S4041 ₁ And a ₂ Third similarity S between ₁₂ 91%, a ₁ And a ₃ Third similarity S between ₁₃ 95%, a ₂ And a ₃ Third similarity S between ₂₃ 93%.

Assuming that the second similarity threshold is 90%, since a ₁ 、a ₂ And a ₃ The second similarity between the second characteristic values included in each two shooting records is greater than a second similarity threshold value, namely 91 percent>90％、95％>90％、93％>90% the electronic device can convert a ₁ 、a ₂ And a ₃ Divided into the same shooting record group.

In the embodiment of the present invention, each of the pre-stored shooting records is obtained by shooting a shooting area covered by a shooting point with a camera. When a certain person is present in a shooting area of a certain shooting point, the probability that only one shooting record of the person is shot in the shooting records stored in advance is extremely small. Therefore, the probability that a second shot record similar to a certain second shot record does not exist in all the acquired second shot records is extremely small.

In an alternative embodiment, when a second shot record similar to a certain second shot record does not exist in all the acquired second shot records, the electronic device may divide the second shot records into one shot record group alone or discard the second shot records.

In this embodiment of the present invention, the second feature values included in all the second shooting records in each shooting record group are similar, that is, the second similarity between the second feature values included in each two second shooting records is greater than the second similarity threshold. Thus, each group of shooting records is characterized as a person. For example, the second group of shooting records results in 4 shooting record groups, and the electronic device can determine that each shooting record group is characterized as one person, i.e., 4 persons.

In an optional embodiment, after grouping the second shooting records to obtain at least one shooting record group, the electronic device may generate a group identifier of each shooting record group, so as to facilitate distinguishing each shooting record group.

In an alternative embodiment, the electronic device may store the group identifier of each shooting record group, the task identifier of the task identifier, and the shooting record identifier of the shooting record included in each shooting record group, so as to obtain the correspondence shown in table 1.

TABLE 1

Task identification	Group identification	Shooting record mark
			Task 1	Group 1	Shooting record 1
Task 1	Group 1	Shooting record 2
			Task 1	Group 2	Shooting record 3

In table 1, the shooting record 1 and the shooting record 2 belong to the same shooting record group. I.e., group 1, the shot record 3 belongs to another shot record group, i.e., group 2. Shot record 1, shot record 2, and shot record 3 are all the second shot records determined for task 1.

Step S4043, determining the person characterized by the group of shooting records including the second shooting records with the number greater than the preset number threshold as the associated target of the target person.

In this step, for each photographing record group, when the number of second photographing records included in the photographing record group is greater than a preset number threshold, the electronic apparatus may determine a person characterized by the photographing record group as an associated target of the target person.

The preset number threshold may be a value set by the user according to specific needs, for example, the preset number threshold may be 2, 3, 4, etc. Here, the preset number threshold is not particularly limited.

In the embodiment of the invention, the number of the shooting records included in each shooting record group can represent the same-line times of the target personnel and the associated targets thereof, and the electronic equipment can accurately determine the personnel with more same-line times of the target personnel by comparing the number of the second shooting records included in each shooting record group with the preset number threshold value, so that the accuracy of the associated target identification of the target personnel is improved.

In an alternative embodiment, the electronic device may further determine each person characterized by each shooting record group as a target person associated target directly.

In another optional embodiment, the electronic device may further determine a duration corresponding to each shooting record group according to a time point of the second shooting record included in each shooting record group, so as to determine a person represented by the shooting record group with the duration greater than a preset same-line duration threshold as an associated target of the target person.

Through the step S4041 and the step S4043, the electronic device can accurately determine each associated target of the target person, thereby improving the accuracy of identifying the associated targets.

In an alternative embodiment, according to the method shown in fig. 5, an associated object identifying method is further provided in an embodiment of the present invention. Fig. 6 is a schematic diagram of a sixth flowchart of an associated object recognition method according to an embodiment of the present invention, as shown in fig. 6.

Specifically, after the step S4043 is performed, step S405 may be further performed, which is specifically expressed as:

step S405 outputs a group identifier that characterizes each associated target corresponding to the group of shooting records, the number of shooting records in the group of shooting records, and one second shooting record in the group of shooting records.

For example, the electronic device may send a group identifier characterizing a corresponding group of shots of each associated target, the number of shots within the group of shots, and a second shot within the group of shots to the device on which the facial application platform is installed.

In the associated object recognition, data required by the user to determine the associated objects generally includes the number of associated objects, the number of same lines, and the person images of the associated objects. Thus, in an embodiment of the present invention, the electronic device may output a group identifier that characterizes each associated target as corresponding to a group of shooting records, the number of shooting records within the group of shooting records, and a second shooting record within the group of shooting records. The number of group identifications included in the output data may represent the number of associated targets of the target person, and the number of shooting records in each shooting record group may represent the number of times that the target person and each associated target are in the same line, and one second shooting record in each shooting record group includes the shot person image of the associated target.

Only one shooting record in each shooting record group is included in the output data, so that the data quantity of the output data can be effectively reduced, and network transmission resources are saved.

In an alternative embodiment, according to the method shown in fig. 6, an associated object identifying method is further provided in an embodiment of the present invention. Fig. 7 is a schematic diagram of a seventh flowchart of an associated object recognition method according to an embodiment of the present invention, as shown in fig. 7.

After the step S405, step S406 may further include a specific step of:

step S406, when receiving the query request containing the group identifier to be queried, outputting all the second photographing records in the photographing record group corresponding to the group identifier to be queried.

In this step, the user may trigger a query request for the shooting record of the associated target, that is, the user sends a query request including the group identifier to be queried to the electronic device according to the group identifier in the data output by the electronic device. When the electronic equipment receives the query request, acquiring shooting records in the shooting record group corresponding to the group identification to be queried.

Taking the group to be queried as the group 1 in the table 1 as an example, when the electronic device receives the query request containing the group 1, it may determine, according to the table 1, that the shooting records included in the group 1 have the shooting record 1 and the shooting record 2. The electronic device may acquire the photographing record 1 and the photographing record 2 and transmit the acquired photographing record 1 and photographing record 2 to the user.

Through the above step S406, the user is facilitated to query all shooting records of the associated target.

In an alternative embodiment, in the process of determining the associated target of the target person, the electronic device may further display an identification progress of the associated target identification of the target person

In the embodiment of the present invention, a large number of photographing records are included in the photographing records stored in advance, which results in a long time consumed in the associated object recognition process. Thus, to facilitate a user in timely knowing the recognition progress of the associated target recognition process, the electronic device may provide a recognition progress query service.

In an alternative embodiment, the electronic device may determine the progress of recognition of the associated target of the target person, thereby displaying the progress of recognition.

In another alternative embodiment, according to the task identifier, the user may trigger the current recognition progress of the associated target recognition, such as sending a progress query request including the task identifier to be queried to the electronic device. After receiving the progress query request, the electronic device can send the current recognition progress of the recognition task corresponding to the task identification to be queried to the user.

In the embodiment of the present invention, the task query progress may be determined according to the number of the first grabbing records and the number of grabbing records in the pre-stored grabbing records, and a specific determination method is not described herein.

For easy understanding, the associated object recognition method provided by the embodiment of the present invention is described below with reference to fig. 8. Fig. 8 is a signaling diagram of an associated target recognition procedure according to an embodiment of the present invention. The first device is a device provided with the face application platform, the second device is the feature extraction server, and the third device is the electronic device.

In step S801, the first device receives a target period, at least one target shooting point, and a person image of a target person input by a user.

In step S802, the first device transmits the person image of the target person to the second device.

In step S803, the second device performs feature extraction on the person image of the target person, to obtain a first feature value of the target person.

In step S804, the second device sends the first characteristic value of the target person to the first device.

In step S805, the first device transmits an identification request for the target person to the third device.

The identification request may include the target time period, at least one target photographing point, and a first characteristic value of a target person. In addition, the identification request may also include a priority identification or the like.

In step S806, the third device receives the identification request.

In step S807, the third device adds the identification task corresponding to the identification request to the preset task queue according to the priority identifier carried in the identification request.

Each identification task in the preset task queue has a corresponding task identifier.

In step S808, the third device sends the task identifier corresponding to the identification request to the first device.

In step S809, when the priority of the identification task corresponding to the identification request is the highest priority in the preset task queue, the third device obtains at least one first shooting record from the pre-stored shooting records.

In step S810, for each first shooting record, the third device acquires a second shooting record from the candidate shooting records.

The candidate shooting records are shooting records except for the first shooting record in the pre-stored shooting records, the time interval between the shooting time point included in the second shooting record and the shooting time point included in the first shooting record is smaller than or equal to a first preset time threshold, and the shooting point included in the second shooting record is identical to the shooting point included in the first shooting record.

In step S811, the third device calculates a second similarity between the second feature values included in each two second shooting records according to the second feature values included in all the acquired second shooting records.

In step S812, the third device divides the acquired second shooting records into at least one shooting record group according to the second similarity and the second similarity threshold.

Wherein, the second similarity between the second characteristic values included in each two second shooting records in each shooting record group is larger than a second similarity threshold value.

In step S813, the third device determines a person characterized by the group of shooting records including the second shooting records having the number greater than the preset number threshold as an associated target of the target person.

In step S814, the third device sends the identification result of the associated target of the target person to the first device.

The identification result includes a group identifier for representing the corresponding shooting record group of each associated target, the number of shooting records in the shooting record group, and a second shooting record in the shooting record group.

In step S815, the first device sends a shooting record query request including the group identifier to be queried to the third device.

In step S816, the third device receives a shooting record query request.

In step S817, the third device sends all the second shooting records in the shooting record group corresponding to the group identifier to be queried to the first device.

Based on the same inventive concept, the embodiment of the invention also provides a related object recognition device according to the related object recognition method provided by the embodiment of the invention. Fig. 9 is a schematic structural diagram of an associated object recognition device according to an embodiment of the present invention, as shown in fig. 9. The device comprises the following modules.

A first obtaining module 901, configured to obtain a target time period, at least one target shooting point, and a first feature value of a target person;

a second obtaining module 902, configured to obtain at least one first shooting record from pre-stored shooting records, where a shooting time point included in the first shooting record is within a target time period, and the shooting point included in the first shooting record is one of at least one target shooting point, and a first similarity between a first feature value and a second feature value included in the first shooting record is greater than a first similarity threshold;

the determining module 903 is configured to determine an associated target of the target person according to a shooting time point and a shooting point included in the at least one first shooting record.

Optionally, the determining module 903 may include:

a first obtaining sub-module, configured to obtain, for each first shooting record, a second shooting record from candidate shooting records, where the candidate shooting records are shooting records other than the first shooting record in the pre-stored shooting records, and a time interval between a shooting time point included in the second shooting record and a shooting time point included in the first shooting record is less than or equal to a first preset time threshold, and a shooting point included in the second shooting record is the same as a shooting point included in the first shooting record;

Optionally, the determining submodule may be specifically configured to calculate, according to second feature values included in all acquired second shooting records, a second similarity between second feature values included in each two second shooting records;

dividing two second shooting records with second similarity larger than or equal to a second similarity threshold value into the same shooting record group;

Optionally, the above-mentioned associated target recognition device may further include:

According to the device provided by the embodiment of the invention, at least one first shooting record is obtained from the pre-stored shooting records according to the obtained target time period, at least one target shooting point and the first characteristic value of the target personnel, so that the associated target of the target personnel is determined. In the embodiment of the invention, the associated targets are identified based on the shooting time points, the shooting points and the characteristic values, namely, the associated targets are not required to be identified by depending on the tag IDs of the face images, so that the face images in the shooting library are not required to be clustered to obtain the tag IDs of the face images, GPU resources or CPU resources required to be consumed by the clustering process are saved, and GPU resources or CPU resources consumed by the associated target identification are reduced.

Based on the same inventive concept, according to the related object recognition method provided by the above embodiment of the present invention, as shown in fig. 10, the embodiment of the present invention further provides an electronic device, which includes a processor 1001, a communication interface 1002, a memory 1003, and a communication bus 1004, where the processor 1001, the communication interface 1002, and the memory 1003 complete communication with each other through the communication bus 1004;

A memory 1003 for storing a computer program;

the processor 1001 is configured to execute a program stored in the memory 1003, and implement the following steps:

acquiring at least one first shooting record from prestored shooting records, wherein the shooting time point included in the first shooting record is within a target time period, the shooting point included in the first shooting record is one of at least one target shooting point, and the first similarity between the first characteristic value and the second characteristic value included in the first shooting record is larger than a first similarity threshold;

and determining an associated target of the target person according to the shooting time point and the shooting point included in the at least one first shooting record.

According to the electronic equipment provided by the embodiment of the invention, at least one first shooting record is acquired from the pre-stored shooting records according to the acquired target time period, at least one target shooting point and the first characteristic value of the target personnel, so that the associated target of the target personnel is determined. In the embodiment of the invention, the associated targets are identified based on the shooting time points, the shooting points and the characteristic values, namely, the associated targets are not required to be identified by depending on the tag IDs of the face images, so that the face images in the shooting library are not required to be clustered to obtain the tag IDs of the face images, GPU resources or CPU resources required to be consumed by the clustering process are saved, and GPU resources or CPU resources consumed by the associated target identification are reduced.

The communication bus mentioned above for the electronic devices may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.

The communication interface is used for communication between the electronic device and other devices.

The Memory may include random access Memory (Random Access Memory, RAM) or may include Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.

The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but also digital signal processors (Digital Signal Processing, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

Based on the same inventive concept, according to the related object recognition method provided by the above embodiment of the present invention, the embodiment of the present invention further provides a computer readable storage medium, in which a computer program is stored, where the computer program implements the steps of any one of the related object recognition methods when executed by a processor.

Based on the same inventive concept, according to the associated object recognition method provided by the above embodiment of the present invention, the embodiment of the present invention further provides a computer program product containing instructions, which when run on a computer, cause the computer to execute any of the associated object recognition methods of the above embodiment.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, tape), an optical medium (e.g., DVD), a Solid State Disk (SSD), etc.

It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for embodiments of the apparatus, electronic device, computer readable storage medium, and computer program product, which are substantially similar to method embodiments, the description is relatively simple, and reference is made to the section of the method embodiments for relevance.

The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method of associative target recognition, the method comprising:

acquiring at least one first shooting record from prestored shooting records, wherein the shooting time point included in the first shooting record is within the target time period, the shooting point included in the first shooting record is one of the at least one target shooting point, and the first similarity between the first characteristic value and the second characteristic value included in the first shooting record is larger than a first similarity threshold; the first shooting record is a shooting record in which the target person is shot in the target time period at the target shooting point;

and determining an associated target of the target person according to the pre-stored shooting records, the shooting time points and the shooting points included in the at least one first shooting record.

2. The method of claim 1, wherein the step of determining the associated target of the target person based on the photographing time point and photographing point included in the at least one first photographing record comprises:

3. The method according to claim 2, wherein the step of determining the associated target of the target person based on the second feature values included in all the acquired second photographing records includes:

4. A method according to claim 3, characterized in that the method further comprises:

5. The method according to claim 1, wherein in the case where the number of the acquired first photographing records is 2 or more, a time interval of photographing time points included in each two first photographing records is equal to or more than a second preset time threshold.

6. The method according to claim 1, wherein the method further comprises:

7. An associated object recognition apparatus, the apparatus comprising:

a second obtaining module, configured to obtain at least one first shooting record from pre-stored shooting records, where a shooting time point included in the first shooting record is within the target time period, a shooting point included in the first shooting record is one of the at least one target shooting point, and a first similarity between the first feature value and a second feature value included in the first shooting record is greater than a first similarity threshold; the first shooting record is a shooting record in which the target person is shot in the target time period at the target shooting point;

and the determining module is used for determining the associated target of the target person according to the pre-stored shooting records, the shooting time points and the shooting points included in the at least one first shooting record.

8. The apparatus of claim 7, wherein the determining module comprises:

9. The apparatus according to claim 8, wherein the determining submodule is specifically configured to calculate a second similarity between second feature values included in each two second shooting records according to second feature values included in all acquired second shooting records;

10. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;

a memory for storing a computer program;

a processor for carrying out the method steps of any one of claims 1-6 when executing a program stored on a memory.

11. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored therein a computer program which, when executed by a processor, implements the method steps of any of claims 1-6.