KR20220038593A - Clustering method and apparatus, electronic device and storage medium - Google Patents

Abstract

An embodiment of the present invention relates to a clustering method and apparatus, an electronic device, and a storage medium, wherein the clustering method performs quantitative analysis on a first cluster of facial features, so that the first cluster meets a preset clustering condition Determining whether or not - The first cluster is obtained by clustering based on a first threshold, and the preset clustering condition indicates that a plurality of facial features included in the first cluster correspond to the same identity. - ; releasing the plurality of facial features included in the first cluster when the first cluster does not meet the preset clustering condition; and determining a second cluster by clustering the plurality of facial features based on a second threshold, wherein the second threshold is greater than the first threshold. includes An embodiment of the present invention can effectively improve the accuracy of a cluster. In this way, cluster accuracy can be effectively improved by canceling clusters that do not satisfy the preset clustering condition and re-clustering the canceled facial features using a higher threshold value.

Description

Clustering method and apparatus, electronic device and storage medium

Cross-referencing of related applications

The present invention is filed based on a Chinese patent application with an application number of 202010981204.3 and an filing date of September 17, 2020, and claims priority to the Chinese patent application, all contents of the Chinese patent application are incorporated herein by reference. are cited

The present invention relates to the field of computer technology, and more particularly to a clustering method and apparatus, an electronic device and a storage medium.

Face clustering is an important research direction in the field of smart video analysis. A face snapshot in smart video includes spatiotemporal information, and by performing face clustering on a face snapshot including spatiotemporal information to form a file, the trajectory of a person can be well analyzed. Urban-level video sources have disadvantages such as complex environments, poor light conditions, and low resolution, so the accuracy of clustering results cannot be ensured.

The present invention provides a technical solution for a clustering method and apparatus, an electronic device, and a storage medium.

According to a first aspect of the present invention, performing quantitative analysis on a first cluster of facial features to determine whether the first cluster meets a preset clustering condition, wherein the first cluster is a first threshold value is obtained by clustering based on , and the preset clustering condition indicates that a plurality of facial features included in the first cluster all correspond to the same identity; releasing the plurality of facial features included in the first cluster when the first cluster does not meet the preset clustering condition; and determining a second cluster by clustering the plurality of facial features based on a second threshold, wherein the second threshold is greater than the first threshold. It provides a clustering method comprising

In one possible implementation form, the plurality of facial features includes one class-centric facial feature corresponding to the first cluster; The performing quantitative analysis on the first cluster to determine whether the first cluster satisfies a preset clustering condition includes: a cosine distance between the class-centered facial feature and the other facial feature in the first cluster determining; determining a mean distance and a standard deviation distance according to a cosine distance between the class-centric facial feature and the other facial feature in the first cluster; and when the average distance is greater than a third threshold value and the standard deviation distance is less than a fourth threshold value, determining that the first cluster does not meet the preset clustering condition includes

In one possible implementation form, after determining a second cluster by clustering the plurality of facial features based on the second threshold value, the clustering method may include: determining, according to at least one of the first clusters, N known clusters, where N≧1; and classifying facial features to be clustered according to the N known clusters.

In one possible implementation form, after determining N known clusters according to at least one of the second cluster and the first cluster meeting the preset clustering condition, the clustering method includes: clustering corresponding class-centric facial features to obtain a class-centric cluster; and merging the known clusters corresponding to each class-centric facial feature included in the class-centric cluster to obtain a merged cluster corresponding to the same identity.

In one possible implementation form, the classifying the facial feature to be clustered according to the N known clusters includes: determining whether a target known cluster exists in the N known clusters - the clustering facial feature and the target the cosine distance between the class-centric facial features corresponding to the known cluster is less than the fifth threshold; and when the target known clusters exist in the N known clusters, classifying the facial features to be clustered into the target known clusters.

In one possible implementation form, the step of determining whether a target known cluster exists in the N known clusters includes using a k-nearest neighbor algorithm to cluster the clustering in class-centric facial features corresponding to the N known clusters. determining k class-centric facial features with closest Euclidean distances to the facial features to be performed, where N≥k≥1; determining a cosine distance between the facial features to be clustered and the k class-centered facial features, respectively; and determining, as the target known cluster, a known cluster corresponding to a class-centered facial feature having a cosine distance smaller than the fifth threshold.

In one possible implementation form, after classifying facial features to be clustered according to the N known clusters, the clustering method includes, based on a result of classifying the facial features to be clustered, included in the N known clusters. updating the plurality of facial features; and updating, for any one of the known clusters, a class-centric facial feature corresponding to the known cluster according to the plurality of updated facial features corresponding to the known cluster.

In one possible implementation form, after determining whether a target known cluster exists in the N known clusters, the clustering method includes: When the target known cluster does not exist in the N known clusters, the face to be clustered determining the feature as an unclassified facial feature; and clustering a plurality of unclassified facial features at a preset duration by using a k-nearest neighbor algorithm and a graph algorithm to determine a newly added cluster.

In one possible implementation form, after determining a newly added cluster by clustering a plurality of unclassified facial features at a preset duration using the k-nearest neighbor algorithm and the graph algorithm, the clustering method comprises: The method further includes updating the N base clusters according to the newly added cluster.

According to one aspect of the present invention, a quantitative analysis module, configured to perform quantitative analysis on a first cluster of facial features to determine whether the first cluster meets a preset clustering condition, wherein the first cluster is a first cluster. It is obtained by clustering based on a threshold value of 1, and the preset clustering condition indicates that a plurality of facial features included in the first cluster correspond to the same identity; a release module, configured to cancel the plurality of facial features included in the first cluster when the first cluster does not meet the preset clustering condition; and a clustering module, configured to cluster the plurality of facial features to determine a second cluster based on a second threshold, wherein the second threshold is greater than the first threshold; It provides a clustering device comprising a.

In one possible implementation form, the plurality of facial features includes one class-centric facial feature corresponding to the first cluster; The quantitative analysis module includes: a first determining submodule, configured to determine a cosine distance between the class-centric facial feature and the other facial feature in the first cluster; a second determining submodule, configured to determine a mean distance and a standard deviation distance according to a cosine distance between the class-centric facial feature and the other facial feature in the first cluster; and when the average distance is greater than a third threshold value and the standard deviation distance is less than a fourth threshold value, it is determined that the first cluster does not meet the preset clustering condition. and a third determining sub-module.

In one possible implementation form, the clustering device includes: a first determining module, configured to determine N known clusters according to at least one of the second cluster and the first cluster meeting the preset clustering condition, wherein: N≥1 - ; and a classification module, configured to classify facial features to be clustered according to the N known clusters.

In one possible implementation form, the clustering module is further configured to cluster class-centric facial features corresponding to the N known clusters to obtain a class-centric cluster; The clustering device further includes a merging module, configured to merge the known clusters corresponding to each class-centric facial feature included in the class-centric cluster to obtain a merged cluster corresponding to the same identity.

In one possible implementation form, the classification module includes: a fourth determining submodule, configured to determine whether a target known cluster exists in the N known clusters - the facial feature to be clustered and a class center corresponding to the target known cluster cosine distance between facial features is less than the fifth threshold - ; and a classification sub-module, configured to classify the facial feature to be clustered into the target known cluster when the target known cluster exists in the N known clusters.

In one possible implementation form, the fourth determining submodule is, by using a k-nearest neighbor algorithm, the Euclidean distance from the class-centered facial feature corresponding to the N known clusters to the facial feature to be clustered is the greatest. a first determining unit, configured to determine the nearest k class-centric facial features, wherein N≥k≥1; a second determining unit, configured to respectively determine a cosine distance between the facial features to be clustered and the k class-centric facial features; and a third determining unit, configured to determine, as the target known cluster, a known cluster corresponding to a class-centric facial feature whose cosine distance is less than the fifth threshold value.

In one possible implementation form, the clustering device may include: a first update module configured to update a plurality of facial features included in the N known clusters based on a result of classifying the facial features to be clustered; and a second update module, configured to update, for any one of the known clusters, a class-centric facial feature corresponding to the known cluster according to the plurality of updated facial features corresponding to the known cluster.

In one possible implementation form, the clustering device further comprises a second determining module, configured to determine the clustered facial feature as an unclassified facial feature when the target known cluster does not exist in the N known clusters; The clustering module is also configured to cluster a plurality of unclassified facial features at a preset duration, using a k-nearest neighbor algorithm and a graph algorithm, to determine a newly added cluster.

In one possible implementation form, the clustering device further includes a third update module, configured to update the N base clusters according to the newly added cluster.

According to one aspect of the present invention, there is provided an electronic device including a processor and a memory configured to store instructions executable by the processor, the processor calls the instructions stored in the memory, and the clustering method is performed.

According to one aspect of the present invention, there is provided a computer readable storage medium storing computer program instructions, and when the computer program instructions are executed by a processor, the processor causes the clustering method to be implemented.

The present invention provides a computer program including computer readable code, and when the computer readable code is executed in an electronic device, causes a processor of the electronic device to implement the clustering method according to any one of the above.

In an embodiment of the present invention, quantitative analysis is performed on a first cluster of facial features obtained by clustering based on a first threshold to determine whether the first cluster meets a preset clustering condition, and to perform preset clustering The condition indicates that a plurality of facial features included in the first cluster all correspond to the same identity, and when the first cluster does not meet the preset clustering condition, cancel the plurality of facial features included in the first cluster, A second cluster is determined by clustering a plurality of released facial features based on a second threshold value greater than the first threshold value. By canceling the clusters that do not satisfy the preset clustering condition and re-clustering the canceled facial features using a higher threshold, the accuracy of the clusters can be effectively improved.

The general description above and the description of the details set forth below are illustrative and for interpretation purposes only, and not for limiting the present invention. Other features and aspects of the present invention will become apparent when exemplary embodiments are described in detail below with reference to the drawings.

The drawings herein are incorporated in and constitute a part of this specification, and these drawings show embodiments consistent with the present invention, and together with the specification are for interpreting the technical solutions of the present invention.
1A is a schematic diagram of a network architecture of a clustering method according to an embodiment of the present invention.
1B shows a flowchart of a clustering method according to an embodiment of the present invention.
2 shows a schematic diagram of a clustering method according to an embodiment of the present invention.
3 is a block diagram of a clustering apparatus according to an embodiment of the present invention.
Fig. 4 is a schematic structural diagram of a computer device according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Various exemplary embodiments, features and aspects of the present invention will be described in detail below with reference to the drawings. In the drawings, the same reference numbers indicate elements having the same or similar functions. Although the drawings show various aspects of the embodiment, it is not necessary to draw the drawings according to the proportions unless otherwise indicated.

The dedicated word “exemplary” herein means “used as an example, embodiment, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as superior or superior to another embodiment. .

As used herein, the term “and/or” is an association relationship describing an object to be associated, and indicates that three relationships may exist. For example, A and/or B may represent three cases in which only A exists, A and B exist simultaneously, and only B exists. In addition, as used herein, the term “at least one” refers to any one of various or any combination of at least two of various. For example, it may indicate that at least one of A, B, and C includes any one or a plurality of elements selected from the set consisting of A, B, and C.

In addition, in order to better explain the embodiments of the present invention, numerous specific details are provided in the specific embodiments below. It should be understood by those skilled in the art that the embodiments of the present invention may be likewise implemented without some specific details. In some embodiments, the gist of the embodiments of the present invention is obscured by not describing in detail methods, means, elements and circuits familiar to those skilled in the art.

In an embodiment of the present invention, clustering for a plurality of facial features can be implemented through the network architecture shown in Fig. 1A, and Fig. 1A shows a network architecture schematic diagram of a clustering method according to an embodiment of the present invention, the network architecture includes a user terminal 101 , a network 102 , and a clustering terminal 103 of facial features. In order to support one exemplary application, the user terminal 101 and the clustering terminal 103 of facial features establish a communication connection through the network 102, and the user terminal 101 collects a plurality of acquired facial features. In the case of clustering, first, the first cluster of facial features must be transmitted to the clustering terminal 103 of the facial features through the network 102, and then, the clustering terminal 103 of the facial features performs quantitative analysis on the first cluster. to determine whether the first cluster meets the preset clustering condition, and then, if the first cluster does not meet the preset clustering condition, release a plurality of facial features included in the first cluster, and finally , a second cluster is obtained by re-clustering a plurality of facial features using a second threshold, which is a higher threshold. In this way, cluster accuracy can be effectively improved by canceling clusters that do not satisfy the preset clustering condition and re-clustering the canceled facial features using a higher threshold value.

1B shows a flowchart of a clustering method according to an embodiment of the present invention. The clustering method may be performed by an electronic device such as a terminal device or a server, and the terminal device is a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a wireless telephone, and a personal information terminal (Personal). Digital Assistant (PDA), a handheld device, a computing device, an in-vehicle device, a wearable device, and the like, and the clustering method may be implemented by calling a computer readable command stored in a memory through a processor. Alternatively, the clustering method may be performed through a server. 1 , the clustering method may include the following steps.

In step S11, quantitative analysis is performed on the first cluster of facial features to determine whether the first cluster meets a preset clustering condition, and the first cluster is obtained by clustering based on a first threshold value , the preset clustering condition indicates that a plurality of facial features included in the first cluster correspond to the same identity.

In step S12, if the first cluster does not meet the preset clustering condition, a plurality of facial features included in the first cluster are released.

In step S13, a second cluster is determined by clustering a plurality of facial features based on the second threshold, wherein the second threshold is greater than the first threshold.

By canceling the clusters that do not satisfy the preset clustering condition and re-clustering the canceled facial features using a higher threshold, the accuracy of the clusters can be effectively improved.

Since the video source has disadvantages such as complex environment, poor light conditions, and low resolution, when image processing is performed on a facial snapshot from a non-disso source to acquire facial features, The similarity of facial features becomes very high, and after clustering is performed according to the similarity of facial features, facial features of different identities can be clustered in the same cluster, which affects the clustering accuracy. Therefore, it is necessary to secure clustering accuracy by performing quantitative analysis on clusters that already exist in the cluster database.

In one possible implementation form, the plurality of facial features includes one class-centric facial feature corresponding to the first cluster; The performing quantitative analysis on the first cluster to determine whether the first cluster satisfies a preset clustering condition may include: determining a cosine distance between a class-centered facial feature and another facial feature in the first cluster; determining a mean distance and a standard deviation distance according to a cosine distance between a class-centric facial feature and another facial feature in the first cluster; and determining that the first cluster does not meet a preset clustering condition when at least one of the mean distance is greater than the third threshold value and the standard deviation distance is less than the fourth threshold value.

The first cluster is a cluster that already exists in the cluster database, and performing quantitative analysis on the first cluster to determine whether the first cluster satisfies the preset clustering condition includes: a plurality of faces included in the first cluster Determining whether the features correspond to the same identity (eg, the same person).

를 계산하고, 여기서, n은 제1 클러스터 중 클래스 중심 얼굴 특징을 제외한 다른 얼굴 특징의 개수이며, 이러한 코사인 거리의 표준편차 거리

를 계산한다.Determine the cosine distance between the class-centric facial feature in the first cluster and the other facial feature in the first cluster, and record as S _i , the average distance of these cosine distances

, where n is the number of facial features other than class-centered facial features in the first cluster, and the standard deviation distance of this cosine distance

to calculate

When the number of facial features included in the first cluster is large, in order to improve computational efficiency, sampling extraction is performed on other facial features except for the class-centered facial feature in the first cluster, and the class-centered facial feature and sampling are extracted By determining the cosine distance between the calculated facial features, the mean distance and standard deviation distance can be calculated. Sampling conditions may be determined according to actual circumstances, and the present invention is not specifically limited thereto. For example, since facial features are obtained according to facial snapshots collected from a video source, the facial snapshots have spatiotemporal information, which makes the facial features also have spatiotemporal information. Accordingly, according to the spatiotemporal information of the facial feature, sampling and extraction may be performed on other facial features except for the class-oriented facial feature in the first cluster.

After quantitative analysis is performed on the first cluster to obtain the average distance and standard deviation distance, it is determined whether the first cluster satisfies a preset clustering condition according to the average distance and standard deviation distance. The smaller the average distance and the larger the standard deviation distance, the higher the similarity of each facial feature in the first cluster. That is, a plurality of facial features included in the first cluster all correspond to the same identity, and as the first cluster satisfies the preset clustering condition, the clustering accuracy of the first cluster increases. In other words, the larger the mean distance and the smaller the standard deviation distance, the lower the clustering accuracy of the first cluster.

Accordingly, if at least one of the mean distance is greater than the third threshold value and the standard deviation distance is less than the fourth threshold value, it is determined that the first cluster does not meet the preset clustering condition, and the first cluster has low clustering accuracy.

In order to improve the clustering accuracy, a plurality of facial features in the first cluster that do not meet the preset clustering condition are removed. If the first cluster is obtained by clustering with a first threshold (the degree of similarity between each facial feature included in the first cluster is greater than the first threshold), using a second threshold higher than the first threshold , by re-clustering a plurality of facial features to obtain the second cluster (the degree of similarity between each facial feature included in the second cluster is greater than the second threshold), it is possible to effectively improve the clustering accuracy.

2 shows a schematic diagram of a clustering method according to an embodiment of the present invention. The cluster de-cluster may periodically perform the quantitative analysis, de-clustering, and re-clustering operations on the first cluster already existing in the cluster database in a regular task manner. As shown in FIG. 2 , the cluster merging/deleting task 202 is performed by the task scheduler 201 , the first cluster in the cluster database 203 is read, and the quantitative analysis and release of the first cluster are performed. , a re-clustering operation is performed, and then the second cluster obtained by newly clustering is returned to the cluster database 203 . In the process where the task scheduler 201 performs the cluster merging task, class-oriented facial features corresponding to N known clusters in the cluster database 2 are read, and the clustering task 204 is performed to obtain a class-oriented facial cluster. do. In addition, the task scheduler 201 is used to perform the classification task 205 of facial features on the face, read the facial features to be clustered in the spatiotemporal database 206, and assign N number of known clusters in the cluster database 203. The corresponding class-centric facial features are read.

In one possible implementation form, according to at least one of the second cluster and the first cluster meeting a preset clustering condition, determining N known clusters, where N≥1; and classifying facial features to be clustered according to the N known clusters.

After the quantitative analysis is completed on the first cluster in the cluster database, the cluster database contains a first cluster that meets the preset clustering condition (that is, a first cluster that is not released after quantitative analysis) and a newly clustered second cluster. At least one is included, and these clusters are collectively referred to as N known clusters in the cluster database. Face clustering efficiency can be improved by classifying facial features to be clustered using N known clusters in the cluster database.

In one possible implementation form, clustering class-centric facial features corresponding to N known clusters to obtain a class-centric cluster; and merging known clusters corresponding to each class-centric facial feature included in the class-centric cluster.

Since the video source has disadvantages such as complex environment, poor light conditions, and low resolution, when image processing is performed on the face staff photo from the VDISO source to obtain facial features, the The similarity of facial features becomes very low, and after clustering is performed according to the similarity of facial features, facial features of the same identity can be clustered in different clusters, which affects the clustering accuracy. Therefore, clustering accuracy should be improved by merging clusters that already exist in the cluster database and corresponding to the same identity.

Determine the Euclidean distances between class-centric facial features corresponding to N known clusters, and use a k-nearest-neighbor algorithm to obtain k class-centric facial features with the closest Euclidean distances, the k We use the cosine distance by determining the cosine distance between the class-centric facial features. A class-centric cluster is obtained by using the distributed graph processing framework (Spark GraphX) graph algorithm, constructing a point set and a side set, to obtain a connected subset. It can be determined that a plurality of class-centric facial features included in the class-centric cluster correspond to the same identity, so that the known clusters corresponding to each class-centric facial feature included in the class-centric cluster are merged, that is, different clusters of facial features of the same identity. By implementing merging between them, the clustering accuracy can be effectively improved.

Cluster merging is also a regular task method, and the clustering and cluster merging operations of the class-oriented facial features may be periodically performed for N known clusters in the cluster database. Still taking Fig. 2 as an example, as shown in Fig. 2, the cluster merging task is performed by the task scheduler 201, and class-oriented facial features corresponding to N known clusters in the cluster database 203 are read. , perform the clustering task 204 of the class-centric facial feature to obtain a class-centric facial cluster. Further, according to the class-centric face cluster, in the cluster database, known clusters corresponding to each class-centric facial feature included in the class-centric cluster are merged.

In one possible implementation form, the step of classifying the facial features to be clustered according to the N known clusters includes: determining whether a target known cluster exists in the N known clusters - corresponding to the facial features to be clustered and the target known clusters the cosine distance between the class-centric facial features being and when the target known clusters exist in the N known clusters, classifying the facial features to be clustered into the target known clusters.

The resident population at the city level is fixed, which means that after the smart video snapshot system has been running for a certain period of time, the entire cluster in the cluster database is relatively stable, that is, the facial features corresponding to the daily generated facial snapshots are This means that most can be directly classified into N known clusters that already exist in the cluster database. Therefore, by first classifying the facial features to be clustered according to the N known clusters in the cluster database, without the need to directly perform the clustering operation, it is possible to improve the effectiveness of clustering.

Still taking Fig. 2 as an example, the spatiotemporal database includes a newly added face snapshot obtained according to a video source, and facial features to be clustered corresponding to the newly added face snapshot. A classification task 205 is performed by the task scheduler 201, facial features to be clustered in the spatiotemporal database 206 are read, and class-oriented facial features corresponding to N known clusters are read out of the cluster database 203. . If class-oriented facial features corresponding to N known clusters in the cluster database already exist in the display memory of the graphics processing unit (GPU), the task scheduler schedules the GPU to display memory without reading from the cluster database. Class-centric facial features corresponding to N known clusters are directly read from Furthermore, it is determined whether a target known cluster exists in the N known clusters according to class-oriented facial features corresponding to the N known clusters.

In one possible implementation form, the step of determining whether a target known cluster exists in the N known clusters includes, using a k-nearest neighbor algorithm, facial features to cluster in class-centric facial features corresponding to the N known clusters. determining the k class-centric facial features with closest Euclidean distances to, where N≥k≥1; determining a cosine distance between facial features to be clustered and k class-centric facial features, respectively; and determining, as a target known cluster, a known cluster corresponding to a class-centered facial feature whose cosine distance is less than a fifth threshold.

Determine the Euclidean distances between the facial features to be clustered and the class-centric facial features corresponding to the N known clusters, respectively, and use the k-nearest-neighbor algorithm to determine the k class-centric facial features with the closest Euclidean distances. , and acquire k class-centered facial features with the closest Euclidean distance to the facial features to be clustered. In order to make the distance results more accurate, the cosine distances between the facial features to be clustered and the k class-centric facial features are respectively determined to determine whether there exists a class-centric facial feature whose cosine distance from the facial feature to cluster is less than the fifth threshold. It is determined whether or not there is, and a known cluster corresponding to a class-centered facial feature having a cosine distance with a facial feature to be clustered smaller than a fifth threshold is determined as a target known cluster. Here, the fact that the cosine distance between the target known cluster and the facial feature to be clustered is smaller than the fifth threshold value indicates that the similarity between the target known cluster and the facial feature to be clustered is high and clustering can be performed in the same cluster. Accordingly, after the target known cluster is determined, the facial features to be clustered are classified into the target known cluster in the cluster database.

In one possible implementation form, updating a plurality of facial features included in N known clusters based on a result of classifying the facial features to be clustered; and updating, for any one known cluster, the class-centric facial feature corresponding to the known cluster according to the updated plurality of facial features corresponding to the known cluster.

Since new facial features to be clustered are continuously added to the spatiotemporal database, the task scheduler continuously performs the classification task, so that the N known clusters in the cluster database are continuously changed. Therefore, according to the classification task result, a plurality of facial features included in the N known clusters are updated. For example, in the case of a specific known cluster, a new facial feature is added after one classification task.

Considering the continuous update of the known cluster, in order to secure the accuracy of the class-oriented facial feature corresponding to the known cluster, the class-oriented facial feature corresponding to the known cluster is regularly updated according to the updated plurality of facial features corresponding to the known cluster. retrain with

In one possible implementation form, when a target known cluster does not exist in the N known clusters, determining a facial feature to be clustered as an unclassified facial feature; and clustering a plurality of unclassified facial features at a preset duration by using a k-nearest neighbor algorithm and a graph algorithm to determine a newly added cluster.

When the classification task is performed, if the target known cluster does not exist in the N known clusters, the facial feature to be clustered cannot be classified into the N known clusters that already exist in the cluster database, and the facial feature to be clustered is not classified as an unclassified facial feature. indicates that it is determined by After a plurality of unclassified facial features are accumulated within a preset duration, a clustering operation is performed by the task scheduler.

Still taking the above Fig. 2 as an example, a clustering operation is performed by the task scheduler, a plurality of unclassified facial features are clustered at a preset duration to determine a newly added cluster, and a newly added cluster and a newly added cluster Sends the facial features included in the cluster database.

Determine the Euclidean distance between a plurality of unclassified facial features, determine the k unclassified facial features with the closest Euclidean distance using a k-nearest neighbor algorithm, and further determine the cosine distance between the k unclassified facial features Then, by using the cosine distance and using a graph algorithm to construct a set of points and edges, and obtain a connected subset, a newly added cluster is obtained.

In one possible implementation form, the method further comprises updating the N base clusters according to the newly added cluster.

As the clustering task is performed, a newly added cluster is added to the cluster database, and further N base clusters may be updated according to the newly added cluster. For example, before the clustering task is performed, the N base clusters are 6 base clusters, and two newly added clusters are obtained by the clustering task and transmitted to the cluster database. Accordingly, the updated N base clusters are 8 base clusters, and further, a subsequent classification task may be continuously performed using the updated 8 base clusters.

In one possible implementation form, the spatiotemporal database acquires a facial snapshot according to a video source, acquires a corresponding facial feature according to the facial snapshot, and then builds a corresponding feature index for the facial feature, followed by a feature It makes it convenient to search for facial features based on the index. Since the facial features included in N known clusters in the cluster database are all derived from the spatiotemporal database, a cluster index can be built for each known cluster according to the feature index corresponding to each facial feature included in each known cluster, Subsequently, it makes it convenient to inquire the known cluster based on the cluster index, and further acquires each facial feature corresponding to the known cluster. After the N known clusters in the cluster database are updated, the cluster index corresponding to each known cluster may be updated according to the feature index corresponding to each facial feature included in each updated known cluster.

In an actual application scenario, when a user's trajectory is analyzed using a cluster corresponding to a facial feature, it is hoped that the user's trajectory appears in a person's file in a timely manner, that is, there is a certain requirement for clustering staleness. By using the above method of performing the combination of the classification task and the clustering task, it is possible to improve the clustering validity of the facial features to be clustered, so that the trajectory can be studied and determined more comprehensively.

In some case judgments, it is not desired that one person's files have the trajectories of other people mixed, or multiple files do not want to correspond to the same person's trajectories, that is, there are certain requirements for clustering accuracy. By using the above method of performing the cluster de-cluster task and the cluster merging task, the clustering accuracy can be improved, so that the agenda can be determined more accurately.

In an embodiment of the present invention, quantitative analysis is performed on a first cluster of facial features obtained by clustering based on a first threshold to determine whether the first cluster meets a preset clustering condition, and to perform preset clustering The condition indicates that a plurality of facial features included in the first cluster all correspond to the same identity, and when the first cluster does not meet the preset clustering condition, cancel the plurality of facial features included in the first cluster, A second cluster is determined by clustering a plurality of released facial features based on a second threshold value greater than the first threshold value. By canceling the clusters that do not satisfy the preset clustering condition and re-clustering the canceled facial features using a higher threshold, it is possible to effectively improve the clustering accuracy of the clusters.

It will be understood that the respective method embodiments mentioned in the present invention may be combined with each other to form a combined embodiment, provided that the principles and logic are not violated, and the detailed description of the present invention will be omitted due to space limitations. can A person skilled in the art can understand that the specific execution order of each step in the clustering method of the specific embodiment should be determined by its function and possible internal logic.

In addition, an embodiment of the present invention further provides a clustering device, an electronic device, a computer readable storage medium, and a program, all of which may implement any one clustering method provided in the embodiment of the present invention, and corresponding Technical solutions and descriptions refer to corresponding descriptions in the method section, which are not further described.

3 is a block diagram of a clustering apparatus according to an embodiment of the present invention. As shown in Figure 3, the device 30,

Quantitative analysis module 31, configured to perform quantitative analysis on the first cluster of facial features to determine whether the first cluster meets a preset clustering condition, wherein the first cluster is clustered based on a first threshold to be obtained, and the preset clustering condition indicates that a plurality of facial features included in the first cluster correspond to the same identity;

a release module 32, configured to release a plurality of facial features included in the first cluster when the first cluster does not meet the preset clustering condition; and

a clustering module 33, configured to cluster the plurality of facial features to determine a second cluster based on the second threshold, wherein the second threshold is greater than the first threshold; includes

In one possible implementation form, the plurality of facial features includes one class-centric facial feature corresponding to the first cluster;

Quantitative analysis module 31,

a first determining submodule, configured to determine a cosine distance between the class-centric facial feature and another facial feature in the first cluster;

a second determining submodule, configured to determine a mean distance and a standard deviation distance according to a cosine distance between the class-centric facial feature and another facial feature in the first cluster; and

a third determining sub, configured to determine that the first cluster does not meet the preset clustering condition when the average distance is greater than the third threshold value and the standard deviation distance is less than the fourth threshold value contains modules.

In one possible implementation form, device 30 comprises:

a first determining module, configured to determine, according to at least one of the second cluster and the first cluster meeting a preset clustering condition, N known clusters, wherein N≥1; and

and a classification module, configured to classify facial features to be clustered according to the N known clusters.

In one possible implementation form,

The clustering module 33 is also configured to cluster class-centric facial features corresponding to the N known clusters to obtain a class-centric cluster;

Device 30,

The method further includes a merging module, configured to merge known clusters corresponding to each class-centric facial feature included in the class-centric cluster to obtain a merged cluster corresponding to the same identity.

In one possible implementation form, the classification module comprises:

a fourth determining submodule, configured to determine whether a target known cluster exists in the N known clusters, wherein a cosine distance between a facial feature to be clustered and a class-centric facial feature corresponding to the target known cluster is less than a fifth threshold; and

and a classification submodule, configured to classify facial features to be clustered into the target known clusters when the target known clusters exist in the N known clusters.

In one possible implementation form, the fourth determining submodule comprises:

a first determining unit, configured to determine, in class-centric facial features corresponding to the N known clusters, k class-centric facial features that have the closest Euclidean distance from the facial features to cluster by using the k-nearest neighbor algorithm - where N≥k≥1 - ;

a second determining unit, configured to respectively determine a cosine distance between the facial features to be clustered and the k class-centric facial features; and

and a third determining unit, configured to determine, as the target known cluster, the known cluster corresponding to the class-centric facial feature whose cosine distance is less than the fifth threshold.

In one possible implementation form, device 30 comprises:

a first update module configured to update a plurality of facial features included in the N known clusters based on a result of classifying the facial features to be clustered; and

and a second update module, configured to update, for any one known cluster, the class-centric facial feature corresponding to the known cluster according to the updated plurality of facial features corresponding to the known cluster.

In one possible implementation form, device 30 comprises:

a second determining module, configured to determine, when a target known cluster does not exist in the N known clusters, a facial feature to be clustered as an unclassified facial feature;

The clustering module 33 is also configured to cluster a plurality of unclassified facial features at a preset duration, using the k-nearest neighbor algorithm and the graph algorithm, to determine a newly added cluster.

In one possible implementation form, device 30 comprises:

and a third update module, configured to update the N base clusters according to the newly added cluster.

It should be described that, in an embodiment of the present invention, when the clustering method is implemented in the form of a software function module and sold or used as an independent product, it may be stored in one computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention may be implemented in the form of a software product essentially or a part contributing to the prior art, and the computer software product may be a single computer device (terminal, server, etc.) is stored in one storage medium including a plurality of instructions for performing all or part of the method according to each embodiment of the present invention. The above-described storage medium includes various media capable of storing a program code, such as a USB memory, a removable storage device, a read-only memory (ROM), a magnetic disk, or an optical disk. As such, embodiments of the present invention are not limited to any specified combination of hardware and software.

In some embodiments, a function provided or a module included in the apparatus provided in the embodiment of the present invention may perform the method described in the above-described method embodiment, the specific implementation of which is the description of the above-described method embodiment. may be referred to, and a detailed description thereof is omitted here for the sake of brevity.

An embodiment of the present invention further provides a computer-readable storage medium storing computer program instructions, and when the computer program instructions are executed by a processor, the clustering method is implemented. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

An embodiment of the present invention further provides a computer program product including computer readable code, wherein when the computer readable code is executed in a device, the processor of the device implements the clustering method provided in any one of the embodiments described above. run the command to

An embodiment of the present invention further provides another computer program product, configured to store computer readable instructions, when the instructions are executed, cause the computer to execute the operation of the clustering method provided in any one embodiment above. .

Correspondingly, the embodiment of the present invention provides a computer device, FIG. 4 is a schematic structural diagram of the computer device of the embodiment of the present invention, and as shown in FIG. 4 , the electronic device 400 includes one processor 401 , at least one communication bus, a communication interface 402 , at least one external communication interface and a memory 403 . Here, the communication interface 402 is configured to implement connection communication between these components. Here, the communication interface 402 may include a display device, and the external communication interface may include a standard wired interface and a wireless interface. Here, the processor 401 is configured to execute an image processing program in a memory to implement the steps of the clustering method provided in the above embodiment. The electronic device may be provided as a terminal, a server, or another type of device. For example, the electronic device may be a mobile phone, a computer, a digital broadcasting terminal, a message transceiving device, a game console, a tablet device, a medical device, a fitness device, a personal information terminal, and the like.

The description of the embodiments of the clustering device, computer equipment and storage medium is similar to the description of the embodiments of the clustering method, and has similar technical descriptions and beneficial effects to the corresponding method embodiments, and due to space limitations, the description of the embodiments of the clustering method is omitted. It can be referenced, so I won't describe it any further. Technical details not disclosed in the embodiments of the clustering device, computer device and storage medium of the present invention may be understood with reference to the description of the method embodiments of the present invention.

The invention may be a system, method and/or computer program product. The computer program product may include a computer storage medium carrying computer readable program instructions that cause a processor to implement each aspect of the present invention.

A computer-readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. The computer-readable storage medium may be, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include, but are not limited to, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory). , static random access memory (SRAM), compact disk read-only memory (CD-ROM), digital video disk (DVD), memory stick, floppy disk, machine coding device, punch card or grooved structure with instructions stored therein and their any suitable combination. As used herein, a computer-readable storage medium includes a wireless electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (eg, light pulses through a fiber optic cable), or an electrical signal transmitted over an electric wire. It is not interpreted as a temporary signal itself, such as

The computer readable program instructions described herein may be downloaded from a computer readable storage medium to each computing/processing device, or downloaded to an external computer or external storage device via the Internet, local area network, wide area network and/or wireless Internet. can be Networks may include copper transport cables, fiber optic transport, wireless transport, routers, firewalls, switches, gateway computers and/or edge servers. A network adapter or network interface of each computing/processing device receives computer readable program instructions from a network, and transmits the computer readable program instructions to be stored in a computer readable storage medium of each computing/processing device.

The computer program instructions for implementing the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or any combination of one or more programming languages. It can be written source code or object code, and the programming language includes an object-oriented programming language such as Smalltalk, C++, etc., and an existing procedural programming language such as a “C” language or similar programming language. may run entirely on the user's computer, partially on the user's computer, as a standalone software package, in part on the user's computer and partly on the remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network or a wide area network, or may be connected to an external computer (e.g., the Internet using an Internet service provider). In some embodiments, state information in computer readable program instructions is used to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array, or a programmable logic array, the electronic circuit comprising: Each aspect of the embodiments of the present invention may be implemented by executing computer readable program instructions.

Herein, each aspect of an embodiment of the present invention will be described with reference to a flowchart and/or a block diagram of a method, an apparatus (system) and a computer program product of the embodiment of the present invention. It should be understood that each block in the flowcharts and/or block diagrams and combinations of respective blocks in the flowcharts and/or block diagrams may all be implemented by computer readable program instructions.

These computer readable program instructions are provided to the processor of a general purpose computer, special purpose computer, or other programmable data processing device to create a machine, such that when these instructions are executed by the processor of the computer or other programmable data processing device, the flow chart and/or devices that implement the functions/acts specified in one or more blocks of the block diagrams. Further, such computer readable program instructions may be stored in a computer readable storage medium, which instructions cause a computer, programmable data processing apparatus, and/or other device to work in a specified manner, thereby causing the computer readable medium having the instructions stored thereon. includes an article of manufacture comprising instructions for implementing each aspect of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.

Also, a process implemented by a computer by loading computer readable program instructions into a computer, other programmable data processing device, or other device causes a sequence of steps to be performed on the computer, other programmable data processing device, or other device. By causing them to be generated, instructions executing on a computer, other programmable data processing device, or other device may implement functions/actions specified in one or more blocks of the flowcharts and/or block diagrams.

The flowchart and block diagrams in the drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products in accordance with multiple embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, program segment, or portion of an instruction comprising one or more executable instructions for implementing a specified logical function. It should be noted that, in some alternative implementations, the functions indicated in the blocks may be implemented in a different order than shown in the figures. For example, two consecutive blocks may actually be performed in parallel, or may sometimes be performed in the reverse order according to a related function. Further, each block in the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts, may be implemented by a dedicated hardware-based system for performing specified functions or operations, or a combination of dedicated hardware and computer instructions It should be noted that it may be implemented by

The computer program product may be specifically implemented in hardware, software, or a combination thereof. In one selectable embodiment, the computer program product is specifically embodied as a computer storage medium, and in another selectable embodiment, the computer program product is specifically embodied as a software product such as a Software Development Kit (SDK) or the like. do.

In the above content, each embodiment of the present invention has already been described, and the above description is only illustrative and not exhaustive, and is not limited to each disclosed embodiment. Many modifications and changes made without departing from the scope and spirit of each described embodiment will be apparent to those skilled in the art. The choice of terminology used herein best interprets the principle of each embodiment, practical application, or technological improvement over market technology, or may allow others skilled in the art to understand each embodiment disclosed herein. is to make it

Industrial Applicability

An embodiment of the present invention provides a clustering method and apparatus, an electronic device, and a storage medium, wherein the clustering method performs quantitative analysis on a first cluster of facial features, so that the first cluster is determined by a preset clustering condition determining whether the first cluster is obtained by clustering based on a first threshold, and the preset clustering condition indicates that a plurality of facial features included in the first cluster correspond to the same identity. instruct - ; releasing the plurality of facial features included in the first cluster when the first cluster does not meet the preset clustering condition; and clustering the plurality of facial features to determine a second cluster based on a second threshold, wherein the second threshold is greater than the first threshold; includes

Claims (20)

A clustering method comprising:
performing quantitative analysis on a first cluster of facial features to determine whether the first cluster meets a preset clustering condition - the first cluster is obtained by clustering based on a first threshold, the preset clustering condition indicates that a plurality of facial features included in the first cluster correspond to the same identity;
releasing the plurality of facial features included in the first cluster when the first cluster does not meet the preset clustering condition; and
determining a second cluster by clustering the plurality of facial features based on a second threshold, wherein the second threshold is greater than the first threshold; A clustering method comprising According to claim 1,
the plurality of facial features includes one class-centric facial feature corresponding to the first cluster;
The step of performing quantitative analysis on the first cluster to determine whether the first cluster satisfies a preset clustering condition includes:
determining a cosine distance between the class-centric facial feature and the other of the first clusters;
determining an average distance and a standard deviation distance according to the cosine distance; and
determining that the first cluster does not meet the preset clustering condition when at least one of the average distance is greater than a third threshold value and the standard deviation distance is less than a fourth threshold value; A clustering method that includes. 3. The method of claim 1 or 2,
After determining the second cluster by clustering the plurality of facial features based on the second threshold value, the clustering method includes:
determining N known clusters according to at least one of the second cluster and the first cluster meeting the preset clustering condition, wherein N≥1; and
and classifying facial features to be clustered according to the N known clusters. 4. The method of claim 3,
After determining N known clusters according to at least one of the second cluster and the first cluster meeting the preset clustering condition, the clustering method includes:
clustering class-centric facial features corresponding to the N known clusters to obtain a class-centric cluster; and
and merging the known clusters corresponding to each class-centric facial feature included in the class-centric cluster to obtain a merged cluster corresponding to the same identity. 5. The method according to claim 3 or 4,
Classifying facial features to be clustered according to the N known clusters includes:
determining whether a target known cluster exists in the N known clusters, wherein a cosine distance between the clustered facial feature and a class-centered facial feature corresponding to the target known cluster is less than a fifth threshold; and
and when the target known clusters exist in the N known clusters, classifying the facial features to be clustered into the target known clusters. 6. The method of claim 5,
Determining whether a target base cluster exists in the N base clusters comprises:
determining k class-centric facial features having the closest Euclidean distance from the class-centric facial features corresponding to the N known clusters to the clustered facial features using a k-nearest neighbor algorithm, wherein: N≥k≥1 - ;
determining a cosine distance between the facial features to be clustered and the k class-centered facial features, respectively; and
and determining, as the target known cluster, a known cluster corresponding to a class-centered facial feature having a cosine distance smaller than the fifth threshold. 7. The method according to any one of claims 3 to 6,
After classifying facial features to be clustered according to the N known clusters, the clustering method includes:
updating a plurality of facial features included in the N known clusters based on a result of classifying the facial features to be clustered; and
The clustering method further comprising: for any one of the known clusters, updating a class-centric facial feature corresponding to the known cluster according to a plurality of updated facial features corresponding to the known cluster. 8. The method according to any one of claims 5 to 7,
After determining whether a target base cluster exists in the N base clusters, the clustering method includes:
determining the clustered facial feature as an unclassified facial feature when the target known cluster does not exist in the N known clusters; and
clustering a plurality of unclassified facial features at a preset duration using a k-nearest neighbor algorithm and a graph algorithm to determine a newly added cluster. 9. The method of claim 8,
After determining a newly added cluster by clustering a plurality of unclassified facial features at a preset duration using the k-nearest neighbor algorithm and the graph algorithm, the clustering method comprises:
The clustering method further comprising the step of updating the N base clusters according to the newly added cluster. A clustering device comprising:
A quantitative analysis module, configured to perform quantitative analysis on a first cluster of facial features to determine whether the first cluster meets a preset clustering condition, wherein the first cluster is clustered based on a first threshold obtained, and the preset clustering condition indicates that a plurality of facial features included in the first cluster correspond to the same identity;
a release module, configured to cancel the plurality of facial features included in the first cluster when the first cluster does not meet the preset clustering condition; and
a clustering module, configured to cluster the plurality of facial features to determine a second cluster based on a second threshold, wherein the second threshold is greater than the first threshold; A clustering device comprising a. 11. The method of claim 10,
the plurality of facial features includes one class-centric facial feature corresponding to the first cluster; The quantitative analysis module,
a first determining submodule, configured to determine a cosine distance between the class-centric facial feature and the other one of the first clusters;
a second determining submodule, configured to determine a mean distance and a standard deviation distance according to a cosine distance between the class-centric facial feature and the other facial feature in the first cluster; and
and when the average distance is greater than a third threshold value and the standard deviation distance is less than a fourth threshold value, it is determined that the first cluster does not meet the preset clustering condition. A clustering device comprising a third determining sub-module. 12. The method of claim 10 or 11,
The clustering device is
a first determining module, configured to determine, according to at least one of the second cluster and the first cluster meeting the preset clustering condition, N known clusters, wherein N≥1; and
and a classification module, configured to classify facial features to be clustered according to the N known clusters. 13. The method of claim 12,
the clustering module is further configured to cluster class-centric facial features corresponding to the N known clusters to obtain a class-centric cluster;
The clustering device is
and a merging module, configured to merge the known clusters corresponding to each class-centric facial feature included in the class-centric cluster to obtain a merged cluster corresponding to the same identity. 14. The method of claim 12 or 13,
The classification module is
a fourth determining submodule, configured to determine whether a target known cluster exists in the N known clusters; a cosine distance between the clustered facial feature and a class-centric facial feature corresponding to the target known cluster is greater than a fifth threshold. small - ; and
and a classification submodule configured to classify the facial feature to be clustered into the target known cluster when the target known cluster exists in the N known clusters. 15. The method of claim 14,
The fourth determining sub-module,
a first, configured to determine, in class-centric facial features corresponding to the N known clusters, k class-centric facial features that have the closest Euclidean distance to the clustered facial features by using a k-nearest neighbor algorithm Determining unit, wherein N≥k≥1;
a second determining unit, configured to respectively determine a cosine distance between the facial features to be clustered and the k class-centric facial features; and
and a third determining unit, configured to determine, as the target known cluster, a known cluster corresponding to a class-centric facial feature whose cosine distance is less than the fifth threshold value. 16. The method according to any one of claims 12 to 15,
The clustering device is
a first update module configured to update a plurality of facial features included in the N known clusters based on a result of classifying the facial features to be clustered; and
The clustering device further comprising a second update module, configured to update, with respect to any one of the known clusters, a class-oriented facial feature corresponding to the known cluster according to the plurality of updated facial features corresponding to the known cluster. . 17. The method according to any one of claims 14 to 16,
The clustering device is
a second determining module, configured to determine the clustered facial feature as an unclassified facial feature when the target known cluster does not exist in the N known clusters;
and the clustering module is further configured to cluster a plurality of unclassified facial features at a preset duration by using a k-nearest neighbor algorithm and a graph algorithm to determine a newly added cluster. 18. The method of claim 17,
The clustering device is
and a third update module, configured to update the N base clusters according to the newly added cluster. As an electronic device,
a memory storing instructions executable by the processor; and
a processor for executing the method according to any one of claims 1 to 9 by calling the instruction stored in the memory; An electronic device comprising a. A computer readable storage medium having computer program instructions stored thereon, comprising:
10. A computer-readable storage medium that, when the computer program instructions are executed by a processor, causes the processor to implement the method according to any one of claims 1 to 9.

Images