CN114139008A

CN114139008A - Image clustering processing method, computer equipment and storage device

Info

Publication number: CN114139008A
Application number: CN202111205564.5A
Authority: CN
Inventors: 江中毅; 张宏; 陈立力; 周明伟
Original assignee: Zhejiang Dahua Technology Co Ltd
Current assignee: Zhejiang Dahua Technology Co Ltd
Priority date: 2021-10-15
Filing date: 2021-10-15
Publication date: 2022-03-04

Abstract

The application discloses a processing method of image clustering, computer equipment and a storage device. The method comprises the following steps: acquiring the time-space information of each centroid in two clusters to be clustered, wherein each cluster to be clustered comprises at least one centroid respectively; matching the centroids in the two clusters to be clustered to form a centroid pair based on the time-space information of each centroid, and distributing the centroid pair to a corresponding constraint group; and clustering the two clusters to be clustered based on the similarity of the constraint groups. By the scheme, the image clustering efficiency can be improved.

Description

Image clustering processing method, computer equipment and storage device

Technical Field

The present application relates to the field of image processing technologies, and in particular, to a processing method, a computer device, and a storage device for image clustering.

Background

The clustering algorithm is an important unsupervised learning method and has good applicability in the fields of machine learning, bioinformatics, pattern recognition, multimedia and the like. For example, clustering of facial images plays an important role in the field of intelligent security. The clustering of the face images can lead the face images belonging to the same person to be classified into the same cluster, and the purpose of grouping the face images of a plurality of persons by taking the person as a unit is realized. Currently, however, each cluster class may include one or more centroids, which may be used to represent all of the face images in the cluster class. Clustering is carried out on the face images through a single mass center, the clustering effect is poor, the face images are clustered through a plurality of mass centers, and the clustering efficiency is low.

Disclosure of Invention

The technical problem mainly solved by the application is to provide an image clustering processing method, a computer device and a storage device, which can improve the image clustering efficiency.

In order to solve the above problem, a first aspect of the present application provides a method for processing image clusters, the method including: acquiring the time-space information of each centroid in two clusters to be clustered, wherein each cluster to be clustered comprises at least one centroid; matching the centroids in the two clusters to be clustered to form a centroid pair based on the time-space information of each centroid, and distributing the centroid pair to a corresponding constraint group; and clustering the two clusters to be clustered based on the similarity of the constraint groups.

In order to solve the above problem, a second aspect of the present application provides a computer device, which includes a memory and a processor coupled to each other, the memory having stored therein program data, and the processor being configured to execute the program data to implement any one of the steps in the processing method for image clustering described above.

In order to solve the above problem, a third aspect of the present application provides a storage device storing program data executable by a processor, the program data being used to implement any one of the steps in the above-described image clustering processing method.

According to the scheme, the spatio-temporal information of each centroid in two clusters to be clustered is acquired, wherein each cluster to be clustered comprises at least one centroid; matching the centroids in the two clusters to be clustered to form a centroid pair based on the time-space information of each centroid, and distributing the centroid pair to a corresponding constraint group; based on the similarity of the constraint group, two clusters to be clustered are clustered, and the clusters are clustered through the spatio-temporal information of the mass center, so that the image clustering effect is improved.

Drawings

In order to more clearly illustrate the technical solutions in the present application, the drawings required in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings described below are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive labor. Wherein:

FIG. 1 is a schematic flowchart of an embodiment of a processing method for image clustering according to the present application;

FIG. 2 is a flowchart illustrating an embodiment of step S12 in FIG. 1;

FIG. 3 is an exemplary diagram illustrating an embodiment of a class cluster of the image clustering of the present application;

FIG. 4 is a diagram illustrating an example of distance information of a location A, a location B, and a location C according to an embodiment of the present invention;

FIG. 5 is an exemplary diagram of one embodiment of a centroid of a cluster class of the present application;

FIG. 6 is a flowchart illustrating an embodiment of step S13 of FIG. 1;

FIG. 7 is a schematic structural diagram of an embodiment of a processing apparatus for image clustering according to the present application;

FIG. 8 is a schematic block diagram of an embodiment of a computer apparatus of the present application;

FIG. 9 is a schematic structural diagram of an embodiment of a memory device according to the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The terms "first" and "second" in this application are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present application, "plurality" means at least two, e.g., two, three, etc., unless explicitly specifically limited otherwise. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the specification. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.

Clustering is the process of dividing a collection of physical or abstract objects into classes composed of similar objects, called Clustering. The cluster generated by clustering is a collection of a set of data objects that are similar to objects in the same cluster and distinct from objects in other clusters. The clustering process classifies data into different classes or clusters, so that objects in the same cluster have great similarity, and objects in different clusters have great dissimilarity. For example, clustering may partition a data set into different clusters according to a certain criterion (e.g., distance or time), so that the similarity of data objects in the same cluster is as large as possible, and the difference of data objects not in the same cluster is as large as possible. That is, after clustering, the data of the same class are gathered together as much as possible, and the data of different classes are separated as much as possible.

Clustering algorithms can be classified into Partition-based clustering Methods, Density-based clustering Methods, Hierarchical clustering Methods, and the like. For example, the clustering algorithm is a k-means clustering algorithm, a k-center clustering algorithm, a BIRCH clustering algorithm or a DBSCAN clustering algorithm, and the like, which is not limited in this application.

In some application scenarios, clustering may be performed on images, for example, clustering may be performed on a large number of accumulated face images, and by clustering the face images, archiving images containing personal information in units of persons may be achieved.

In the clustering method of the face images, each cluster usually corresponds to only a single centroid, that is, one cluster corresponds to one centroid, and one centroid represents the whole cluster. However, in the actual process of clustering face images, a single centroid is used for image clustering, and a good image clustering effect cannot be achieved.

In some embodiments, in the face image clustering method, a plurality of centroids may be set for each cluster, that is, each cluster may correspond to a plurality of centroids, so that the cluster may be represented more comprehensively from a plurality of different angles. However, the number of comparisons among a plurality of centroids is increased in the clustering process, thereby reducing the clustering efficiency of the images.

In order to solve the above technical problems, the present application provides the following embodiments, each of which is specifically described below.

Referring to fig. 1, fig. 1 is a schematic flowchart illustrating an embodiment of a processing method for image clustering according to the present application. The method may comprise the steps of:

s11: the method comprises the steps of obtaining the time-space information of each centroid in two clusters to be clustered, wherein each cluster to be clustered comprises at least one centroid.

The clustering of the face images is taken as an example for explanation, and the clustering method can also be applied to clustering of other data sets, but the method is not limited to this.

The spatio-temporal information of each centroid in the two clusters to be clustered is obtained, and the spatio-temporal information of each centroid can comprise time information and space information. Wherein each cluster to be clustered comprises at least one centroid respectively. Each cluster may include one centroid, each centroid may also include a plurality of centroids, and the centroids may be used to represent all face images in the cluster to be clustered.

In some embodiments, a face image set to be clustered is obtained, the face image set may be clustered to obtain a plurality of clusters to be clustered, and any two clusters may be selected from the plurality of clusters as the obtained two clusters to be clustered. The purpose of clustering the face images is to classify the face images belonging to the same person into the same cluster, so that the situation of one person and one file is realized. One cluster may include one centroid, and one cluster may also include a plurality of centroids, which is not limited in this application.

In some embodiments, a set of face images to be clustered may be obtained; dividing a face image set into a plurality of face image subsets; and clustering each face image subset into a plurality of clusters to be clustered. The face image set can be divided into a plurality of face image subsets according to similar spatio-temporal information. For example, facial images with close shooting time and close shooting space (or place) are divided into the same facial image subset. Of course, the face image set may also be divided into a plurality of face image subsets according to similar spatial information.

In some embodiments, in an intelligent video monitoring application scene, a person may be shot through monitoring shooting, and a plurality of shot face images are used as a face image set to be clustered. Each face image of the face image set corresponds to a shooting time and a shooting place. Time information and space information corresponding to each face image can be obtained based on the shooting time and the shooting place corresponding to each face image, and therefore the space-time information of each centroid of the cluster can be obtained through the space-time information of the face images.

S12: and matching the centroids in the two clusters to be clustered to form a centroid pair based on the spatio-temporal information of each centroid, and distributing the centroid pair to a corresponding constraint group.

After obtaining the spatio-temporal information of each centroid of a cluster, that is, after obtaining the time information and the spatial information of each centroid, for example, the centroids in two clusters to be clustered may be paired to form a centroid pair according to the time information and the spatial information, and two centroids included in one centroid pair may be centroids from different clusters. Each centroid pair is thus assigned to a corresponding set of constraints, where a set of constraints may be a set of constraints that are constrained based on temporal information and spatial information.

S13: and clustering the two clusters to be clustered based on the similarity of the constraint groups.

And acquiring the similarity between the centroid pairs in each constraint group, and clustering the two clusters to be clustered based on the similarity of the constraint groups.

The similarity between the face image/centroid pair mentioned in the present application is the similarity of the face image/centroid pair in the feature space, that is, the similarity of the features of the face image/centroid pair.

In some embodiments, whether the similarity of the constraint group is greater than a preset similarity threshold value or not may be determined according to the determination result, so as to perform clustering processing on the two clusters to be clustered. If the similarity of the constraint group is greater than the preset similarity threshold, it is determined that the two clusters are similar, and merging (filing) processing can be performed on the two clusters to be clustered. Otherwise, the two clusters to be clustered may be retained without merging, or the two clusters to be clustered and the other clusters are respectively executed with the above steps S11 to S13, so as to further perform clustering processing on the clusters to be clustered.

In the embodiment, the spatiotemporal information of each centroid in two clusters to be clustered is acquired, wherein each cluster to be clustered comprises at least one centroid; matching the centroids in the two clusters to be clustered to form a centroid pair based on the time-space information of each centroid, and distributing the centroid pair to a corresponding constraint group; based on the similarity of the constraint group, two clusters to be clustered are clustered, and the clusters are clustered through the spatio-temporal information of the mass center, so that the image clustering effect is improved.

In some embodiments, referring to fig. 2, the step S12, based on the spatio-temporal information of each centroid, pairing centroids in two clusters to be clustered to form a centroid pair, and assigning the centroid pair to a corresponding constraint group, may further include the following steps:

s121: based on the spatiotemporal constraint levels, constraint groups of a plurality of spatiotemporal constraints are set, and a weight of each constraint group is set.

After the spatiotemporal information of each centroid in two clusters to be clustered is obtained, a plurality of constraint groups of spatiotemporal constraints can be set based on spatiotemporal constraint levels, wherein the spatiotemporal constraint levels can carry out different levels of constraints on the spatiotemporal information.

The spatio-temporal information of each centroid includes temporal information and spatial information, and the spatio-temporal constraint levels include temporal constraint levels and spatial constraint levels for better constraining the temporal information and the spatial information. So that the temporal information and the spatial information can be separately constrained. The temporal constraint level may be determined according to the distribution of the temporal information of the face image set or all the class clusters, for example, the temporal constraint may be divided into 3 levels, 4 levels, and so on. The spatial constraint level may be determined according to the distribution of the spatial information of the face image set or all the class clusters, for example, the spatial constraint may be divided into 3 levels, 4 levels, and so on. This is not limited by the present application.

After constraint groups of a plurality of space-time constraints are set, the weight of each constraint group is set, wherein the constraint groups comprise constraint groups with weights different from 0 and constraint groups with weights of 0, and the sum of the weights of all the constraint groups is 1. And the weight of the set of constraints at a higher level of spatio-temporal constraints is greater than the weight of the set of constraints at a lower level of spatio-temporal constraints. The weights of the constraint groups may be set according to specific clustering data, which is not limited in this application.

As an example, referring to fig. 3, face images obtained by shooting a plurality of times may be acquired to form a face image set 20, and the face image set 20 may be clustered. As described with reference to fig. 3, the face image set 20 is clustered to obtain three clusters, that is, the face image set 20 is divided into a cluster 1(21), a cluster 2(22), and a cluster 3(23), where each cluster includes a plurality of face images.

Included in cluster 1(21) are 4 centroids, respectively centroid 1(211), centroid 2(212), centroid 3(213), and centroid 4(214), which may own the representation cluster 1 (21).

Included in class cluster 2(22) are 2 centroids, centroid 5(221) and centroid 6(222), respectively, which may hold a representation of class cluster 2 (22).

Included in cluster 3(23) are 2 centroids, centroid 7(231) and centroid 8(232), respectively, which 2 centroids may hold the representation cluster 3 (23).

Referring to fig. 4, fig. 4 is an exemplary diagram of distance information of a location a, a location B, and a location C according to an embodiment of the present application. If the human face images are shot at the position A, the position B and the position C respectively, the human face image sets of the three positions are obtained. The distance between the site A and the site B is 1200 meters, the distance between the site A and the site C is 200 meters, and the distance between the site B and the site C is 1000 meters. The spatial information of the face image acquired at the location a may be the location a, the spatial information of the face image acquired at the location B may be the location B, and the spatial information of the face image acquired at the location C may be the location C.

And clustering the face image sets obtained from the position A, the position B and the position C to obtain at least two clusters to be clustered. Referring to fig. 5, for example, the face image set is divided into a cluster 1(31) and a cluster 2(32), the cluster 1(31) may include 4 centroids, and the 4 centroids may be a centroid 1, a centroid 2, a centroid 3, and a centroid 4. The centroid may be the face image in the cluster 1(31), and the centroid 1, the centroid 2, the centroid 3, or the centroid 4 may be used to represent all the face images in the cluster 1 (31). The cluster 2(32) may contain 2 centroids, and the 2 centroids may be the centroid 5 and the centroid 6, and the centroid 5 or the centroid 6 may be used to represent all the face images in the cluster 2 (32).

In some embodiments, the time information corresponding to each centroid may be time information corresponding to a face image, or may be time information corresponding to each face image obtained by dividing shooting time of the face image according to a preset time interval. For example, at 1 hour intervals, the shooting time may be changed from 0 to 24 points at 1 hour intervals, for example, the shooting time may be changed from 8 to 9 points at 9 points, the shooting time may be changed from 9 to 10 points at 10 points, and so on. Of course, the time information corresponding to each face image may also be set by using other preset time intervals, and the shooting time of the face image may also be directly used as the time information, which is not limited in the present application.

In some embodiments, each centroid in the cluster class may be represented, that is, cluster class information, time information, and spatial information of the centroid may be represented in the form of: { class cluster identification [ centroid sequence number, spatial information, time information ] }. Of course, other representations of the information of the centroid of the cluster of classes may be used.

With continued reference to fig. 4, each centroid in cluster 1(31) may be represented, and centroid 1(311) may be represented as: { class 1[1, a, 8 }, centroid 2(312) can be expressed as: { class 1[2, C, 9] }, centroid 3(313) can be expressed as: { class 1[3, B, 9] }, centroid 4(314) can be represented as: { class 1[4, C, 22] }. Similarly, each centroid in the cluster 2(32) is represented, and the centroid 5(321) can be represented as: { class 2[1, a, 7] }, centroid 6(322) can be represented as: { class 2[2, B, 9 }.

If the distinction between the time information and the space information of the centroid in the cluster is not considered, in order to measure whether the cluster 1(31) is similar to the cluster 2, the centroids of the cluster 1(31) and the cluster 2(32) need to be compared in pairs, at this time, the cluster 1(31) comprises 4 centroids, and the cluster 2(32) comprises 2 centroids, which needs to be compared for 8 times. In order to reduce the number of comparisons of centroids of the clusters 1(31) and 2(32), a plurality of constraint groups of spatio-temporal constraints may be set based on spatio-temporal constraint levels, wherein the spatio-temporal constraint levels include temporal constraint levels and spatial constraint levels.

As an example one implementation, the centroids are grouped by a temporal constraint level and a spatial constraint level, the temporal constraint level may include a first temporal constraint, a second temporal constraint, and a third temporal constraint. The first time constraint may be that the time information differs by 0 or equal, i.e. the time information of the centroid is the same, indicating that the time of the centroid is the closest. The second time constraint may be that the time information differs by less than or equal to 2 hours, i.e., the times representing the centroids are close. A third time constraint may be that the time information differs by more than 2 hours, i.e. the times representing the centroids are not similar. Wherein the levels of the first time constraint, the second time constraint and the third time constraint are sequentially decreased.

The spatial constraint levels may include a first spatial constraint, a second spatial constraint, and a third spatial constraint. The first spatial constraint may be that the distances of the spatial information are 0 or the same, that is, the distance representing the centroid is closest, and the second spatial constraint may be that the distances of the spatial information are less than or equal to 500 meters, that is, the distances representing the centroids are close. A third spatial constraint may be that the distance of the spatial information is greater than 500 meters, i.e. that the centroid is that i are not close in distance. Wherein the levels of the first spatial constraint, the second spatial constraint and the third spatial constraint are sequentially decreased.

Based on the spatiotemporal constraint levels, constraint groups of a plurality of spatiotemporal constraints are set, and a weight of each constraint group is set. Its constraint groups and the weights of each of its constraint groups can be represented by the following table:

TABLE 1 weight table for constraint groups

In table 1, a to i represent a constraint group in which a plurality of spatio-temporal constraints are set based on spatio-temporal constraint levels, and constraint group a is set based on a first constraint time and a first spatial constraint, and the weight of constraint group a is 0.25. In addition, the weight of the constraint group with a higher spatio-temporal constraint level is greater than the weight of the constraint group with a lower spatio-temporal constraint level, for example, the weight corresponding to constraint group a is greater than the weight corresponding to constraint group b or constraint group d.

In the step, different weights are set for the constraint groups corresponding to each piece of space-time information, so that weights with different sizes can be set for the different constraint groups according to the importance degree of the constraint groups, the weight of the constraint group with a higher space-time constraint level is greater than the weight of the constraint group with a lower space-time constraint level, and the importance of similarity comparison of centroids with close time information and space information is met.

S122: and pairing the centroids in the two clusters to be clustered to form a centroid pair.

The pairs of centroids in two clusters to be clustered can be combined into a centroid pair based on the space-time constraint level, that is, any centroids from different clusters can be combined into a centroid pair.

As an example, pairs of centroids may be formed for the 4 centroids of cluster 1 and 2 centroids of cluster 2 described above. The centroids of class 1 and class 2 may be paired into centroid pairs according to temporal constraint levels and/or spatial constraint levels.

S123: the centroid pairs are assigned to corresponding constraint groups based on spatiotemporal information of the centroids in the centroid pairs.

And distributing the centroid pairs to corresponding constraint groups according to the time constraint level and the space constraint level based on the spatio-temporal information of the centroids in the centroid pairs.

In some embodiments, the spatiotemporal constraint levels corresponding to the centroid pairs, that is, the temporal constraint levels and/or the spatial constraint levels corresponding to the centroid pairs, may be determined based on the temporal information and/or the spatial information of the centroids in the centroid pairs, so that the centroid pairs are assigned to the corresponding constraint groups according to the temporal constraint levels and the spatial constraint levels.

As an example, the centroid pairs of cluster 1 and cluster 2 are assigned to corresponding constraint groups. The spatio-temporal constraint level corresponding to the centroid pair, that is, the time constraint level and/or the space constraint level corresponding to the centroid pair, may be determined based on the spatio-temporal information of the centroids in the centroid pair, so that the centroids of the cluster 1 and the cluster 2 are grouped into the centroid pair to be assigned to the corresponding constraint group according to the time constraint level and the space constraint level.

Wherein the constraint groups and the centroid pairs of each of the constraint groups can be represented by the following table:

TABLE 2 constrained group of centroids

In table 2 above, the pair of centroids included in constraint group a is centroid 3{ class 1[3, B, 9] } and centroid 6{ class 2[2, B, 9] }.

The pair of centroids included in the constraint group c is centroid 1{ class 1[1, a, 8] } and centroid 6{ class 2[2, B, 9] }.

The pair of centroids included in the constraint group d is centroid 1{ class 1[1, A, 8] } and centroid 5{ class 2[1, A, 7] }.

The pair of centroids included in the constraint group e is centroid 2{ class 1[2, C, 9] } and centroid 5{ class 2[1, a, 7] }.

The pair of centroids included in the constraint group f is centroid 2{ class 1[2, C, 9] } and centroid 6{ class 2[2, B, 9] }.

The pair of centroids included in the constraint group h is centroid 4{ class 1[4, C, 22] } and centroid 5{ class 2[1, a, 7] }.

The pair of centroids included in the constraint group i is centroid 3{ class 1[3, B, 9] } and centroid 5{ class 2[1, a, 7] }, centroid 4{ class 1[4, C, 22] } and centroid 6{ class 2[2, B, 9] }.

In addition, in the above example of cluster 1 and cluster 2, the constraint group b and constraint g do not include a centroid pair, that is, no centroid pair is assigned to the constraint group b and constraint g.

In this embodiment, under the condition that the cluster includes a plurality of centroids, the time constraint level and the space constraint level are constrained according to the time information and the space information of the centroids, and the centroid pairs are allocated to the corresponding constraint groups in consideration of the conditions of the centroids in time and space, so that the final clustering effect on the images can be improved, the comparison times of the multiple centroids among the clusters can be reduced, and the clustering efficiency of the images can be improved.

In some embodiments, referring to fig. 6, the step S13, performing clustering processing on two clusters to be clustered based on the similarity of the constraint groups, may further include the following steps:

s131: and acquiring the similarity of the centroid pairs in the constraint groups of the parts as the similarity of the corresponding constraint groups.

The similarity between the centroid pairs in the constraint group can be obtained, and the similarity of the centroid pairs is taken as the similarity of the corresponding constraint group. Wherein, the similarity of the centroid pair in one constraint group can be used as the similarity of the one constraint group.

In some embodiments, the similarity of the centroid pairs in the constraint groups in the part of all the constraint groups may be obtained, so as to obtain the similarity of all the constraint groups through the similarity of the centroid pairs in the constraint groups in the part of the constraint groups.

In some embodiments, the partial constraint groups include a constraint group with a weight not being 0, that is, the similarity of the centroid pair in the constraint group with a weight not being 0 may be obtained, so as to obtain the similarity of all the constraint groups by using the similarity of the centroid pair in the constraint group with a weight not being 0.

In some embodiments, when assigning pairs of centroids to corresponding constraint groups according to spatio-temporal constraint levels based on spatio-temporal information of centroids in pairs of centroids, one constraint group may include one pair of centroids, may include multiple pairs of centroids, and of course, may not include pairs of centroids.

If any constraint group includes a pair of centroids, the similarity of centroids in the pair of centroids included in the constraint group can be obtained as the similarity of the corresponding constraint group.

If any one restraint group comprises a plurality of pairs of centroid pairs, the similarity of the centroids in each centroid pair is obtained, namely the similarities of the plurality of pairs of centroid pairs are respectively obtained, and the average similarity of the similarities of the plurality of pairs of centroid pairs is obtained to be used as the similarity of the corresponding restraint group.

If no centroid pair is included in any of the constraint groups, the similarity of the constraint group may not be obtained.

If the weight corresponding to any constraint group is 0, the similarity of the constraint group may not be obtained.

S132: and acquiring the comprehensive similarity based on the similarity of the constraint group and the weight of the constraint group.

After the similarity of the constraint groups is obtained, the comprehensive similarity may be obtained based on the similarity of the constraint groups and the weights of the constraint groups. Specifically, the acquired similarity of the constraint group may be multiplied by the similarity of the constraint group, and the products are summed, and the sum is taken as the integrated similarity of all the constraint groups.

S133: and based on the comprehensive similarity, clustering the two clusters to be clustered.

Whether the two clusters are similar or not can be determined according to the comprehensive similarity, so that the two clusters to be clustered are clustered. If the comprehensive similarity is larger than the preset similarity threshold, the two clusters are determined to be similar, and the two clusters to be clustered can be merged. Otherwise, the two clusters to be clustered are retained and not merged, or the two clusters to be clustered are respectively compared with the similarity of other clusters, so as to further perform clustering processing on the other clusters to be clustered.

In this embodiment, by obtaining the similarity of the weight non-0 constraint group, the number of times of comparison between the plurality of centroid pairs can be reduced, that is, the number of times of calculating the similarity between the centroids in the centroid pairs can be reduced. In addition, the comprehensive similarity is obtained based on the similarity of the constraint groups and the weight of the constraint groups, the two clusters to be clustered are clustered based on the comprehensive similarity, the similarity between the clusters can be comprehensively considered in two aspects of time and space, the two clusters are clustered, the clustering effect can be improved, and the clustering efficiency is improved.

With respect to the above embodiments, the present application provides a processing apparatus for image clustering. Referring to fig. 7, fig. 7 is a schematic structural diagram of an embodiment of a processing device for image clustering according to the present application. The image clustering processing device 70 may comprise an obtaining module 71, an assigning module 72 and a clustering module 73, wherein the obtaining module 71, the assigning module 72 and the clustering module 73 are connected.

The obtaining module 71 is configured to obtain spatio-temporal information of each centroid in two clusters to be clustered, where each cluster to be clustered includes at least one centroid.

The distribution module 72 is configured to pair the centroids in the two clusters to be clustered to form a centroid pair based on the spatiotemporal information of each centroid, and distribute the centroid pair to a corresponding constraint group;

the clustering module 73 is configured to perform clustering processing on two clusters to be clustered based on the similarity of the constraint groups.

In some embodiments, the assigning module 72 is configured to pair centroids in two clusters to be clustered into centroid pairs based on the spatiotemporal information of each centroid, and assign the centroid pairs to corresponding constraint groups, including: setting a plurality of constraint groups of space-time constraint based on the space-time constraint level, and setting the weight of each constraint group; matching centroids in two clusters to be clustered to form a centroid pair; the centroid pairs are assigned to corresponding constraint groups based on spatiotemporal information of the centroids in the centroid pairs.

In some embodiments, the spatiotemporal constraint levels include temporal constraint levels and spatial constraint levels, and the spatiotemporal information for each centroid includes temporal information and spatial information.

In some embodiments, the clustering module 73 is configured to perform clustering processing on two clusters to be clustered based on the similarity of the constraint groups, including: acquiring the similarity of the centroid pairs in the partial constraint groups as the similarity of the corresponding constraint groups; acquiring comprehensive similarity based on the similarity of the constraint groups and the weight of the constraint groups; and based on the comprehensive similarity, clustering the two clusters to be clustered.

In some embodiments, the set of constraints for the portion includes a set of constraints with weights other than 0. The sum of the weights of the constraint groups is 1.

In some embodiments, the set of constraints having a higher level of spatiotemporal constraints are weighted more heavily than the set of constraints having a lower level of spatiotemporal constraints.

In some embodiments, the clustering module 73 is configured to obtain the similarity of the centroid pairs in the constraint groups of the parts as the similarity of the corresponding constraint groups, and includes: if any constraint group comprises a pair of centroid pairs, acquiring the similarity of the centroids in the centroid pairs as the similarity of the corresponding constraint group; if any one of the constraint groups comprises a plurality of pairs of centroids, the similarity of the centroids in each pair of centroids is obtained, and the average similarity is obtained to be used as the similarity of the corresponding constraint group.

The specific implementation of this embodiment can refer to the implementation process of the above embodiment, and is not described herein again.

With reference to fig. 8, fig. 8 is a schematic structural diagram of an embodiment of a computer device according to the present application. The computer device 80 comprises a memory 81 and a processor 82, wherein the memory 81 and the processor 82 are coupled to each other, the memory 81 stores program data, and the processor 82 is configured to execute the program data to implement the steps in any of the embodiments of the image clustering processing method described above.

In the present embodiment, the processor 82 may also be referred to as a CPU (Central Processing Unit). The processor 82 may be an integrated circuit chip having signal processing capabilities. The processor 82 may also be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. A general purpose processor may be a microprocessor or the processor 82 may be any conventional processor or the like.

For the method of the above embodiment, it can be implemented in the form of a computer program, so that the present application provides a storage device, please refer to fig. 9, where fig. 9 is a schematic structural diagram of an embodiment of the storage device of the present application. The storage device 90 stores therein program data 91 executable by a processor, the program data being executable by the processor to implement the steps of any one of the embodiments of the processing method for image clustering described above.

The storage device 90 of this embodiment may be a medium that can store program data, such as a usb disk, a portable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, or may be a server that stores the program data, and the server may transmit the stored program data to another device for operation, or may self-operate the stored program data.

In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a module or a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some interfaces, and may be in an electrical, mechanical or other form.

Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in a storage device, which is a computer-readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing an electronic device (which may be a personal computer, a server, a network device, or the like) or a processor (processor) to execute all or part of the steps of the method according to the embodiments of the present application.

It will be apparent to those skilled in the art that the modules or steps of the present application described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and they may alternatively be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, or fabricated separately as individual integrated circuit modules, or fabricated as a single integrated circuit module from multiple modules or steps. Thus, the present application is not limited to any specific combination of hardware and software.

The above description is only an example of the present application and is not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings, or which are directly or indirectly applied to other related technical fields, are intended to be included within the scope of the present application.

Claims

1. A method for processing image clusters, the method comprising:

acquiring spatiotemporal information of each centroid in two clusters to be clustered, wherein each cluster to be clustered comprises at least one centroid respectively;

matching the centroids in the two clusters to be clustered to form a centroid pair based on the spatiotemporal information of each centroid, and allocating the centroid pair to a corresponding constraint group;

and clustering the two clusters to be clustered based on the similarity of the constraint groups.

2. The method according to claim 1, wherein the clustering the two clusters to be clustered based on the similarity of the constraint group comprises:

acquiring the similarity of the centroid pairs in the constraint groups of the parts as the similarity of the corresponding constraint groups;

acquiring comprehensive similarity based on the similarity of the constraint groups and the weight of the constraint groups;

and based on the comprehensive similarity, clustering the two clusters to be clustered.

3. The method of claim 2, wherein the set of constraints for a portion comprises a set of constraints having a weight other than 0.

4. The method of claim 3, wherein obtaining the similarity of the centroid pair in the constraint group of the portion as the similarity of the corresponding constraint group comprises:

if any one of the constraint groups comprises a pair of the centroids, acquiring the similarity of the centroids in the centroids pair as the similarity corresponding to the constraint group;

if any one of the constraint groups comprises a plurality of pairs of the centroids, obtaining the similarity of the centroids in each pair of the centroids, and obtaining the average similarity as the similarity corresponding to the constraint group.

5. The method according to claim 1, wherein the pairing the centroids in the two clusters to be clustered into centroid pairs based on the spatiotemporal information of each centroid, and assigning the centroid pairs into corresponding constraint groups comprises:

setting said set of constraints of a plurality of spatio-temporal constraints based on spatio-temporal constraint levels and setting a weight for each said set of constraints;

pairing the centroids in the two clusters to be clustered to form the centroid pair;

assigning the pairs of centroids to the corresponding constraint groups based on spatiotemporal information of the centroids in the pairs of centroids.

6. The method of claim 5, wherein the sum of the weights of the set of constraints is 1.

7. The method of claim 5, wherein the spatiotemporal constraint levels comprise a temporal constraint level and a spatial constraint level, and wherein the spatiotemporal information for each centroid comprises temporal information and spatial information.

8. The method of claim 5 wherein said set of constraints having a higher level of spatiotemporal constraints are weighted more heavily than said set of constraints having a lower level of spatiotemporal constraints.

9. A computer device comprising a memory and a processor coupled to each other, the memory having stored therein program data for execution by the processor to perform the steps of the method of any one of claims 1 to 8.

10. A storage device, characterized by program data stored therein which can be executed by a processor for carrying out the steps of the method according to any one of claims 1 to 8.