CN113673550A - Clustering method, clustering device, electronic equipment and computer-readable storage medium - Google Patents
Clustering method, clustering device, electronic equipment and computer-readable storage medium Download PDFInfo
- Publication number
- CN113673550A CN113673550A CN202110736795.2A CN202110736795A CN113673550A CN 113673550 A CN113673550 A CN 113673550A CN 202110736795 A CN202110736795 A CN 202110736795A CN 113673550 A CN113673550 A CN 113673550A
- Authority
- CN
- China
- Prior art keywords
- clusters
- feature
- similarity threshold
- similarity
- threshold
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 230000004044 response Effects 0.000 claims description 32
- 238000004364 calculation method Methods 0.000 claims description 19
- 230000015654 memory Effects 0.000 claims description 19
- 238000004422 calculation algorithm Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 230000000007 visual effect Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000010606 normalization Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000003064 k means clustering Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000000047 product Substances 0.000 description 2
- 206010000117 Abnormal behaviour Diseases 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005206 flow analysis Methods 0.000 description 1
- 230000006386 memory function Effects 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
Abstract
The invention provides a clustering method, a device, electronic equipment and a computer readable storage medium, wherein the clustering method comprises the following steps: clustering the feature set in the database based on a first similarity threshold value to obtain a plurality of first clusters; merging at least two first clusters in the plurality of first clusters based on a second similarity threshold and a constraint condition; wherein the second similarity threshold is less than the first similarity threshold. Therefore, the situation that one target has multiple gears due to high threshold can be solved, and the clustering precision is improved.
Description
Technical Field
The present invention relates to the field of image clustering technologies, and in particular, to a clustering method, an apparatus, an electronic device, and a computer-readable storage medium.
Background
In the clustering process, features corresponding to the targets are generally extracted, similarity is calculated through feature matching, and the similarity is compared with a set threshold value to judge whether the features are combined into one class or one cluster. Therefore, the discrimination of the features and the setting of the threshold play an important role in the clustering result. The high threshold strategy can ensure the precision, but due to factors such as incomplete targets (truncation, object shielding and the like), the distinctiveness of parts of the same target is very poor, and the characteristic distinctiveness is not enough, so that the target characteristics of the same identity are clustered to generate a plurality of clusters, thereby causing one target to be clustered into a plurality of clustering files, and seriously affecting the clustering precision.
Disclosure of Invention
The invention provides a clustering method, a clustering device, electronic equipment and a computer readable storage medium, which are used for improving clustering precision.
In order to solve the above technical problems, a first technical solution provided by the present invention is: provided is a clustering method including: clustering the feature set in the database based on a first similarity threshold value to obtain a plurality of first clusters; merging at least two first clusters in the plurality of first clusters based on a second similarity threshold and a constraint condition; wherein the second similarity threshold is less than the first similarity threshold; the constraint is used to reduce the number of first clusters.
Wherein the constraint condition comprises a space-time constraint condition, and the step of merging at least two first clusters in the plurality of first clusters based on the second similarity threshold and the constraint condition comprises: combining at least two first clusters in the plurality of first clusters based on a second similarity threshold and a space-time constraint condition to obtain a plurality of second clusters; the spatiotemporal constraint constrains the merged first cluster using two different dimensions of time and space.
Wherein the constraint condition further includes a central feature, and after the step of combining at least two first clusters of the plurality of first clusters based on the second similarity threshold and the space-time constraint condition to obtain the plurality of second clusters, the method further includes: merging at least two second clusters in the plurality of second clusters based on a second similarity threshold and the central feature of each second cluster; wherein the central feature characterizes an average of the features in each second-class cluster.
The step of merging at least two first clusters of the plurality of first clusters based on the second similarity threshold and the space-time constraint condition includes: selecting a first characteristic from part of the first clusters in the plurality of first clusters, and selecting at least one second characteristic from the rest of the first clusters; respectively determining the distance between the capturing device corresponding to each first feature and the capturing device corresponding to each second feature, and respectively determining the capturing time of each first feature and the time difference between the capturing time of each second feature; the rest first clusters comprise first clusters except part of the first clusters in the plurality of first clusters; at least two of the plurality of first clusters are merged based on the second similarity threshold, the distance, and the time difference.
Wherein the step of merging at least two of the plurality of first clusters based on the second similarity threshold, the distance, and the time difference comprises: and in response to the distance not being greater than the distance threshold, and in response to the time difference not being greater than the time threshold, and in response to the similarity not being less than the second similarity threshold, merging the first cluster to which the first feature belongs with the first cluster to which the second feature belongs.
The clustering method further comprises the following steps: setting a second similarity threshold set, a distance threshold set and a time threshold set, wherein the time threshold in the time threshold set and the second similarity threshold in the second similarity threshold set accord with a linear relation; in response to the distance being greater than the distance threshold, adjusting the distance threshold based on the set of distance thresholds; or adjusting the time threshold based on the set of time thresholds in response to the time difference being greater than the time threshold; or in response to the similarity being less than a second similarity threshold, adjusting the second similarity threshold based on the second set of similarities.
The step of merging the corresponding at least two second clusters based on the second similarity threshold and the central feature includes: performing weighted average calculation on the features in each second cluster to obtain the central feature of each second cluster; and combining at least two second clusters by using the central feature and a second similarity threshold.
The step of merging at least two second clusters by using the central feature and the second similarity threshold includes: respectively calculating the similarity of the central features of every two second clusters; and in response to the similarity not being smaller than the second similarity threshold, merging the second clusters corresponding to the central features.
Wherein the constraint condition includes a central feature, and the step of merging at least two first clusters of the plurality of first clusters based on the second similarity threshold and the constraint condition includes: merging at least two first clusters in the plurality of first clusters based on the second similarity threshold and the central feature of each first cluster; wherein the central features characterize an average of the features in each first-type cluster.
The step of merging at least two first clusters of the plurality of first clusters based on the second similarity threshold and the central feature includes: performing weighted average calculation on the features in each first cluster to obtain the central feature of each first cluster; and combining at least two first clusters in the plurality of first clusters by using the central feature and the second similarity threshold.
The step of combining at least two first clusters of the plurality of first clusters by using the central feature and the second similarity threshold includes: respectively calculating the similarity of the central features of every two first clusters; and in response to the similarity not smaller than the second similarity threshold, merging the first clusters corresponding to the central features.
Wherein the first similarity threshold is greater than 90%.
Wherein the feature set comprises features corresponding to the image, or the feature set is determined based on the features of the image.
In order to solve the above technical problems, a second technical solution provided by the present invention is: provided is a clustering device including: the clustering module is used for clustering the feature set in the database based on a first similarity threshold value so as to obtain a plurality of first clusters; a merging module, configured to merge at least two first clusters in the plurality of first clusters based on the second similarity threshold and the constraint condition; wherein the second similarity threshold is less than the first similarity threshold; the constraint is used to reduce the number of first clusters.
In order to solve the above technical problems, a third technical solution provided by the present invention is: provided is an electronic device including: a memory storing program instructions and a processor retrieving the program instructions from the memory to perform any of the above methods.
In order to solve the above technical problems, a fourth technical solution provided by the present invention is: there is provided a computer readable storage medium having stored thereon a program file executable to implement the method of any of the above.
The method has the beneficial effects that the method is different from the prior art, and the characteristic set in the database is clustered based on the first similarity threshold value, so that a plurality of first clusters are obtained; and merging at least two first clusters in the plurality of first clusters based on a second similarity threshold smaller than the first similarity threshold and the constraint condition. Therefore, the situation of one target and multiple gears during high-threshold clustering can be solved, and the clustering precision is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without inventive efforts, wherein:
FIG. 1 is a schematic flow chart of a clustering method according to a first embodiment of the present invention;
FIG. 2 is a schematic flowchart of a first embodiment of step S12 in FIG. 1;
FIG. 3 is a flowchart illustrating a second embodiment of step S12 in FIG. 1;
FIG. 4 is a schematic structural diagram of a clustering device according to an embodiment of the present invention;
FIG. 5 is a schematic structural diagram of an electronic device according to an embodiment of the invention;
FIG. 6 is a schematic structural diagram of an embodiment of a computer-readable storage medium according to the present invention.
Detailed Description
In recent years, with the development of artificial intelligence technology, a large amount of video image data can be generated in the fields of intelligent security, new retail, internet and the like, and the intelligent data analysis of video image content has important application value. Compared with the traditional video image retrieval technology, the method has the advantages that archives can be built for pedestrians and vehicles through the image clustering technology, for example, images and motion data features of personnel with the same identity are gathered into clusters, archives indexes are built, a personnel archive database is built, the higher-order archive retrieval function can be achieved, and therefore more intelligent product experience is provided for users. In addition, the image clustering technology plays an important role in the aspects of space-time behavior trajectory analysis, user portrait analysis, abnormal behavior detection, criminal distribution control, urban traffic flow analysis and the like, and has wide application prospect.
In large-scale video data analysis, conventional clustering algorithms (such as K-means clustering, spectral clustering, density clustering and the like) cannot meet actual business requirements. Especially when human body data (such as walking personnel and riding personnel) are clustered, the change of the visual angle and the posture makes the traditional clustering method difficult to obtain better clustering precision. The clustering algorithm based on the human face features can realize landing and filing of the identity of the pedestrian, effective human face feature extraction usually requires a camera to capture a clear image of the front of the pedestrian at a short distance, but in a natural monitoring scene, the visual angle and the posture of the pedestrian are not controlled, and the behavior of the pedestrian such as head lowering or mask wearing can influence the human face clustering effect. Therefore, the human body apparent characteristics of the pedestrians are extracted to carry out human body clustering, and the human body apparent characteristics can be used as effective supplement of face clustering, so that the clustering recall rate and the identity landing rate in a natural monitoring state are improved.
In the human body clustering process, human body image features are generally extracted, similarity is calculated through feature matching and compared with a set threshold value to judge whether the human body image features are combined into one class or one cluster. Therefore, the discriminativity and the threshold setting of the features play a crucial role in clustering results, the high-threshold strategy can guarantee the precision, the human body features are very poor in discriminativity compared with human face features due to the factors of incomplete human body images (truncation, object shielding and the like), resolution ratio of snap images, visual angle difference and the like, the human body features are often caused to generate multiple clusters due to insufficient feature discriminativity, so that multiple grades are caused, the low-threshold strategy can enable the clustering to generate a large wrong clustering risk, and therefore the technical problem to be solved urgently is that how to design an algorithm improves the clustering recall rate while guaranteeing the clustering precision. The invention provides a clustering method which can improve clustering recall rate and clustering precision. The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1, a schematic flow chart of a clustering method according to a first embodiment of the present invention specifically includes:
step S11: clustering the feature set in the database based on the first similarity threshold value to obtain a plurality of first clusters.
Specifically, a first similarity threshold is used to cluster feature sets in the database, where the feature sets may be feature sets corresponding to images in the database or feature sets corresponding to characters in the database, and in another embodiment, the feature sets may also be determined based on features of the images, and are not specifically limited.
In one embodiment, the image data is taken as an example, and the K-means clustering algorithm is taken as an example for explanation. Specifically, a feature extraction algorithm, such as a convolutional neural network algorithm, is used for extracting features of the images in the database, so as to obtain a feature set. And dividing the feature set into K groups, randomly selecting K objects as initial clustering centers, calculating the distance between each object and each clustering center, namely the similarity, comparing the similarity with a first similarity threshold, and allocating each object with the similarity larger than the first similarity threshold to the corresponding clustering center to further obtain a plurality of first clusters.
Step S12: and merging at least two first clusters in the plurality of first clusters based on the second similarity threshold and the constraint condition.
Specifically, when the first similarity threshold is large, although the clustering accuracy can be ensured, the human body features are much less distinguishable than the human face features due to factors such as the target image incompleteness (truncation, object occlusion, and the like). For example, if the snap-shot image of a person is incomplete, the resolution of the snap-shot image is different, or there are factors such as visual angle differences, the characteristic distinctiveness is not enough to cause a plurality of clusters to be generated on the human body characteristics of the same identity, thereby causing one person to have a plurality of files. Therefore, the first clusters are merged by using the second similarity threshold and the constraint condition, so that multiple gears corresponding to the same person can be merged, the clustering precision can be guaranteed, and the problem of multiple gears can be solved. In a specific embodiment, the second similarity threshold is less than the first similarity threshold.
In the prior art, only the second similarity threshold is introduced for merging the first clusters, but the merging is inaccurate or insufficient.
In one embodiment, the constraint is a spatio-temporal constraint that constrains the merged first cluster using two different dimensions of time and space. For example, at the same time, the same target appears in different spaces, which is unlikely to happen; alternatively, it is not reasonable to have the same target appear in a distant space in a close time. Based on the idea, the first clusters are combined with the second similarity threshold value to obtain a plurality of second clusters.
Specifically, in a specific embodiment, at least two first clusters in the plurality of first clusters are merged based on the second similarity threshold and the space-time constraint condition, so as to obtain a plurality of second clusters. Specifically, referring to fig. 2, fig. 2 is a schematic flowchart of the first embodiment of step S12, which specifically includes:
step S21: selecting a first feature from a part of the first clusters in the plurality of first clusters, and selecting at least one second feature from the rest of the first clusters; respectively determining the distance between the capturing device corresponding to each first feature and the capturing device corresponding to each second feature, and respectively determining the capturing time of each first feature and the time difference between the capturing time of each second feature; the remaining first clusters include first clusters of the plurality of first clusters other than a portion of the first clusters.
In a specific embodiment, the first feature is selected from a part of the first clusters in the plurality of first clusters, and at least one second feature is selected from the rest of the first clusters, where the rest of the first clusters are the first clusters except the part of the first clusters in the plurality of first clusters. Determining the distance between the capturing device corresponding to each first feature and the capturing device corresponding to each second feature, and determining the time difference between the capturing time of each first feature and the capturing time of each second feature. Wherein, the distance of the snapshot device is the GPS position of the snapshot device.
For example, the first clusters include a first cluster a, a first cluster B, and a first cluster C, where the first cluster a, the first cluster B, and the first cluster C each include a plurality of images. An image is selected from the first cluster a as the first feature, for example, the image with the highest quality score in the first cluster a may be selected as the first feature. At least one second feature may be selected from each of the first cluster B and the first cluster C, and similarly, an image with a higher quality score in the first cluster B and the first cluster C may be selected as the second feature. Determining the distance between a snapshot device corresponding to a first feature in a first cluster A and a snapshot device corresponding to a second feature in a first cluster B, determining the snapshot time of the first feature in the first cluster A and the time difference between the snapshot times of the second feature of the first cluster B, determining the distance between the snapshot device corresponding to the first feature in the first cluster A and the snapshot device corresponding to the second feature in the first cluster C, and determining the time difference between the snapshot time of the first feature in the first cluster A and the snapshot time of the second feature in the first cluster C.
Step S22: at least two of the plurality of first clusters are merged based on the second similarity threshold, the distance, and the time difference.
Specifically, the distance is compared with a distance threshold, the time difference is compared with a time threshold, and the similarity between the first feature and the second feature is compared with a second similarity threshold. And in response to the distance not being greater than the distance threshold, in response to the time difference not being greater than the time threshold, and in response to the similarity not being less than the second similarity threshold, merging the corresponding first cluster of the first feature and the first cluster to which the second feature belongs.
In a specific embodiment, a first feature may be selected from the first cluster a, at least one second feature may be selected from the first cluster B and the first cluster C, a distance between the first feature and the capturing device corresponding to the second feature is determined, a time difference between capturing times corresponding to the first feature and the second feature is determined, a similarity between the first feature and the second feature is calculated, and if the distance between one of the second features and the first feature is not greater than a distance threshold, the time difference is not greater than a time threshold, and the similarity is not less than a second similarity threshold, the first cluster B or C corresponding to the second feature is merged with the first cluster a corresponding to the first feature. For example, if a second feature in the first cluster B and a first feature in the first cluster a satisfy the following condition:
wherein d isn-kThe distance, t, between the capturing device representing the first feature n and the capturing device representing the second feature kn-tkRepresents the time difference between the capturing time of the first feature n and the capturing time of the second feature k, sim (n, k) represents the similarity of the first feature n to the second feature k, Δ d represents the distance threshold, Δ t represents the time threshold, and s1 represents the second similarity threshold.
If a second feature in the first cluster B and a first feature in the first cluster A satisfy the condition of formula (1), the first cluster A and the first cluster B are merged.
In another embodiment, to ensure the merging accuracy, the number of the second features may be further increased, for example, if a plurality of second features in the first cluster B and one first feature in the first cluster a satisfy the condition of formula (1), i.e., the first cluster a and the first cluster B are merged.
Specifically, the distance threshold and the time threshold are set to reduce the influence of interference data, that is, under the constraint condition of the distance threshold and the time difference, the difference between different human bodies is large, and further, the second similarity threshold is set to merge clusters with large similarity under the constraint condition of the distance threshold and the time difference.
In a specific embodiment, one first type cluster is merged at most once. Specifically, when the first cluster B is merged into the first cluster a, the feature data in the first cluster B is not compared again as the first feature in the subsequent merging judgment process of the clusters. Thereby, the clustering accuracy can be further improved.
In another embodiment, the second set of similarity thresholds, the set of distance thresholds, and the set of time thresholds are set, wherein a time threshold of the set of time thresholds and a second similarity threshold of the set of second similarity thresholds conform to a linear relationship. In response to the distance being greater than the distance threshold, adjusting the distance threshold based on the set of distance thresholds; or adjusting the time threshold based on the set of time thresholds in response to the time difference being greater than the time threshold; or in response to the similarity being less than a second similarity threshold, adjusting the second similarity threshold based on the second set of similarities.
In particular, an adaptive second similarity threshold may be set for different first features from strong to weak according to the spatiotemporal constraint. Specifically, the distance threshold or the time threshold is adjustable. For example, the set distance threshold Δ d is 500m, and the set time threshold Δ t is adjustable. It can be understood that the larger the time threshold Δ t is, the weaker the spatio-temporal constraint condition is, and in order to ensure the clustering accuracy, a higher second similarity threshold needs to be set. I.e. the time threshold is in positive correlation with the second similarity threshold.
In a specific embodiment, in the case that the distance threshold Δ d is set to be a fixed value, for example, Δ d is 500m, a discrete set of second similarity thresholds S1 may be set (84,85,86), and the corresponding time threshold is calculated according to the linear calculation formula (2):
wherein, the value range of the second similarity threshold S1 is [ S ]min,Smax]The value range [ Delta t ] of the time threshold Delta tmin,Δtmax]。
For example, the set of calculated time thresholds Δ t is Δ t ═ 150,200,300. The second similarity threshold s1 and the time threshold Δ t may be traversed from large to small, that is, s1 ═ 86 and Δ t ═ 300, at which time, if there is no second feature matching the first feature, that is, if all the second features do not satisfy formula (1), s1 ═ 85 and Δ t ═ 200 are tried until the second feature satisfying formula (1) is matched. If the second feature satisfying the above formula (1) is not matched in both s1 ═ 84 and Δ t ═ 150, it is determined that there is no first cluster that can be merged.
In an embodiment, the first similarity threshold is greater than 90%.
By the space-time constraint and the relaxation threshold (namely, the second similarity threshold is smaller than the first similarity threshold) of the embodiment, the efficiency and the precision of the combination of the clusters and the clusters can be effectively improved, and the probability of the independent clustering of difficult cases is reduced. In addition, in practical application, based on space-time constraint and a relaxation threshold strategy, the space-time relationship of adjacent time of adjacent cameras can be modeled, and cross-camera relay tracking is realized through human body feature comparison under the condition of reducing interference.
In another embodiment, the constraint includes a central feature. For example, in one embodiment, at least two first clusters are merged based on the second similarity threshold and the center feature of each first cluster. Specifically, referring to fig. 3, fig. 3 is a flowchart illustrating a second embodiment of step S12, which specifically includes:
step S31: and performing weighted average calculation on the features in each first-class cluster to obtain the central feature of each first-class cluster.
Specifically, the weighted average calculation may be performed on the features in each first-type cluster, so as to obtain the central feature of each first-type cluster. In one embodiment, the central feature of each first cluster may be obtained by performing a weighted average calculation on the features in each first cluster based on the quality score of each image in the first cluster. In another embodiment, the images in the first clusters may be screened to select an image with a better quality score, and the central feature of each first cluster is obtained by performing weighted average calculation on the features in each first cluster by using the quality score corresponding to the image.
Further, in an embodiment, the weighted average calculation is performed on the features in each first-class cluster, and the normalization processing is performed on the result of the weighted average calculation, so as to obtain the central feature of each first-class cluster. The weighted average calculation method of the features in each first cluster is as follows:
f＝(q1f1+q2f2+…+qkfk)/K；
wherein q iskIs the quality score of the kth feature, fkIs the kth feature.
And carrying out normalization processing on the result after weighted average calculation so as to obtain the central characteristics:
step S32: and combining at least two first clusters in the plurality of first clusters by using the central feature and the second similarity threshold.
Specifically, the similarity of the central features of the two first clusters is calculated, the similarity is compared with a second similarity threshold, and in response to the similarity not being smaller than the second similarity threshold, the first clusters corresponding to the central features are merged. For example, if the similarity between the central feature of the first cluster a and the central feature of the first cluster B is not less than the second similarity threshold, the first cluster a and the first cluster B are merged.
In this embodiment, the central features of the first-class clusters are used to merge clusters, so that the distinction between clusters can be improved, the calculation amount is reduced, and the clustering accuracy is improved.
In another embodiment, the constraints include spatio-temporal constraints and central feature constraints. That is, in a specific embodiment, at least two first clusters in the plurality of first clusters are merged based on the second similarity threshold and the space-time constraint condition to obtain a plurality of second clusters; and then merging at least two second-class clusters in the plurality of second-class clusters based on the second similarity threshold and the central feature of each second-class cluster.
In this embodiment, a manner of combining at least two first clusters in the plurality of first clusters based on the second similarity threshold and the space-time constraint condition to obtain the plurality of second clusters is the same as that in the above-described embodiment, and is not described herein again. Merging at least two second clusters of the plurality of second clusters based on the second similarity threshold and the central feature of each second cluster specifically includes: and performing weighted average calculation on the features in each second cluster to further obtain the central feature of each second cluster, wherein the weighted average calculation mode is as follows:
f＝(q1f1+q2f2+…+qkfk)/K；
wherein q iskIs the quality score of the kth feature, fkIs the kth feature.
And carrying out normalization processing on the result after weighted average calculation, and further obtaining the center characteristic of the second cluster:
and calculating the similarity of the central features of the two second clusters, comparing the similarity with a second similarity threshold, and combining the second clusters corresponding to the central features in response to the similarity not less than the second similarity threshold.
In this embodiment, through the space-time constraint and the relaxation threshold (that is, the second similarity threshold is smaller than the first similarity threshold), the efficiency and the accuracy of merging the clusters and the clusters can be effectively improved, and the probability of difficult cases forming clusters alone is reduced. In addition, in practical application, based on space-time constraint and a relaxation threshold strategy, the space-time relationship of adjacent time of adjacent cameras can be modeled, and cross-camera relay tracking is realized through human body feature comparison under the condition of reducing interference. In addition, under the space-time constraint and relaxation threshold strategy, the second cluster already contains abundant posture and visual characteristic data, and the central characteristic of the second cluster is utilized to merge clusters, so that the cluster-to-cluster merging in a space-time domain can be increased, the distinguishing performance between the clusters is further improved, the calculated amount is reduced, and the clustering precision is improved. The clustering method can improve the target clustering capability in a complex environment, enhance the convergence capability to difficult cases and improve the clustering recall rate.
Referring to fig. 4, a schematic structural diagram of a clustering apparatus according to an embodiment of the present invention is shown, the clustering apparatus includes: a clustering module 51 and a merging module 52.
The clustering module 51 is configured to cluster the feature sets in the database based on a first similarity threshold, so as to obtain a plurality of first clusters.
The merging module 52 is configured to merge at least two first clusters in the plurality of first clusters based on a second similarity threshold and a constraint condition, where the second similarity threshold is smaller than the first similarity threshold; the constraint is used to reduce the number of first clusters.
In an embodiment, the merging module 52 is configured to merge at least two first clusters of the plurality of first clusters based on the second similarity threshold and the space-time constraint condition to obtain a plurality of second clusters; the spatiotemporal constraint constrains the merged first cluster using two different dimensions of time and space.
In an embodiment, the merging module 52 is configured to merge at least two first clusters of the plurality of first clusters based on the second similarity threshold and the space-time constraint condition to obtain a plurality of second clusters; the spatiotemporal constraint constrains the merged first cluster using two different dimensions of time and space. Merging at least two second clusters in the plurality of second clusters based on a second similarity threshold and the central feature of each second cluster; wherein the central features characterize the average features of the features in each second-class cluster.
Specifically, the merging module 52 selects a first feature from a part of the first clusters in the plurality of first clusters, and selects at least one second feature from each of the rest of the first clusters; respectively determining the distance between the capturing device corresponding to each first feature and the capturing device corresponding to each second feature, and respectively determining the capturing time of each first feature and the time difference between the capturing time of each second feature; the rest first clusters comprise first clusters except part of the first clusters in the plurality of first clusters; at least two of the plurality of first clusters are merged based on the second similarity threshold, the distance, and the time difference.
Specifically, in response to the distance not being greater than the distance threshold, in response to the time difference not being greater than the time threshold, and in response to the similarity not being less than the second similarity threshold, the merging module 52 merges the first cluster to which the first feature belongs with the first cluster to which the second feature belongs.
In an embodiment, the merging module 52 may further set a second similarity threshold set, a distance threshold set, and a time threshold set, wherein a time threshold of the time threshold set and a second similarity threshold of the second similarity threshold set conform to a linear relationship; in response to the distance being greater than the distance threshold, adjusting the distance threshold based on the set of distance thresholds; or adjusting the time threshold based on the set of time thresholds in response to the time difference being greater than the time threshold; or in response to the similarity being less than a second similarity threshold, adjusting the second similarity threshold based on the second set of similarities.
In an embodiment, the merging module 52 may further perform weighted average calculation on the features in each second-type cluster, so as to obtain a central feature of each second-type cluster; and merging at least two second clusters by using the central feature and a second similarity threshold value.
Specifically, the merging module 52 calculates the similarity of the central features of every two second clusters; and in response to the similarity not smaller than the second similarity threshold, merging the second clusters corresponding to the central features.
In another embodiment, the merging module 52 may further merge at least two first clusters in the plurality of first clusters based on the second similarity threshold and the central feature of each first cluster; wherein the central features characterize an average of the features in each first-type cluster.
Specifically, the merging module 52 performs weighted average calculation on the features in each first cluster, so as to obtain the central feature of each first cluster; and combining at least two first clusters in the plurality of first clusters by using the central feature and the second similarity threshold.
Specifically, the merging module 52 calculates the similarity of the central features of each two first clusters; and in response to the similarity not smaller than the second similarity threshold, merging the first clusters corresponding to the central features.
The clustering device of the embodiment can effectively improve the efficiency and the accuracy of cluster and cluster combination and reduce the probability of difficult independent clustering through space-time constraint and a relaxation threshold (namely, the second similarity threshold is smaller than the first similarity threshold). In addition, in practical application, based on space-time constraint and a relaxation threshold strategy, the space-time relationship of adjacent time of adjacent cameras can be modeled, and cross-camera relay tracking is realized through human body feature comparison under the condition of reducing interference. In addition, under the space-time constraint and relaxation threshold strategy, the second cluster already contains abundant attitude and visual characteristic data, and the central characteristic of the second cluster is utilized to merge clusters, so that the cluster-to-cluster merging in a space-time domain can be increased, the distinguishing performance between the clusters is further improved, the calculated amount is reduced, and the clustering precision is improved. The clustering method can improve the target clustering capability in a complex environment, enhance the convergence capability to difficult cases and improve the clustering recall rate.
Referring to fig. 5, a schematic structural diagram of an electronic device according to an embodiment of the present invention is shown, where the electronic device includes a memory 202 and a processor 201 that are connected to each other.
The memory 202 is used to store program instructions implementing the method of any of the above.
The processor 201 is used to execute program instructions stored by the memory 202.
The processor 201 may also be referred to as a Central Processing Unit (CPU). The processor 201 may be an integrated circuit chip having signal processing capabilities. The processor 201 may also be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The storage 202 may be a memory bank, a TF card, etc., and may store all information in the electronic device of the device, including input raw data, computer programs, intermediate operation results, and final operation results. It stores and retrieves information based on the location specified by the controller. With the memory, the electronic device can only have the memory function to ensure the normal operation. The memories of electronic devices are classified into a main memory (internal memory) and an auxiliary memory (external memory) according to their purposes, and also into an external memory and an internal memory. The external memory is usually a magnetic medium, an optical disk, or the like, and can store information for a long period of time. The memory refers to a storage component on the main board, which is used for storing data and programs currently being executed, but is only used for temporarily storing the programs and the data, and the data is lost when the power is turned off or the power is cut off.
In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a module or a unit may be divided into only one logical function, and may be implemented in other ways, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a system server, a network device, or the like) or a processor (processor) to execute all or part of the steps of the method according to the embodiments of the present application.
Please refer to fig. 6, which is a schematic structural diagram of a computer-readable storage medium according to the present invention. The storage medium of the present application stores a program file 203 capable of implementing all the methods described above, wherein the program file 203 may be stored in the storage medium in the form of a software product, and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute all or part of the steps of the methods of the embodiments of the present application. The aforementioned storage device includes: various media capable of storing program codes, such as a usb disk, a mobile hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, or terminal devices, such as a computer, a server, a mobile phone, and a tablet.
The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all equivalent structures or equivalent flow transformations that are made by using the contents of the present specification and the drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (16)
1. A method of clustering, the method comprising:
clustering the feature set in the database based on a first similarity threshold value to obtain a plurality of first clusters;
merging at least two of the first clusters in the plurality of first clusters based on a second similarity threshold and a constraint condition;
wherein the second similarity threshold is less than the first similarity threshold; the constraint is for reducing the number of the first cluster.
2. The method of claim 1, wherein the constraint comprises a spatio-temporal constraint, and wherein the step of merging at least two of the first clusters of the plurality of first clusters based on a second similarity threshold and a constraint comprises:
combining at least two first clusters in the plurality of first clusters based on the second similarity threshold and the space-time constraint condition to obtain a plurality of second clusters; the spatiotemporal constraint constrains the merged first cluster using two different dimensions of time and space.
3. The method according to claim 2, wherein the constraint further includes a central feature, and wherein the step of combining at least two of the first clusters of the plurality of first clusters to obtain a plurality of second clusters based on the second similarity threshold and the spatio-temporal constraint further comprises:
merging at least two of the second clusters in the plurality of second clusters based on the second similarity threshold and the center feature of each of the second clusters; wherein the central feature characterizes an average feature of the features in each of the second-class clusters.
4. The method according to claim 2 or 3, wherein said step of combining at least two of said first clusters of said plurality of first clusters based on said second similarity threshold and said spatio-temporal constraint comprises:
selecting first features from a part of the first clusters in the plurality of first clusters, and selecting at least one second feature from the rest of the first clusters; respectively determining the distance between the capturing device corresponding to each first feature and the capturing device corresponding to each second feature, and respectively determining the capturing time of each first feature and the time difference between the capturing time of each second feature; the remaining first clusters include first clusters of the plurality of first clusters other than the portion of the first clusters;
merging at least two of the first clusters of the plurality of first clusters based on the second similarity threshold, the distance, and the time difference.
5. The method of claim 4, wherein the step of combining at least two of the first clusters of the plurality of first clusters based on the second similarity threshold, the distance, and the time difference comprises:
in response to the distance not being greater than the distance threshold, and in response to the time difference not being greater than the time threshold, and in response to the similarity not being less than the second similarity threshold, merging a first cluster to which the first feature belongs with a first cluster to which the second feature belongs.
6. The method of claim 5, further comprising:
setting a second set of similarity thresholds, a set of distance thresholds, and a set of time thresholds, wherein the time thresholds of the set of time thresholds and the second similarity thresholds of the second set of similarity thresholds conform to a linear relationship;
in response to the distance being greater than the distance threshold, adjusting the distance threshold based on the set of distance thresholds; or
In response to the time difference being greater than the time threshold, adjusting the time threshold based on the set of time thresholds; or
In response to the similarity being less than the second similarity threshold, adjusting the second similarity threshold based on the second set of similarities.
7. The method according to claim 3, wherein the step of merging the corresponding at least two second clusters based on the second similarity threshold and the central feature comprises:
performing weighted average calculation on the features in each second cluster to obtain the central feature of each second cluster;
and merging at least two second clusters by using the central feature and the second similarity threshold.
8. The method of claim 7, wherein the step of merging at least two clusters of the second type using the center feature and the second similarity threshold comprises:
respectively calculating the similarity of the central features of every two second clusters;
and in response to the similarity not being smaller than the second similarity threshold, merging the second clusters corresponding to the central features.
9. The method of claim 1, wherein the constraint comprises a central feature, and wherein the step of merging at least two of the first clusters of the plurality of first clusters based on the second similarity threshold and the constraint comprises:
merging at least two of the first clusters in the plurality of first clusters based on the second similarity threshold and the central feature of each of the first clusters; wherein the central feature characterizes an average feature of the features in each of the first clusters.
10. The method of claim 9, wherein said step of merging at least two of said first clusters of said plurality of first clusters based on said second similarity threshold and said center feature comprises:
performing weighted average calculation on the features in each first cluster to obtain the central feature of each first cluster;
and combining at least two first clusters in the plurality of first clusters by using the central feature and the second similarity threshold.
11. The method of claim 10, wherein said step of merging at least two of said first clusters of said plurality of first clusters using said central feature and said second similarity threshold comprises:
respectively calculating the similarity of the central features of every two first clusters;
and in response to the similarity not being smaller than the second similarity threshold, merging the first clusters corresponding to the central features.
12. The method of claim 1, wherein the first similarity threshold is greater than 90%.
13. The method of claim 1, wherein the feature set comprises features corresponding to the image, or wherein the feature set is determined based on features of the image.
14. A clustering apparatus, the apparatus comprising:
the clustering module is used for clustering the feature set in the database based on a first similarity threshold value so as to obtain a plurality of first clusters;
a merging module, configured to merge at least two of the first clusters in the plurality of first clusters based on a second similarity threshold and a constraint condition;
wherein the second similarity threshold is less than the first similarity threshold; the constraint is for reducing the number of the first cluster.
15. An electronic device, comprising: a memory storing program instructions and a processor retrieving the program instructions from the memory to perform the clustering method according to any one of claims 1 to 13.
16. A computer-readable storage medium, characterized in that a program file is stored, which can be executed to implement the clustering method according to any one of claims 1 to 13.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110736795.2A CN113673550A (en) | 2021-06-30 | 2021-06-30 | Clustering method, clustering device, electronic equipment and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110736795.2A CN113673550A (en) | 2021-06-30 | 2021-06-30 | Clustering method, clustering device, electronic equipment and computer-readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113673550A true CN113673550A (en) | 2021-11-19 |
Family
ID=78538392
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110736795.2A Pending CN113673550A (en) | 2021-06-30 | 2021-06-30 | Clustering method, clustering device, electronic equipment and computer-readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113673550A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115795318A (en) * | 2022-11-17 | 2023-03-14 | 北京白龙马云行科技有限公司 | Classification method of use case object, model training method, equipment and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109145844A (en) * | 2018-08-29 | 2019-01-04 | 北京旷视科技有限公司 | Archive management method, device and electronic equipment for city safety monitoring |
| CN109815788A (en) * | 2018-12-11 | 2019-05-28 | 平安科技（深圳）有限公司 | A kind of picture clustering method, device, storage medium and terminal device |
| CN110888981A (en) * | 2019-10-30 | 2020-03-17 | 深圳价值在线信息科技股份有限公司 | Title-based document clustering method and device, terminal equipment and medium |
| CN110969215A (en) * | 2019-12-18 | 2020-04-07 | 浙江大华技术股份有限公司 | Clustering method and device, storage medium and electronic device |
| CN111401300A (en) * | 2020-04-07 | 2020-07-10 | 浙江大华技术股份有限公司 | Face clustering archiving method and device and storage medium |
| CN111738221A (en) * | 2020-07-28 | 2020-10-02 | 腾讯科技（深圳）有限公司 | Face clustering method, face clustering device and computer-readable storage medium |
| CN112232148A (en) * | 2020-09-28 | 2021-01-15 | 浙江大华技术股份有限公司 | Image clustering method, target track tracking method, electronic device and storage medium |
| CN112818149A (en) * | 2021-01-21 | 2021-05-18 | 浙江大华技术股份有限公司 | Face clustering method and device based on space-time trajectory data and storage medium |
| CN113052245A (en) * | 2021-03-30 | 2021-06-29 | 重庆紫光华山智安科技有限公司 | Image clustering method and device, electronic equipment and storage medium |
-
2021
- 2021-06-30 CN CN202110736795.2A patent/CN113673550A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109145844A (en) * | 2018-08-29 | 2019-01-04 | 北京旷视科技有限公司 | Archive management method, device and electronic equipment for city safety monitoring |
| CN109815788A (en) * | 2018-12-11 | 2019-05-28 | 平安科技（深圳）有限公司 | A kind of picture clustering method, device, storage medium and terminal device |
| CN110888981A (en) * | 2019-10-30 | 2020-03-17 | 深圳价值在线信息科技股份有限公司 | Title-based document clustering method and device, terminal equipment and medium |
| CN110969215A (en) * | 2019-12-18 | 2020-04-07 | 浙江大华技术股份有限公司 | Clustering method and device, storage medium and electronic device |
| CN111401300A (en) * | 2020-04-07 | 2020-07-10 | 浙江大华技术股份有限公司 | Face clustering archiving method and device and storage medium |
| CN111738221A (en) * | 2020-07-28 | 2020-10-02 | 腾讯科技（深圳）有限公司 | Face clustering method, face clustering device and computer-readable storage medium |
| CN112232148A (en) * | 2020-09-28 | 2021-01-15 | 浙江大华技术股份有限公司 | Image clustering method, target track tracking method, electronic device and storage medium |
| CN112818149A (en) * | 2021-01-21 | 2021-05-18 | 浙江大华技术股份有限公司 | Face clustering method and device based on space-time trajectory data and storage medium |
| CN113052245A (en) * | 2021-03-30 | 2021-06-29 | 重庆紫光华山智安科技有限公司 | Image clustering method and device, electronic equipment and storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115795318A (en) * | 2022-11-17 | 2023-03-14 | 北京白龙马云行科技有限公司 | Classification method of use case object, model training method, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhang et al. | Improving triplet-wise training of convolutional neural network for vehicle re-identification | |
| US10402627B2 (en) | Method and apparatus for determining identity identifier of face in face image, and terminal | |
| CN110516586B (en) | Face image clustering method, system, product and medium | |
| Zhang et al. | MoWLD: a robust motion image descriptor for violence detection | |
| CN103207898B (en) | A kind of similar face method for quickly retrieving based on local sensitivity Hash | |
| KR101711677B1 (en) | Clustering videos by location | |
| US20050129276A1 (en) | Object detection | |
| GB2409028A (en) | Face detection | |
| CN103929685A (en) | Video abstract generating and indexing method | |
| Ming et al. | Activity recognition from RGB-D camera with 3D local spatio-temporal features | |
| CN107909044B (en) | People counting method combining convolutional neural network and track prediction | |
| CN110781733B (en) | Image duplicate removal method, storage medium, network equipment and intelligent monitoring system | |
| CN112232148A (en) | Image clustering method, target track tracking method, electronic device and storage medium | |
| CN107944381B (en) | Face tracking method, face tracking device, terminal and storage medium | |
| CN108550132B (en) | Image collaborative salient target detection method | |
| Zhao et al. | Key-frame extraction based on HSV histogram and adaptive clustering | |
| Gao et al. | Key-frame selection for video summarization: An approach of multidimensional time series analysis | |
| Yang et al. | A method of pedestrians counting based on deep learning | |
| Foucher et al. | Automatic detection and clustering of actor faces based on spectral clustering techniques | |
| CN113673550A (en) | Clustering method, clustering device, electronic equipment and computer-readable storage medium | |
| CN110807790A (en) | Image data extraction and compression method for video target trajectory tracking content | |
| Chen et al. | Cholesky decomposition-based metric learning for video-based human action recognition | |
| CN114639143B (en) | Portrait archiving method, device and storage medium based on artificial intelligence | |
| CN116258748A (en) | Track tracking method | |
| CN112257666B (en) | Target image content aggregation method, device, equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |