CN113052083A - Action behavior segmentation method for multi-neighbor graph constraint matrix decomposition - Google Patents
Action behavior segmentation method for multi-neighbor graph constraint matrix decomposition Download PDFInfo
- Publication number
- CN113052083A CN113052083A CN202110326354.5A CN202110326354A CN113052083A CN 113052083 A CN113052083 A CN 113052083A CN 202110326354 A CN202110326354 A CN 202110326354A CN 113052083 A CN113052083 A CN 113052083A
- Authority
- CN
- China
- Prior art keywords
- sequence
- constraint
- graph
- matrix
- similarity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000011159 matrix material Substances 0.000 title claims abstract description 64
- 238000000034 method Methods 0.000 title claims abstract description 53
- 230000011218 segmentation Effects 0.000 title claims abstract description 46
- 238000000354 decomposition reaction Methods 0.000 title claims abstract description 26
- 230000006399 behavior Effects 0.000 claims abstract description 71
- 230000033001 locomotion Effects 0.000 claims abstract description 24
- 238000005457 optimization Methods 0.000 claims description 20
- 238000012545 processing Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000003595 spectral effect Effects 0.000 claims description 4
- 239000013598 vector Substances 0.000 claims description 4
- 239000000126 substance Substances 0.000 claims description 3
- 238000005520 cutting process Methods 0.000 claims description 2
- 238000005259 measurement Methods 0.000 description 7
- 238000004458 analytical method Methods 0.000 description 4
- 238000013461 design Methods 0.000 description 4
- 238000000513 principal component analysis Methods 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- 230000002123 temporal effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000007781 pre-processing Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000003909 pattern recognition Methods 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- BQIMPGFMMOZASS-CLZZGJSISA-N (6r,7r)-7-amino-3-(hydroxymethyl)-8-oxo-5-thia-1-azabicyclo[4.2.0]oct-2-ene-2-carboxylic acid Chemical compound S1CC(CO)=C(C(O)=O)N2C(=O)[C@@H](N)[C@H]21 BQIMPGFMMOZASS-CLZZGJSISA-N 0.000 description 1
- 102100028524 Lysosomal protective protein Human genes 0.000 description 1
- 101710162021 Lysosomal protective protein Proteins 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000003542 behavioural effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001427 coherent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 230000010076 replication Effects 0.000 description 1
- 238000012916 structural analysis Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/215—Motion-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Mathematical Analysis (AREA)
- Computational Mathematics (AREA)
- Mathematical Optimization (AREA)
- General Engineering & Computer Science (AREA)
- Pure & Applied Mathematics (AREA)
- Evolutionary Biology (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Algebra (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of human behavior segmentation, and discloses an action behavior segmentation method for multi-neighbor graph constraint matrix decomposition, which comprises the following steps of: s1, designing a constraint regular term of the multi-graph constraint structure through the behavior action sequence data; s2, constructing a regular term constraint semi-NMF matrix decomposition model; s3, solving the semi-NMF matrix decomposition model to obtain the low-dimensional representation of the behavior sequence; s4, calculating sequence point similarity weight by using low-dimensional representation to generate a similarity relation matrix; and S5, obtaining behavior segmentation by using a graph segmentation method according to the similarity relation matrix, wherein the behavior segmentation method has few phenomena of wrong segmentation in a motion sequence and can accurately distinguish similar motions in human body behaviors.
Description
Technical Field
The invention relates to the technical field of human behavior segmentation, in particular to an action behavior segmentation method based on multi-neighbor graph constraint matrix decomposition.
Background
Human behavior and action recognition is a research direction crossed by multiple subjects, relates to multiple subjects such as image processing, computer vision, pattern recognition, machine learning, artificial intelligence and the like, and is an important research subject in the field of computer vision. With the rapid development of image processing technology and intelligent hardware manufacturing technology, human behavior data is more and more abundant and more various. There can be generally divided into two categories, sensor and vision. Behavior data information is obtained with the respective sensors worn based on the behavior recognition of the sensors. The behavior recognition based on the vision can be divided into recognition based on a single-frame image and recognition based on a video, the behavior recognition based on the single-frame image cannot effectively acquire coherent information between the behavior images, misjudgment can be generated generally, the behavior recognition based on the video can well acquire space and time information in the video, and the accuracy rate is greatly improved. The behavior analysis based on the video has been widely researched and applied due to strong expansibility and high flexibility. In practice, the human behavior data, whether it is sensor or visually acquired data, is a sequence data set.
The human body behavior segmentation is the accurate behavior segmentation of the original human body motion capture data, is the basis of the structural analysis, understanding and application of the human body motion data, and is a hot spot concerned by researchers at home and abroad. A great deal of research is carried out to deeply discuss the problem, and a series of results are obtained, so that three types of methods are roughly formed.
Aiming at the difficult problems of high dimensionality and large processing calculation amount of original human motion capture data, some researchers provide a segmentation method based on data dimension reduction. Barbic et al suggested that different body motion behaviors can be represented by different intrinsic dimensions, and proposed methods such as PCA, PPCA, and GMM. The methods reduce the dimension of human motion data by Principal Component Analysis (PCA). Suppose that: the intrinsic dimension of the motion sequence containing a single behavior should be smaller than the intrinsic dimension of the motion sequence containing multiple behaviors, and therefore, the position of the behavior segmentation point can be detected by a projection error in the subspace. Yantao et al propose a method for extracting keyframes from motion capture data based on hierarchical curve simplification. The method based on PCA has low requirements on hardware, is relatively simple to realize, and can be easily realized for some applications which accord with assumptions.
Methods based on deep learning are a new focus of research in recent years. Like the TS-CNN method, spatio-temporal CNN (ST-CNN) of low-level coded visual information is used for fine-grained tasks of coding object states, positions, and relationships between objects. A semi-Markov model and a Conditional Random Field (CRF) are introduced to capture high-level temporal information, and the actions are jointly segmented and classified. The TCN method introduces ED-TCN and related TCN coding and decoding network. Misra I et al utilize temporal coherence information to train a depth model for behavior recognition and pose estimation. In summary, the method based on deep learning generally uses a combination of CNN (or an automatic encoder) and other machine learning methods to implement behavior segmentation, which is superior to other methods in effect, but has a high requirement on hardware configuration, and relies on a large amount of data, which is difficult to implement.
Compared with the two ideas, the clustering-based method has the characteristics of relative simplicity and good effect, and is also the key point for researching human behavior segmentation. Aca (aligned clusterinonalysis), for example, is based on two extended kmeans clustering that ensures that a cluster can contain a variable number of features. In order to overcome the shortcomings of ACA, Zhou et al have proposed HACA methods. As an important method, subspace clustering (subspace clustering) is often used to solve the motion segmentation problem. Xia et al propose a method based on SSC (sparse subspace clustering), perform subspace clustering through SSC, and solve the problem that similar frames cannot be grouped into the same cluster in different time periods by using triangle constraint, thereby ensuring temporal continuity. Considering that a human behavior sequence is a time series data, and similarity information is contained between adjacent sequence points, some methods design a clustering algorithm for the sequence data and apply to human behavior segmentation. Such as osc (ordered Subspace clustering) embeds neighborhood structure information into the optimization target of the Subspace representation. Li et al propose TSC (temporal spatial clustering) algorithm, and adopt different constraint regular terms to embed similarity constraint between sequence data into a subspace projection process, so as to obtain better action representation.
However, the difficulty of human behavior segmentation is: 1) it is difficult to design a reasonable and easy-to-calculate behavior sequence point similarity metric criterion. Because the behavior movement is continuous sequence data over a period of time, the similarity of the behavior data must consider the similarity problem between the behavior movements besides the similarity between the sequence points in the behavior movements, and the comparison of the similarity between the behavior movements needs to consider the problems of alignment, clipping and the like; 2) generally, the behavior sequence data is high-dimensional sequence data, and preprocessing such as similarity measurement, data alignment, data dimension reduction and the like needs to be performed on the high-dimensional data, so that actual implementation is difficult.
Disclosure of Invention
The invention provides a motion behavior segmentation method based on multi-neighbor graph constraint matrix decomposition, which has the advantage that the phenomenon of wrong segmentation exists in a motion sequence, and can accurately distinguish similar motions in human body behaviors.
The invention provides an action behavior segmentation method for multi-neighbor graph constraint matrix decomposition, which comprises the following steps of:
s1, designing a constraint regular term of the multi-graph constraint structure through behavior action high-dimensional sequence data;
s2, constructing a regular term constraint non-semi-negative matrix semi-NMF decomposition model;
s3, solving a non-semi-negative matrix semi-NMF decomposition model to obtain a low-dimensional expression matrix H of the behavior sequence;
s4, generating a relational graph G of sequence points by using a low-dimensional representation matrix H;
s5, dividing the relation graph G of the sequence points by using a graph cutting method to obtain the division of the action behaviors.
The specific step of designing the constraint regular term of the multi-graph constraint structure in step S1 includes:
s11, calculating a neighbor constraint graph R1
Let input X ∈ Rd×nThe low dimensional representation is written as: h is belonged to Rp×nWherein d represents the original dimension of each data point in the sequence, p represents the dimension of the data points in the low-dimensional space, and n represents the total number of the data points in the sequence; because the human body behavior is formed by continuous sequences, the adjacent sequence points have higher similarity; the closer the sequence points are, the higher the similarity is; the farther the sequence points are, the lower the similarity is;
let the current data point i, the inclusion ratioThe sum of the number of the adjacent neighbors is q, q is a positive even number, and the error between i and the adjacent neighbors is as small as possible, namely:
and (5) approaching 0, constructing structural constraint by using the current data point and a plurality of neighbors thereof, and defining a banded matrix R1∈Rn×n；
When i is less than or equal to q/2, the number of neighbors before the current sequence point is less than q/2, and at the moment, q neighbors after the sequence point are complemented in sequence; similarly, when i is larger than n-q/2, the number of neighbors behind the current sequence point is less than q/2, and q neighbors ahead of the sequence point are complemented in sequence;
s12, calculating a similarity graph R2
For input X ∈ Rd×nCalculating the similarity between each node and all other nodes, wherein the first q nodes with the highest similarity form a similarity graph R2∈Rn×nAnd is used for representing a plurality of node sets similar to the nodes in the data.
The specific method for constructing the regular term constraint non-semi-negative matrix semi-NMF decomposition model in the step S2 is as follows:
for X ∈ Rd×nThe method is characterized by high-dimensional data, and after multi-graph constraint is obtained, dimension reduction processing is carried out on original data by using semi-NMF; let Z be an element of Rd×pRepresenting a feature space, H ∈ Rp×nA matrix of representation coefficients representing the input data in a feature space, the elements of the matrix being non-negative values;
the optimization objective was constructed as follows:
wherein the content of the first and second substances,
representing Frobenius norm, wherein alpha and beta are used as weight parameters to adjust the weight of a regular constraint term in an optimization target, and tau is a weight vector of different constraint graphs, wherein L1＝D1-R1，D1＝diag(∑R1) And L2＝D2-R2，D2＝diag(∑R2)。
The step S3 of solving the non-semi-negative matrix semi-NMF decomposition model to obtain the low-dimensional expression of the behavior sequence as H includes the following specific steps:
solving the semi-NMF matrix decomposition model is a non-convex optimization problem without a global optimal solution, obtaining a local optimal solution of the optimization model by using an alternative iteration thought, setting iteration times, respectively fixing other variables each time, and sequentially calculating Z, H and tau until the iteration times reach a set value, wherein the obtained H is final output;
at each iteration, Z, H and τ are calculated as follows:
the iterative formula for Z: according to the matrix operation rule, the closed solution is obtained as follows:
Z＝XHT(HHT)-1
the iterative formula of H: according to the rule of non-negative matrix factorization, the iterative formula of H requires solving the following optimization objectives:
command 2]+Denotes a non-negative element containing only the original matrix, [ alpha ]]-Representing only non-positive elements of the original matrix, the iterative formula of H is:
wherein
The iteration rule of τ is: the computational requirements of τ solve the following optimization objectives:
let γ be α/β, and by the above calculation, the low-dimensional representation matrix H is obtained.
The specific method for generating the relationship graph G of the sequence points by using the low-dimensional representation matrix H in step S4 is as follows:
generating a relation Graph G epsilon R of sequence points by using H according to a spectrogram theory (Spectral Graph)n×nThe vertex of the graph represents a sequence point, the edge of the graph is obtained by calculating the similarity between the sequence point on the H and other points, the first k points with the highest similarity form a k adjacent set N (i) of the point, and the edge G (i, j) is defined as:
obviously, because the i +1 th point is very close to the point i, and the similarity between the points is high due to the structural constraint, an edge is generated, and thus, the points in the action sequence are ensured to be divided into the same class as much as possible; if the similarity between i and i +1 is still not high even if structural constraint is utilized, i is in the ending position of the action sequence, and if the similarity value between points far away from i is also high, i belongs to the repeated action sequence, so that the repeated action sequence can be classified into the same class.
The graph segmentation method in step S5 is to segment G by using the eigenvector corresponding to the second smallest laplacian eigenvalue of G.
Compared with the prior art, the invention has the beneficial effects that:
the invention designs a reasonable and easily-calculated behavior sequence point similarity measurement criterion, and carries out preprocessing such as similarity measurement, data alignment, data dimension reduction and the like on high-dimensional data according to the similarity measurement criterion, so that the accuracy of segmentation actions is higher, the phenomenon of wrong division in the action sequence is few, and the more similar actions in human body behaviors can be accurately distinguished.
Drawings
Fig. 1 is a flow chart of an action behavior segmentation method of multi-neighbor graph constraint matrix decomposition according to the present invention.
Fig. 2 is a schematic diagram of a number 1 data action segmentation result according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of a number 2 data action segmentation result according to an embodiment of the present invention.
Fig. 4 is a schematic diagram of a result of dividing the number 3 data action according to an embodiment of the present invention.
Detailed Description
An embodiment of the present invention will be described in detail below with reference to fig. 1-4, but it should be understood that the scope of the present invention is not limited to the embodiment.
As shown in fig. 1, an action behavior segmentation method based on multi-neighbor graph constraint matrix decomposition according to an embodiment of the present invention provides a semi-non-negative matrix decomposition (semi-NMF) method based on multi-graph constraint for representing action sequence data, and then performs clustering by using a graph segmentation method, so as to implement segmentation of human body behaviors. The main work comprises the following steps: 1. designing a multi-graph constraint structure constraint regular term; 2. constructing a regular term constraint semi-NMF matrix decomposition model; 3. solving the model to obtain a low-dimensional representation of the behavior sequence; 4. calculating sequence point similarity weight by using the low-dimensional representation to generate a similarity relation matrix; 5. and obtaining the behavior segmentation by using a graph segmentation method.
1. Designing multi-graph constraint regularization terms
Firstly: computing a neighbor constraint graph R1
Let input X ∈ Rd×nThe low dimension notation is: h is belonged to Rp×nWherein d represents the original dimension of each data point in the sequence, p represents the dimension of the data point in the low-dimensional space, and n represents the total number of the data points in the sequence. Since human behavior is composed of continuous sequences, there is a high similarity between adjacent sequence points. In general, the closer the sequence points are, the higher the similarity is; the farther the sequence points are, the lower the similarity. We want to be as similar as possible between neighboring points when constructing the sequence data representation to ensure that the data representation is as compact as possible within the behavioral actions. Assuming that the current data point i, the sum of the numbers of the adjacent neighbors before and after the comparison is q (q is a positive even number), we want the error of i and the adjacent neighbors as small as possible, namely:
as close to 0 as possible. Accordingly, structural constraints can be constructed using the current data point and its neighbors, for which a special band matrix R is defined1∈Rn×n：
When i is less than or equal to q/2, the number of neighbors before the current sequence point is less than q/2, and at the moment, q neighbors after the sequence point can be sequentially complemented; similarly, when i is larger than n-q/2, the number of neighbors behind the current sequence point is less than q/2, and q neighbors ahead of the sequence point can be sequentially complemented. For example, when n is 10 and q is 2, the 1 st sequence point is not immediately before, and the third sequence point after it can be included in the structural constraint.
Secondly, the method comprises the following steps: calculate the similarity graph R2
For input X ∈ Rd×nThe similarity between each node and all other nodes can be calculated, and the first q nodes with the highest similarity can form a similarity graph R2∈Rn×nAnd is used for representing a plurality of node sets similar to the nodes in the data. R2The specific computational Process reference (Cai D, He X, Han J, et al, graph regulated nonlinear Matrix Factorization for Data replication [ J.].IEEE Transactions on Pattern Analysis and Machine Analysis, 2011,33(8): 1548-.
Through the above calculation, two different graphs can be used to characterize the neighbor similarity information of the node.
2. Constructing a regularization term constraint semi-NMF matrix decomposition model
For X ∈ Rd×nThe method is characterized by high-dimensional data, and after multi-graph constraint is obtained, dimension reduction processing is carried out on the original data by using semi-NMF. Let Z be an element of Rd×pRepresenting a feature space, and H represents a coefficient matrix representing input data in the feature space, wherein elements of the matrix are non-negative values.
The optimization objective can be constructed as follows:
wherein the content of the first and second substances,
representing Frobenius norm, taking alpha and beta as weight parameters to adjust the weight of a regular constraint term in an optimization target, and tau is a weight vector of different constraint graphs, wherein L is1＝D1-R1，D1＝diag(∑R1) And L2＝D2-R2，D2＝diag(∑R2)。
3. Solving the model to obtain a low-dimensional representation H of the behavior sequence
Solving the model is a non-convex optimization problem, has no global optimal solution, and can obtain the local optimal solution of the optimization model by using an alternative iteration thought. The number of iterations can be set to 100, and each time the other variables are fixed, Z, H and τ are calculated in turn until after 100 iterations, H is obtained as the final output.
At each iteration, Z, H and τ are calculated as follows:
the iterative formula for Z: since Z can be a negative value, its closed form solution can be obtained according to the matrix operation rule.
Z＝XHT(HHT)-1
The iterative formula of H: the iterative formula of H requires solving the following optimization objectives according to the rules of the non-negative matrix factorization
Command 2]+Denotes a non-negative element containing only the original matrix, [ alpha ]]-Representing only non-positive elements of the original matrix, the iterative formula of H is:
wherein
The iteration rule of τ is: the computational requirements of τ solve the following optimization objectives:
let γ be α/β, then the optimization objective is a typical constrained quadratic programming problem that can be solved using any method available, see Jie Chen, Hua Mao, Yongsheng Sang, Zhang yi.
Through the above calculation, the low-dimensional representation matrix H can be obtained.
4. Generation of relational graph G by using H
According to the spectrogram theory (Spectral Graph) literature Belkin M.Laplacian eigenmaps and Spectral techniques for embedding and clustering [ A].Advances in neural informationprocessing systems[C]Cambridge, MA 2001.585-591, using H to generate a graph G e R of the sequence pointsn×n. The vertex of the graph represents a sequence point, the edge of the graph is obtained by calculating the similarity between the sequence point on the H and other points, the first k points with the highest similarity form a k adjacent set N (i) of the point, and the edge G (i, j) is defined as:
obviously, because the i +1 th point is very close to the point i, and the similarity between the points is generally high due to the structural constraint, an edge is generated, and thus, the points in the action sequence are ensured to be divided into the same class as much as possible; if the similarity between i and i +1 is not high even if the structural constraint is utilized, it indicates that i may be at the end of the action sequence. Meanwhile, for points far away from i, if the similarity value between the points is high, the points may belong to repeated action sequences, so that the repeated action sequences can be classified into the same class.
5. Classification of G by means of graph Cut (Normalized Cut: Ncut)
G was classified by the method of graph Cut (Normalized Cut: Ncut). Ncut is a commonly used graph division method that divides G by using a eigenvector corresponding to the second smallest laplacian eigenvalue of G. Reference is made in detail to the literature (Shi J. normalized volumes and image segmentation [ J ]. IEEE Transactions on Pattern Analysis and Machine understanding, 2000,22(8): 888-.
The concrete case is as follows: we used the 1 st, 2 nd, and 3 rd of CMU Mocap 86 (see Chu W S, et al, Video Co-simulation by Video Co-simulation A. 2015IEEE Conference on Computer Vision and Pattern Recognition C. Boston, MA, USA, IEEE Computer Society:2015.3584-3592) as an example. The human motion in each frame in the Mocap dataset consists of 42-dimensional vectors. As shown in fig. 2, 3 and 4, the first row in the dataset is the true segmentation point and the second row is the segmentation point for the action decomposition of the invention. The two number sequences need to be divided into more than 9 parts, as shown in fig. 4, and contain 7 action behaviors (classes), that is, there are repeated action behaviors in the sequence, and the two sequences contain negative value data.
Setting parameters: the number of iterations is selected to be 100, the dimension p of the H matrix is selected to be larger than the number of classes, and we select p to be 15, q to be a positive even number, such as 2, 4, 6, 8, 10, and the like. k is selected to be a positive integer, and is generally selected to be greater than 3. Alpha and beta are chosen as weighting parameters, typically between 0.001 and 1. The segmentation results are shown in fig. 2, 3 and 4, respectively.
The invention designs a reasonable and easily-calculated behavior sequence point similarity measurement criterion, and carries out preprocessing such as similarity measurement, data alignment, data dimension reduction and the like on high-dimensional data according to the similarity measurement criterion, so that the accuracy of segmentation actions is high (the difference between a segmentation starting point and a segmentation ending point is not large), the phenomenon of wrong segmentation is rarely generated in the action sequence, and the similar actions in human body behaviors can be accurately distinguished.
The above disclosure is only for a few specific embodiments of the present invention, however, the present invention is not limited to the above embodiments, and any variations that can be made by those skilled in the art are intended to fall within the scope of the present invention.
Claims (6)
1. A multi-neighbor graph constraint matrix decomposition action behavior segmentation method is characterized by comprising the following steps:
s1, designing a constraint regular term of the multi-graph constraint structure through behavior action high-dimensional sequence data;
s2, constructing a regular term constraint non-semi-negative matrix semi-NMF decomposition model;
s3, solving a non-semi-negative matrix semi-NMF decomposition model to obtain a low-dimensional expression matrix H of the behavior sequence;
s4, generating a relational graph G of sequence points by using a low-dimensional representation matrix H;
s5, dividing the relation graph G of the sequence points by using a graph cutting method to obtain the division of the action behaviors.
2. The method for motion behavior segmentation in multi-neighbor graph constraint matrix decomposition according to claim 1, wherein the specific step of designing the constraint regular term of the multi-graph constraint structure in step S1 includes:
s11, calculating a neighbor constraint graph R1
Let input X ∈ Rd×nThe low dimensional representation is written as: h is belonged to Rp×nWherein d represents the original dimension of each data point in the sequence, p represents the dimension of the data points in the low-dimensional space, and n represents the total number of the data points in the sequence; because the human body behavior is formed by continuous sequences, the adjacent sequence points have higher similarity; the closer the sequence points are, the higher the similarity is; the farther the sequence points are, the lower the similarity is;
setting a current data point i, wherein the sum of the numbers of adjacent neighbors before and after comparison is q, q is a positive even number, and the error between i and the adjacent neighbors is as small as possible, namely:
and (5) approaching 0, constructing structural constraint by using the current data point and a plurality of neighbors thereof, and defining a banded matrix R1∈Rn×n；
When i is less than or equal to q/2, the number of neighbors before the current sequence point is less than q/2, and at the moment, q neighbors after the sequence point are complemented in sequence; similarly, when i is larger than n-q/2, the number of neighbors behind the current sequence point is less than q/2, and q neighbors ahead of the sequence point are complemented in sequence;
s12, calculating a similarity graph R2
For input X ∈ Rd×nCalculating the similarity between each node and all other nodes, wherein the first q nodes with the highest similarity form a similarity graphR2∈Rn×nAnd is used for representing a plurality of node sets similar to the nodes in the data.
3. The method for motion behavior segmentation in multi-neighbor graph constraint matrix factorization of claim 2, wherein the step S2 is implemented by constructing a regular term constraint non-semi-negative matrix semi-NMF factorization model by a specific method comprising:
for X ∈ Rd×nThe method is characterized by high-dimensional data, and after multi-graph constraint is obtained, dimension reduction processing is carried out on original data by using semi-NMF; let Z be an element of Rd×pRepresenting a feature space, H ∈ Rp×nA matrix of representation coefficients representing the input data in a feature space, the elements of the matrix being non-negative values;
the optimization objective was constructed as follows:
wherein the content of the first and second substances,
representing Frobenius norm, wherein alpha and beta are used as weight parameters to adjust the weight of a regular constraint term in an optimization target, and tau is a weight vector of different constraint graphs, wherein L1＝D1-R1，D1＝diag(∑R1) And L2＝D2-R2，D2＝diag(∑R2)。
4. The method for motion behavior segmentation in multi-neighbor graph constraint matrix factorization of claim 3, wherein the step S3 of solving the non-semi-negative matrix semi-NMF factorization model obtains a low-dimensional representation of a behavior sequence as H, and the specific method is as follows:
solving the semi-NMF matrix decomposition model is a non-convex optimization problem without a global optimal solution, obtaining a local optimal solution of the optimization model by using an alternative iteration thought, setting iteration times, respectively fixing other variables each time, and sequentially calculating Z, H and tau until the iteration times reach a set value, wherein the obtained H is final output;
at each iteration, Z, H and τ are calculated as follows:
the iterative formula for Z: according to the matrix operation rule, the closed solution is obtained as follows:
Z＝XHT(HHT)-1
the iterative formula of H: according to the rule of non-negative matrix factorization, the iterative formula of H requires solving the following optimization objectives:
command 2]+Denotes a non-negative element containing only the original matrix, [ alpha ]]-Representing only non-positive elements of the original matrix, the iterative formula of H is:
wherein
The iteration rule of τ is: the computational requirements of τ solve the following optimization objectives:
let γ be α/β, and by the above calculation, the low-dimensional representation matrix H is obtained.
5. The method for motion behavior segmentation in constraint matrix decomposition of multi-neighbor graph according to claim 1, wherein the specific method for generating the relationship graph G of sequence points by using the low-dimensional representation matrix H in step S4 is as follows:
generating a relation Graph G epsilon R of sequence points by using H according to a spectrogram theory (Spectral Graph)n×nThe vertex of the graph represents a sequence point, the edge of the graph is obtained by calculating the similarity between the sequence point on the H and other points, the first k points with the highest similarity form a k adjacent set N (i) of the point, and the edge G (i, j) is defined as:
obviously, because the i +1 th point is very close to the point i, and the similarity between the points is high due to the structural constraint, an edge is generated, and thus, the points in the action sequence are ensured to be divided into the same class as much as possible; if the similarity between i and i +1 is still not high even if structural constraint is utilized, i is in the ending position of the action sequence, and if the similarity value between points far away from i is also high, i belongs to the repeated action sequence, so that the repeated action sequence can be classified into the same class.
6. The method for motion behavior segmentation in multi-neighbor graph constraint matrix decomposition according to claim 1, wherein the graph segmentation in step S5 is performed by segmenting G using eigenvectors corresponding to the second smallest laplacian eigenvalue of G.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110326354.5A CN113052083A (en) | 2021-03-26 | 2021-03-26 | Action behavior segmentation method for multi-neighbor graph constraint matrix decomposition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110326354.5A CN113052083A (en) | 2021-03-26 | 2021-03-26 | Action behavior segmentation method for multi-neighbor graph constraint matrix decomposition |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113052083A true CN113052083A (en) | 2021-06-29 |
Family
ID=76515601
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110326354.5A Pending CN113052083A (en) | 2021-03-26 | 2021-03-26 | Action behavior segmentation method for multi-neighbor graph constraint matrix decomposition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113052083A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6009437A (en) * | 1997-03-25 | 1999-12-28 | Nec Research Institute, Inc. | Linear fitting with missing data: applications to structure-from-motion and to characterizing intensity images |
| US20090028433A1 (en) * | 2007-05-03 | 2009-01-29 | David Allen Tolliver | Method for partitioning combinatorial graphs |
| WO2013015528A1 (en) * | 2011-07-27 | 2013-01-31 | Samsung Electronics Co., Ltd. | Apparatus, method, and medium detecting object pose |
| CN109508752A (en) * | 2018-12-20 | 2019-03-22 | 西北工业大学 | A kind of quick self-adapted neighbour's clustering method based on structuring anchor figure |
| CN109657611A (en) * | 2018-12-19 | 2019-04-19 | 河南科技大学 | A kind of adaptive figure regularization non-negative matrix factorization method for recognition of face |
| CN110108914A (en) * | 2019-05-21 | 2019-08-09 | 国网湖南省电力有限公司 | One kind is opposed electricity-stealing intelligent decision making method, system, equipment and medium |
| CN110268634A (en) * | 2016-11-23 | 2019-09-20 | 苏伊士集团 | Use the encoder and decoder for the low consumption application that the half canonical LDPC code of quasi- circulation of short length is used to such as remotely read |
| CN110276049A (en) * | 2019-06-24 | 2019-09-24 | 河南科技大学 | A kind of semi-supervised adaptive figure regularized discriminant non-negative matrix factorization method |
| CN110598728A (en) * | 2019-07-23 | 2019-12-20 | 杭州电子科技大学 | Semi-supervised ultralimit learning machine classification method based on graph balance regularization |
| KR20200063904A (en) * | 2018-11-28 | 2020-06-05 | 서울대학교산학협력단 | Method and apparatus for measuring relevance between nodes of edge-labeled multigraph |
-
2021
- 2021-03-26 CN CN202110326354.5A patent/CN113052083A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6009437A (en) * | 1997-03-25 | 1999-12-28 | Nec Research Institute, Inc. | Linear fitting with missing data: applications to structure-from-motion and to characterizing intensity images |
| US20090028433A1 (en) * | 2007-05-03 | 2009-01-29 | David Allen Tolliver | Method for partitioning combinatorial graphs |
| WO2013015528A1 (en) * | 2011-07-27 | 2013-01-31 | Samsung Electronics Co., Ltd. | Apparatus, method, and medium detecting object pose |
| CN110268634A (en) * | 2016-11-23 | 2019-09-20 | 苏伊士集团 | Use the encoder and decoder for the low consumption application that the half canonical LDPC code of quasi- circulation of short length is used to such as remotely read |
| KR20200063904A (en) * | 2018-11-28 | 2020-06-05 | 서울대학교산학협력단 | Method and apparatus for measuring relevance between nodes of edge-labeled multigraph |
| CN109657611A (en) * | 2018-12-19 | 2019-04-19 | 河南科技大学 | A kind of adaptive figure regularization non-negative matrix factorization method for recognition of face |
| CN109508752A (en) * | 2018-12-20 | 2019-03-22 | 西北工业大学 | A kind of quick self-adapted neighbour's clustering method based on structuring anchor figure |
| CN110108914A (en) * | 2019-05-21 | 2019-08-09 | 国网湖南省电力有限公司 | One kind is opposed electricity-stealing intelligent decision making method, system, equipment and medium |
| CN110276049A (en) * | 2019-06-24 | 2019-09-24 | 河南科技大学 | A kind of semi-supervised adaptive figure regularized discriminant non-negative matrix factorization method |
| CN110598728A (en) * | 2019-07-23 | 2019-12-20 | 杭州电子科技大学 | Semi-supervised ultralimit learning machine classification method based on graph balance regularization |
Non-Patent Citations (4)
| Title |
|---|
| BELKIN M: "Laplacian eigenmaps andspectral techniques for embedding and clustering", 《ADVANCES IN NEURAL INFORMATION PROCESSING》, 3 January 2001 (2001-01-03), pages 585 - 591 * |
| GAO HONGBO: "A structure constraint matrix factorization framework for human behavior segmentation", 《TRANSACTIONS ON CYBERNETICS》, vol. 52, no. 12, 31 December 2022 (2022-12-31), pages 12978 - 12988, XP011927942, DOI: 10.1109/TCYB.2021.3095357 * |
| SHI J: "Normalized cuts and image segmentation", 《TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE》, 31 August 2000 (2000-08-31), pages 888 - 905, XP002413789, DOI: 10.1109/34.868688 * |
| 李国朋: "多图融合约束半非负矩阵分解的动作分割方法", 《智能系统学报》, vol. 18, no. 6, 5 November 2023 (2023-11-05), pages 1223 - 1232 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Sakkos et al. | End-to-end video background subtraction with 3d convolutional neural networks | |
| Hong et al. | Image-based three-dimensional human pose recovery by multiview locality-sensitive sparse retrieval | |
| Dockstader et al. | Multiple camera tracking of interacting and occluded human motion | |
| Xiao et al. | Online meta adaptation for fast video object segmentation | |
| Chen et al. | End-to-end learning of object motion estimation from retinal events for event-based object tracking | |
| CN112232134B (en) | Human body posture estimation method based on hourglass network and attention mechanism | |
| Wang et al. | An active contour model based on local pre-piecewise fitting bias corrections for fast and accurate segmentation | |
| Li et al. | Weighted error entropy-based information theoretic learning for robust subspace representation | |
| CN115661246A (en) | Attitude estimation method based on self-supervision learning | |
| Gao et al. | Tracking-by-fusion via Gaussian process regression extended to transfer learning | |
| Zhang et al. | 3-D deconvolutional networks for the unsupervised representation learning of human motions | |
| CN111402303A (en) | Target tracking architecture based on KFSTRCF | |
| Xu et al. | Graphical modeling for multi-source domain adaptation | |
| Alsanad et al. | Real-time fuel truck detection algorithm based on deep convolutional neural network | |
| Wang et al. | Spatio-temporal online matrix factorization for multi-scale moving objects detection | |
| Mattheus et al. | A review of motion segmentation: Approaches and major challenges | |
| Martı́nez Carrillo et al. | A compact and recursive Riemannian motion descriptor for untrimmed activity recognition | |
| CN113052083A (en) | Action behavior segmentation method for multi-neighbor graph constraint matrix decomposition | |
| Zhang et al. | Machine learning-based multitarget tracking of motion in sports video | |
| Zeng et al. | Probability-based framework to fuse temporal consistency and semantic information for background segmentation | |
| CN112150502A (en) | Method for detecting moving target in slowly-varying moving background | |
| Shedligeri et al. | High frame rate optical flow estimation from event sensors via intensity estimation | |
| Jayant et al. | Study of robust facial recognition under occlusion using different techniques | |
| CN104751448A (en) | Online video tracking method based on PCA (Principal Component Analysis) and noise separation | |
| Teh et al. | Generalised Structural CNNs (SCNNs) for time series data with arbitrary graph topology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |