CN113673411B - Attention mechanism-based lightweight shift graph convolution behavior identification method - Google Patents

Abstract

The invention discloses a light-weight shift mapping convolution behavior identification method based on an attention mechanism, which comprises the steps of preprocessing a data set to generate a joint point information flow data set, a bone length information flow data set, a joint point information flow data set based on motion information and a bone length information flow data set based on the motion information; constructing an ALS-GCN network, and obtaining the time-space characteristics of the information flow through the ALS-GCN network; fusing the time-space characteristics of the information flow to obtain a behavior recognition result; the problem of undersize receptive field is solved by constructing a spatial shifting module based on an attention mechanism; meanwhile, the problem of overhigh parameter quantity caused by a nonlinear stacking mode is solved by constructing the time shifting module, and high identification precision can be achieved under the condition of less calculation amount.

Description

A Lightweight Shifted Graph Convolution Behavior Recognition Method Based on Attention Mechanism

technical field

The technical field to which the present invention relates, in particular, to a light-weight shift graph convolution behavior recognition method based on an attention mechanism.

Background technique

Behavior recognition is one of the important research directions in the field of artificial intelligence, and has important applications in video supervision, intelligent monitoring and human-computer interaction. Behavior recognition is at the same time a challenging task, not only because of the higher computational requirements required to process video clips, but also susceptible to external environmental factors. As a result, it is often difficult for action recognition methods based on RGB video to meet the requirements of timeliness and accuracy at the same time. In recent years, thanks to the development and popularization of depth cameras, such as Microsoft Kinetic, behavior recognition based on depth information has gradually become one of the important research directions in this field. Compared with traditional RGB data, the skeleton sequence has the characteristics of simplicity, easy calibration, and not easily affected by appearance factors because it does not contain color information.

Early behavior recognition methods based on human skeletons mainly characterized behaviors by manually designing features. However, the hand-extracted features often have limited representation ability and require a lot of energy for parameter tuning and optimization.

Although the current mainstream skeleton behavior recognition methods have obvious advantages in representing time information and can extract multi-scale specific local patterns from different time intervals, they all have certain problems, such as high optimization difficulty and easy loss of original joint point information. , There is a problem that the amount of parameters is too large and the calculation requirements are too high.

SUMMARY OF THE INVENTION

The purpose of this section is to outline some aspects of embodiments of the invention and to briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of the application to avoid obscuring the purpose of this section, abstract and title, and such simplifications or omissions may not be used to limit the scope of the invention.

The present invention has been proposed in view of the above-mentioned existing problems.

Therefore, the present invention provides a light-weight shift graph convolution behavior recognition method based on attention mechanism, which can solve the problem of too small receptive field and the problem of too high parameter amount caused by nonlinear stacking.

In order to solve the above technical problems, the present invention provides the following technical solutions: including, preprocessing the data set, generating joint point information flow data set, bone length information flow data set, joint point information flow data set based on motion information and motion-based information flow data set. A data set of bone length information flow of information; construct an ALS-GCN network, and obtain the spatiotemporal characteristics of the information flow through the ALS-GCN network; fuse the spatiotemporal characteristics of the information flow to obtain behavior recognition results.

As a preferred solution of the light-weight shift graph convolution behavior recognition method based on the attention mechanism of the present invention, wherein: the preprocessing includes: removing falsely detected camera data; defining an index for judging the energy of the skeleton sequence , remove the falsely detected joint point data; normalize the joint point coordinate data; normalize the viewing angle.

As a preferred solution of the light-weight shift graph convolution behavior recognition method based on the attention mechanism of the present invention, the method further includes: using a convolutional neural network to analyze the joint point information flow data set, bone length Information flow data set, joint point information flow data set based on motion information and bone length information flow data set based on motion information are extracted to obtain joint point information flow, bone length information flow, motion information flow and bone length information flow based on motion information. long information flow.

As a preferred solution of the light-weight shift graph convolution behavior recognition method based on the attention mechanism of the present invention, wherein: the ALS-GCN network includes a spatial shift module and a temporal shift module; the spatial shift module The shift module includes a spatial attention module and a channel attention module; the temporal shift module is optimized by setting an adaptive temporal shift graph convolution and adding short connections.

As a preferred solution of the light-weight displacement graph convolution behavior recognition method based on the attention mechanism of the present invention, wherein: the bone length information flow includes defining the joint closer to the center of gravity of the skeleton as the source joint, The joint farther from the center of gravity is the target joint; the source joint in frame t is:

V _i,t =( _xi,t ,y _i,t ,z _i,t )

The target joint in frame t is:

V _j,t =(x _j,t ,y _j,t ,z _j,t )

The bone length information flow e _i,j,t is defined as follows:

e _i,j,t =V _j,t -V _i,t

Among them, i and j are joint points, and x, y, and z are joint coordinates.

As a preferred solution of the light-weight shift graph convolution behavior recognition method based on the attention mechanism of the present invention, wherein: the motion information flow includes:

em _i,t,t+1 =V _i,t -V _i,t+1 =( _xi,t+1 -xi _,t ,y _i,t+1 -y _i,t ,z _{i,t +1} -z _i,t )

Wherein, em _i,t,t+1 is the motion information flow.

As a preferred solution of the light-weight shift graph convolution behavior recognition method based on the attention mechanism of the present invention, wherein: the bone length information flow based on motion information includes:

bm _i,t,t+1 =e _i,j,t -e _i,j,t+1

Wherein, bm _i,t,t+1 is the bone length information flow based on the motion information.

As a preferred solution of the light-weight shift graph convolution behavior recognition method based on the attention mechanism of the present invention, wherein: the fusion includes adding the Softmax scores of the spatiotemporal features of the information flow.

Beneficial effects of the present invention: The present invention solves the problem of too small receptive field by constructing a spatial shift module based on an attention mechanism; at the same time, by constructing a temporal shift module, it solves the problem caused by nonlinear stacking. For the problem that the amount of parameters is too high, higher recognition accuracy can be achieved with less computation.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort. in:

FIG. 1 is a schematic structural diagram of the spatial shift module 100 of the light-weight shift graph convolution behavior recognition method based on the attention mechanism according to the first embodiment of the present invention;

2 is a schematic diagram of a forward shift-in operation and a reverse shift-out operation of the light-weight shift graph convolution behavior recognition method based on the attention mechanism according to the first embodiment of the present invention;

FIG. 3 is a schematic diagram of the overall network structure of the light-weight shift graph convolution behavior recognition method based on the attention mechanism according to the first embodiment of the present invention.

Detailed ways

In order to make the above objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Example. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention can also be implemented in other ways different from those described herein, and those skilled in the art can do so without departing from the connotation of the present invention. Similar promotion, therefore, the present invention is not limited by the specific embodiments disclosed below.

Second, reference herein to "one embodiment" or "an embodiment" refers to a particular feature, structure, or characteristic that may be included in at least one implementation of the present invention. The appearances of "in one embodiment" in various places in this specification are not all referring to the same embodiment, nor are they separate or selectively mutually exclusive from other embodiments.

The present invention is described in detail with reference to the schematic diagrams. When describing the embodiments of the present invention in detail, for the convenience of explanation, the sectional views showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the present invention. scope of protection. In addition, the three-dimensional spatial dimensions of length, width and depth should be included in the actual production.

At the same time, in the description of the present invention, it should be noted that the orientation or positional relationship indicated in terms such as "upper, lower, inner and outer" is based on the orientation or positional relationship shown in the accompanying drawings, which is only for the convenience of describing the present invention. The invention and simplified description do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operate in a particular orientation, and therefore should not be construed as limiting the invention. Furthermore, the terms "first, second or third" are used for descriptive purposes only and should not be construed to indicate or imply relative importance.

Unless otherwise expressly specified and limited in the present invention, the term "installation, connection, connection" should be understood in a broad sense, for example: it may be a fixed connection, a detachable connection or an integral connection; it may also be a mechanical connection, an electrical connection or a direct connection. The connection can also be indirectly connected through an intermediate medium, or it can be the internal communication between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood in specific situations.

Example 1

Referring to Figures 1-3, it is the first embodiment of the present invention, which provides a light-weight shift graph convolution behavior recognition method based on attention mechanism, including:

S1: Preprocess the data set to generate a joint point information flow data set, a bone length information flow data set, a joint point information flow data set based on motion information and a bone length information flow data set based on motion information.

Preprocessing the dataset includes the following steps:

(1) Eliminate falsely detected camera data;

In order to reduce the false detection of the camera caused by the use of the Microsoft Kinect camera and ensure the consistency of the id of the human body, in this embodiment, the detection is performed first, that is, it is judged whether a new id will be generated after the original id disappears. , it is considered that the newly generated id is wrongly assigned, and the old id should be assigned to the new id, thereby reducing the false detection rate of the camera.

(2) Define the index for judging the energy of the skeleton sequence, and eliminate the falsely detected joint point data;

Aiming at the problem that the joint point data may be falsely detected, that is, some objects including tables and chairs are falsely detected as human bodies. Therefore, this embodiment introduces an index for judging the energy of the skeleton sequence, which is defined as the standard of three-dimensional coordinates in the time dimension. The average value of the difference; since the energy value of stationary objects is much smaller than the energy value of the human body during training data collection, these objects below the threshold can be filtered out by setting a threshold.

(3) Normalized joint point coordinate data;

In order to unify the distribution of data and simplify the training process of the designed model, the joint point of the first person appearing in the first frame is regarded as the origin of the coordinate system, and then the center point is subtracted from the coordinates of each joint point. coordinates to normalize other frames into this coordinate system.

(4) Normalized perspective.

Rotate the human skeleton in each frame to a specific angle to reduce the impact on model training due to changes in perspective; Rotate the skeleton in all subsequent frames so that the line connecting the left and right shoulders of the skeleton in all subsequent frames is parallel to the x-axis, and the spine is parallel to the z-axis of the coordinate system, thereby obtaining a rotation matrix.

Through the above processing, a joint point information flow data set, a bone length information flow data set, a joint point information flow data set based on motion information, and a bone length information flow data set based on motion information that can be used for network training are generated.

S2: Build an ALS-GCN network and obtain the spatiotemporal features of the information flow through the ALS-GCN network.

The overall architecture of the ALS-GCN network consists of basic blocks. The Basic Block consists of 9 spatial shift modules 100 and temporal shift modules 200. The number of output channels is 64, 64, 64, 128, 128, 128, 256, 256 and 256.

(1) Specifically, referring to FIG. 1, the spatial shift module 100 includes a spatial attention module 101 and a channel attention module 102. It should be noted that the convolution operation in the global Shift graph mentioned in the figure, due to the breakthrough of the The physical connection of the human body is limited, and each node is interconnected with other nodes.

①The spatial attention module 100, the manifestation on the image refers to the different degrees of attention to different positions on the feature map, which is described by mathematical language: for a feature map with a size of H×W×C, an effective Spatial attention corresponds to a matrix of size H×W; the pixel-wise multiply operation (Pixel-Wise Multiply) is performed with the pixel at the corresponding position on the original feature map to obtain the weight of the pixel at the corresponding position; in the joint point-based In the human skeleton data, the spatial attention module 101 can help the ALS-GCN network to assign different degrees of attention weight to each joint. The specific implementation formula is as follows:

M _s (f _in )=σ(g _s (AvgPool(f _in ))

AvgPool是指平均池化操作，

为输入特征，最后生成的基于空间注意力机制的注意力特征图为M_s∈R^1×1×N；通过残差方式将注意力特征图M_s乘以输入特征图，最终实现了赋予了每个关节点不同程度注意力权重的目的。Among them, σ represents the sigmoid activation function operation; g _s is the one-dimensional convolution operation, and the convolution size is

AvgPool refers to the average pooling operation,

As the input feature, the finally generated attention feature map based on the spatial attention mechanism is M _s ∈ R ^1×1×N ; the attention feature map M _s is multiplied by the input feature map through the residual method, and finally the given The purpose of different degrees of attention weights for each joint point.

②The channel attention module 102 usually means that in order to effectively calculate the correlation between the channel and the key information, it is necessary to assign a weight value to each channel. The degree of attention of image channels is different; it is reflected in mathematics: for a feature map of size H×W×C, a matrix of dimension 1×1×C is generated to represent channel attention; through channel multiplication ( Channel-Wise Multiply), thereby compressing the spatial dimension of the input feature map, and summing it element by element to generate a channel attention map. In the human skeleton data based on joint points, the channel attention module 102 can help the model according to different input samples. Channel characteristics, strengthen the distinguishing channel characteristics, the specific implementation is as follows:

M _C (f _in )=σ(W ₂ (δ(W ₁ (AvgPool(f _in )))))

与

为两个全连接层权重，其中f_in是关节和帧的平均值。Among them, δ represents the ReLU activation function operation,

and

are the weights of the two fully connected layers, where f _in is the average of joints and frames.

(2) The time shift module 200 is optimized by setting an adaptive time sequence shift graph convolution and adding short connections.

来表示一个骨架序列特征，其中，T为关键帧的数目，N为关节点数，C为通道数。The time shift module 200 is improved on the basis of Shift-GCN and ST-GCN. In order to easily reveal the essence of the proposed method, we use the notation

to represent a skeleton sequence feature, where T is the number of key frames, N is the number of joint points, and C is the number of channels.

Let {S _i |i=1,2,,...,C} be a learnable shift parameter, through the parameter S _i , the forward shift-in operation and the reverse shift-out operation can be realized, as shown in Figure 2, As an adaptive timing shift graph convolution, a learnable time shift parameter can be learned for each channel. Compared with the manual setting of shift parameters, this method greatly improves timing shift. The generalization ability of graph convolutions.

In addition, in order to better characterize the information of the skeleton sequence in the time dimension and the space dimension, ensure the multiple use of the information in the frame, so as to strengthen the correlation between frames, and at the same time strengthen the expressive ability of features. A short connection is added to the network. In this way, the network structure can be simplified as much as possible while improving the accuracy.

Further, feature extraction is performed on the joint point information flow data set, the bone length information flow data set, the joint point information flow data set based on motion information, and the bone length information flow data set based on motion information through the convolutional neural network to obtain joints. Point information flow, bone length information flow, motion information flow and bone length information flow based on motion information (velocity difference information flow).

Among them, because the information flow of joint points can be obtained directly, it will not be discussed in detail here; the generation process of bone length information flow, motion information flow and bone length information flow based on motion information is as follows:

(a) Bone length information flow

Define the joint closer to the center of gravity of the skeleton as the source joint, and the joint farther from the center of gravity as the target joint;

The source joints in frame t are:

V _i,t =( _xi,t ,y _i,t ,z _i,t )

The target joint in frame t is:

V _j,t =(x _j,t ,y _j,t ,z _j,t )

The bone length information flow e _i,j,t is defined as follows:

e _i,j,t =V _j,t -V _i,t

Among them, i and j are joint points, and x, y, and z are joint coordinates.

(b) Sports information flow

The motion information flow is obtained by calculating the difference between the same joint points in two adjacent frames, and extracting the coordinate difference of the joints and bones between two consecutive frames as the motion information flow can help to extract temporal features. The expression is as follows:

em _i,t,t+1 =V _i,t -V _i,t+1 =( _xi,t+1 -xi _,t ,y _i,t+1 -y _i,t ,z _{i,t +1} -z _i,t )

Among them, em _i,t,t+1 is the motion information flow.

(c) Bone length information flow based on motion information:

bm _i,t,t+1 =e _i,j,t -e _i,j,t+1

Among them, bm _i,t,t+1 is the bone length information flow based on motion information.

Finally, the temporal and spatial characteristics of the information flow are obtained from the joint point information flow, the bone length information flow, the motion information flow and the motion information-based bone length information flow through the ALS-GCN network, that is, the spatial shift module 100 obtains the information flow. Spatial features, the temporal features of the information flow are obtained through the temporal shift module 200 .

S3: Integrate the spatiotemporal features of the information flow to obtain behavior recognition results.

The Softmax scores of the spatio-temporal features of the information flow dataset are added to complete the fusion; as shown in Figure 3, joint information and bone information and their corresponding motion information are integrated in a multi-flow framework, through fusion Generate final predictions, where all streams use the same architecture ALS-GCN network.

Example 2

This embodiment will evaluate the effectiveness of the method based on shift graph convolution, and further verify the effectiveness of the method by comparing the experimental results on NTU60 RGB+D and NTU120 RGB+D with other advanced methods sex.

Experimental setup

The experiments in this example are all performed under the 64-bit Ubuntu16.04.1LTS operating system. All network models are trained using 4 GeForce RTX 2080Ti GPUs. For the NTU60 RGB+D dataset, the batch size is set to 64, and the epoch is 80, the stochastic gradient descent (SGD) method is adopted, the momentum is set to 0.9, the initial learning rate is set to 0.1, and the learning rate is multiplied by 0.1 when the epoch is 40 and 60; for NTU120 RGB+D For the data set, the batch size is set to 64, and the gradient descent method of SGD is also used. Considering the change of the data collective, we set the epoch to 120, the initial learning rate to 0.1, and the learning rate when the epoch is 60 and 90 respectively. Multiply by 0.1. All experiments are performed on PyTorch 1.3.1 deep learning framework with Python version 3.6.2.

In order to verify the effect of this method, in this embodiment, experimental comparison is performed on two databases, NTU60 RGB+D and NTU120 RGB+D.

In the experiment, we use the experimental results of 2s-AGCN as the baseline, and use Part-I and Part-II to represent the spatial attention module 101 and the channel attention module 102, in the X-View of the NTU60 RGB+D dataset Under the judging criteria, the importance of the two attention mechanisms is tested by manually removing one of the two modules, where wo/X means the experimental result after removing the X module.

Table 1: Accuracy comparison based on joint information flow in NTU60 RGB+D dataset.

Benefiting from the success of the Shift-GCN four-stream architecture, X-view in the NTU60 RGB+D dataset is used as the judging criterion, in the joint point information flow, bone length information flow, motion information flow and motion information-based bone length information flow Based on this, the effectiveness of the ALS-GCN method is verified, and the specific performance is shown in Table 2. The specific fusion method is to add the Softmax scores of multiple streams to obtain the fused score. Among them, 1s-ALS-GCN means that only the original skeleton coordinates are used as input, that is, only the joint point information flow is used in X-view. The standard classification result; 2s-ALS-GCN is the result of fusion of joint point information and bone length information; 4s-ALS-GCN is the final result of the fusion of the above four information streams.

Table 2: Comparison of the accuracy (%) of the same information flow.

In order to further verify the effectiveness of this method (ALS-GCN), this example is also compared with other mainstream methods in recent years on the NTU60RGB+D data set. Compared with non-graph convolution methods, such as AGC-LSTM, HCN, etc., this method has great advantages; compared with the mainstream graph convolution methods in the past two years, this method also has excellent performance, although compared with the latest There are still some gaps in some methods, but considering the factors such as the low number of parameters of this method and only 80 epochs, this method still has a high reference value.

Table 3: Comparison of the accuracy (%) of the evaluation criteria of X-Sub and X-View under the NTU60 RGB+D dataset.

method years X-Subject (%) X-View(%) Lie Group 2014 50.1 82.8 DPRL 2018 83.5 89.8 SRN-TSL 2018 84.8 84.8 HCN 2018 86.5 86.5 AGC-LSTM 2019 89.2 89.2 ST-GCN 2018 81.5 88.3 DPRL+GCNN 2018 83.5 89.8 AS-GCN 2019 86.8 94.2 2s-AGCN 2019 88.5 95.1 Mix-Dimension 2020 89.7 96.0 PGCN-TCA 2020 88.0 93.6 PA-ResGCN-B19 2020 90.9 96.0 MS-AAGCN 2020 90.0 96.2 FGCN-spatial+FGCN-motion 2020 90.2 96.3 CGCN 2020 90.3 96.4 Shift-GCN 2020 90.7 96.5 DC-GCN+ADG 2020 90.8 96.6 MDM-GCN 2019 89.2 95.9 4s-ALS-GCN 2021 90.5 96.5

In order to better prove the superiority of this method, this example is also compared on the NTU120 RGB+D data set. The specific experimental results are shown in Table 4. Compared with some methods in 2020, this method is still There are deficiencies, but considering that the size of the NTU120 RGB+D dataset is double that of the NTU60 RGB+D dataset, and our epoch is still only 120, the performance of this method under the NTU120 RGB+D dataset is still remarkable. point.

Table 4: Comparison of the accuracy (%) of the evaluation criteria of X-Subject and X-Set under the NTU120 RGB+D dataset.

method years X-Subject (%) X-Set(%) Body PoseEvolution Map 2018 64.6 66.9 TSRJI (Late Fusion) 2019 67.9 62.8 Logsin-RNN 2019 68.3 67.2 GVFE+AS-GCN with DH-TCN 2019 78.3 79.8 SGN 2020 79.2 81.5 Mix-Dimension 2020 80.5 83.2 ST-TR-agcn 2020 82.7 84.7 FGCN 2020 85.4 87.4 Shift-GCN 2020 85.9 87.6 VPN 2020 86.3 87.8 1s-ALS-GCN 2021 80.9 84.7 2s-ALS-GCN 2021 85.5 88.2 4s-ALS-GCN 2021 86.3 89.1

Finally, in order to show that this method can significantly improve the experimental accuracy while taking into account the amount of parameters, we compared the amount of parameters of the method in this paper on the X-Subject and X-View standards of the NTU60RGB+D dataset, as shown in Table 5. ; Through comparison, we can find that this method has the most balanced performance, not only controlling the parameters within a suitable interval, but also having a higher accuracy than the methods in the past two years.

Table 5: Comparison of parameters of different methods.

According to the experimental results on the behavior recognition datasets NTU60 RGB+D and NTU120 RGB+D, this method can achieve high experimental accuracy with less computational effort.

And it can be found that the method solves the problem of too small receptive field by constructing the spatial shift module 100 , and at the same time solves the problem of excessively high parameters caused by the nonlinear stacking method by the time shift module 200 .

It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be Modifications or equivalent substitutions without departing from the spirit and scope of the technical solutions of the present invention should be included in the scope of the claims of the present invention.

Claims (2)

1. a light-weight shift graph convolution behavior recognition method based on attention mechanism, is characterized in that, comprises: Preprocess the data set to generate joint point information flow data set, bone length information flow data set, joint point information flow data set based on motion information and bone length information flow data set based on motion information; The preprocessing includes: Eliminate falsely detected camera data; Define the index for judging the energy of the skeleton sequence, and eliminate the falsely detected joint point data; Normalized joint point coordinate data; normalized perspective; Feature extraction is performed on the joint point information flow data set, the bone length information flow data set, the joint point information flow data set based on motion information and the bone length information flow data set based on motion information through a convolutional neural network to obtain joint points. Information flow, bone length information flow, motion information flow and motion information-based bone length information flow; The bone length information flow includes: Define the joint closer to the center of gravity of the skeleton as the source joint, and the joint farther from the center of gravity as the target joint; The source joints in frame t are: V _i,t =( _xi,t ,y _i,t ,z _i,t ) The target joint in frame t is: V _j,t =(x _j,t ,y _j,t ,z _j,t ) The bone length information flow e _i,j,t is defined as follows: e _i,j,t =V _j,t -V _i,t Among them, i and j are joint points, and x, y, and z are joint coordinates; The motion information flow includes: em _i,t,t+1 =V _i,t -V _i,t+1 =( _xi,t+1 -xi _,t ,y _i,t+1 -y _i,t ,z _{i,t +1} -z _i,t ) Wherein, em _i,t,t+1 is the motion information flow; The bone length information flow based on motion information includes: bm _i,t,t+1 =e _i,j,t -e _i,j,t+1 Wherein, bm _i,t,t+1 is the bone length information flow based on the motion information; constructing an ALS-GCN network, and obtaining the spatiotemporal characteristics of the information flow through the ALS-GCN network; The ALS-GCN network includes a space shift module (100) and a time shift module (200); The spatial shift module (100) includes a spatial attention module (101) and a channel attention module (102); Optimize the time shift module (200) by setting an adaptive time sequence shift graph convolution and adding short connections; The temporal and spatial features of the information flow are fused to obtain behavior recognition results. 2. The method for recognizing light-weight shift graph convolution behavior based on an attention mechanism according to claim 1, wherein the fusion comprises: The Softmax scores of the spatio-temporal features of the information flow are summed.

Images