CN110796081A - Group behavior identification method based on relational graph analysis - Google Patents

Group behavior identification method based on relational graph analysis Download PDF

Info

Publication number
CN110796081A
CN110796081A CN201911036597.4A CN201911036597A CN110796081A CN 110796081 A CN110796081 A CN 110796081A CN 201911036597 A CN201911036597 A CN 201911036597A CN 110796081 A CN110796081 A CN 110796081A
Authority
CN
China
Prior art keywords
frame
video
group behavior
sampling
group
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201911036597.4A
Other languages
Chinese (zh)
Other versions
CN110796081B (en
Inventor
李楠楠
张世雄
赵翼飞
李若尘
李革
安欣赏
张伟民
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Longgang Intelligent Audiovisual Research Institute
Original Assignee
Shenzhen Longgang Intelligent Audiovisual Research Institute
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Longgang Intelligent Audiovisual Research Institute filed Critical Shenzhen Longgang Intelligent Audiovisual Research Institute
Priority to CN201911036597.4A priority Critical patent/CN110796081B/en
Publication of CN110796081A publication Critical patent/CN110796081A/en
Application granted granted Critical
Publication of CN110796081B publication Critical patent/CN110796081B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/53—Recognition of crowd images, e.g. recognition of crowd congestion
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Psychiatry (AREA)
  • Human Computer Interaction (AREA)
  • Social Psychology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

A group behavior identification method based on relational graph analysis comprises the following steps: s1 sparse sampling is carried out on the video sequence containing the group behavior event to obtain a video sampling frame which is used as a representative of the video event; s2, obtaining the character target characteristics of the single-frame sampling frame through a target detection network and dimensionality reduction operation; s3, constructing a graph model according to the appearance and position relation between individuals and extracting the behavior expression characteristics of the single-frame group by utilizing a graph convolution neural network; and S4, obtaining the group behavior expression characteristics of the whole video by fusing the multi-frame group behavior characteristics. Compared with the traditional video behavior dynamic description method based on the recurrent neural network model, the method provided by the invention can be used for simultaneously processing a plurality of sampling video frames, so that the time efficiency of algorithm operation is improved, and a leading detection level is obtained.

Description

Group behavior identification method based on relational graph analysis
Technical Field
The invention relates to the technical field of machine learning methods and video behavior analysis, in particular to a group behavior identification method based on relational graph analysis.
Background
In recent years, with the great success of deep learning techniques in the field of image analysis and understanding, more and more people begin to use deep neural networks to deal with the problems faced by video behavior analysis, such as video behavior classification, video behavior timeline or spatial axis-timeline positioning. At present, the video behavior analysis problem processed by the deep neural network relates to simple video behaviors, such as single-person sports such as high jump and long jump, and behaviors in daily life such as tooth brushing and head combing. However, in real life, a plurality of persons often participate in the behaviors, and describing the group behaviors not only describes the behaviors of single individuals in a scene, but also considers the interaction relationship among the individuals. At present, the existing method uses a pre-designated graph model to depict the interaction relationship between individuals, so that the flexibility is lacked, and different relationship models need to be designed in advance for different events; secondly, in order to describe the dynamic characteristics of the motion, a Recurrent Neural Network (RNN) is often used to process continuously sampled samples, which is highly complex in calculation, and requires sequential processing of input video frames, and the calculation time cost is correspondingly high.
Disclosure of Invention
The invention aims to provide a group behavior recognition method based on relational graph analysis, which comprises the steps of giving a section of multi-person video sequence as input, designing a relational graph model to model appearance and position relations between paired individuals, and utilizing a graph convolution neural network model to extract characteristics of single-frame multi-person behaviors; meanwhile, a video time axis sampling method is used for fusing single-frame multi-person behavior characteristics to represent the dynamic characteristics of group behaviors, and the purpose of classifying and identifying the group behaviors is achieved.
The technical scheme provided by the invention is as follows:
according to one aspect of the invention, a group behavior identification method based on relational graph analysis is provided, which comprises the following steps: s1 sparse sampling is carried out on the video sequence containing the group behavior event to obtain a video sampling frame which is used as a representative of the video event; s2, obtaining the character target characteristics of the single-frame sampling frame through a target detection network and dimensionality reduction operation; s3, constructing a graph model according to the appearance and position relation between individuals and extracting the behavior expression characteristics of the single-frame group by utilizing a graph convolution neural network; and S4, obtaining the group behavior expression characteristics of the whole video by fusing the multi-frame group behavior characteristics.
Preferably, in the above group behavior recognition method based on the relational graph analysis, in step S1, random sampling or uniform sampling in time order sparsely extracts a number of frames from the video sequence.
Preferably, in the group behavior recognition method based on the relational graph analysis, in step S2, the sample frame P is sampled by using the target detection networksThe figure target in the system is extracted with characteristics, and the characteristic extraction process is that P issInputting the data into a target detection network for forward calculation, extracting the characteristics of the last convolution layer corresponding to the region where the character target is located, taking the characteristics as the characteristics of the character target and recording the characteristics as Fa(ii) a Then use a full connection layer, pair FaAnd performing dimensionality reduction operation.
Preferably, in the group behavior identification method based on the relational graph analysis, in step S3, for the sampling frame PsThe human target appearance feature and the position coordinate of the video sample frame obtained after the processing in step S2 are:
Figure BDA0002251668710000022
constructing a relational network graph in which
Figure BDA0002251668710000023
I.e. the feature after dimension reduction
Figure BDA0002251668710000024
liIs the central coordinate (x) of the area where the object is locatedi，yi)，
Specifically, for N personal object targets, a N x N relationship network graph G is calculated, wherein
Figure BDA0002251668710000025
And hl(li，lj) Respectively, the appearance and the position similarity are defined as follows:
Figure BDA0002251668710000026
wherein (x)i，yi) The coordinates of the center of the area where the human target is located; and
the relationship of target i relative to target j is normalized using the Softmax function, defined as follows:
Figure BDA0002251668710000028
preferably, in the group behavior recognition method based on the relational graph analysis, in step S3, after the construction of the relational network graph among the N human target objects is completed, the features of the individual human target are weighted and fused by using the convolutional neural network, and the convolutional neural network receives the feature after dimension reduction
Figure BDA0002251668710000029
And taking the relation network graph as input and output to a single personObject-object weighted fusion feature, denoted FgBy sampling the frame PsAll N personal object target features FgTaking an average operation to obtain PsThe single-frame group behavior expression characteristics are recorded as
Figure BDA00022516687100000210
Preferably, in the group behavior identification method based on the relational graph analysis, in step S4, the group behavior expression features of all the sampling frames are summed bitwise through the summation operation, so as to obtain the group behavior expression feature of the whole video, which is denoted as FvAnd FvAnd performing behavior category division through the full connection layer to obtain a classification result of the whole video and outputting the result.
Preferably, in the above group behavior identification method based on the relational graph analysis, the classification loss function in the classification is defined as follows:
Figure BDA00022516687100000211
wherein, yGFor the real value of the behavior category of the video group,
Figure BDA00022516687100000212
for model behavior class prediction, L1Is a cross entropy loss function.
According to another aspect of the invention, a group behavior identification system based on relational graph analysis is provided, which comprises a single-frame group behavior expression feature extraction module and a multi-frame behavior expression feature fusion and classification module, and is used for realizing the group behavior identification method, wherein the single-frame group behavior expression feature extraction module is used for extracting group behavior expression features from video frames obtained by sparse sampling; and the multi-frame behavior expression feature fusion and classification module is used for carrying out multi-frame fusion on the group behavior expression features extracted from the plurality of video sampling frames and constructing a classifier to classify the video behaviors.
Compared with the prior art, the invention has the beneficial effects that:
the group behavior identification method can realize the identification of the group behaviors in the video. Compared with the current method which adopts a manually designed graph model, the graph model based on appearance and position characteristics can automatically learn the relationship between pairs of individuals in an input video, and can be flexibly applied to wider occasions; meanwhile, compared with the traditional method for modeling the dynamic behavior characteristic by using the recurrent neural network, the method for sampling the video time axis provided by the invention can depict the behavior characteristic in a longer time period, and simultaneously, the characteristic extraction can be carried out on a plurality of time sampling frames in parallel, so that the characteristic that the recurrent neural network processes a plurality of frames in sequence is avoided, and the time efficiency of video processing is effectively improved.
Drawings
In order to more clearly illustrate the detailed description of the invention or the technical solutions in the prior art, the drawings that are needed in the detailed description of the invention or the prior art will be briefly described below.
FIG. 1 is a flow chart of a group behavior recognition method based on a relational graph analysis according to the present invention.
Fig. 2 is a schematic structural diagram of an overall neural network of the group behavior recognition method based on the relationship diagram analysis of the present invention.
Detailed Description
The principle of the method of the invention is as follows: 1.) constructing a relational graph model between figures according to the appearance and position relation between paired individuals in a video frame, and extracting group behavior relational feature expression of a video single frame by utilizing a graph convolution neural network on the basis; 2.) randomly extracting a plurality of frames from the whole video event segment by utilizing a video time axis sampling method, extracting the single-frame group behavior relation expression characteristics, and performing multi-frame characteristic fusion to represent the dynamic characteristics of group behaviors on the basis of the single-frame group behavior relation expression characteristics to realize video group behavior identification.
A group behavior recognition method based on relational graph analysis extracts group behavior expression characteristics by performing sparse sampling on videos and constructing a pairwise inter-individual appearance and position relational graph model, and realizes video group behavior recognition by fusing multi-frame group behavior expression characteristics and constructing a classifier. Specifically, a system for realizing the identification method can be decomposed into a single-frame group behavior expression feature extraction module and a multi-frame behavior expression feature fusion and classification module. The single-frame group behavior expression feature extraction module is used for extracting group behavior expression features from video frames obtained by sparse sampling, specifically used for constructing a graph model according to appearance and position relations among individuals and extracting the single-frame group behavior expression features by utilizing a graph convolution neural network, namely extracting the features of multi-person behaviors of a single frame; the multi-frame behavior expression feature fusion and classification module is used for performing multi-frame fusion on the group behavior expression features extracted from a plurality of video sampling frames and constructing a classifier to classify video behaviors, namely fusing the single-frame group behavior expression features in an addition mode and constructing a full-connection-layer classifier to classify the video behaviors.
Compared with the existing method, the method provided by the invention has two main improvements: 1.) a relation graph model is automatically constructed based on the appearance and position characteristics among individuals, and compared with the traditional graph model which is artificially constructed based on prior knowledge, the graph model provided by the invention can be more widely applied to different group behavior expressions; 2.) randomly sampling video frames, extracting single-frame group behavior expression characteristics, and then performing characteristic fusion to represent the whole video group event. Compared with the traditional behavior dynamic characteristic description method based on the recurrent neural network, the method has the advantages that the characteristic extraction processes of all sampling frames can be simultaneously carried out, the characteristic that the recurrent neural network sequentially processes the video frames is avoided, and the time efficiency of video processing is greatly improved.
FIG. 1 is a flow chart of the method for identifying group behavior based on graph analysis according to the present invention, which includes steps s 1-s 4; fig. 2 is a schematic diagram of the overall neural network structure of the group behavior recognition method based on the relationship diagram analysis of the present invention, and the steps s 1-s 4 are described as follows in conjunction with fig. 2:
and s1, carrying out sparse sampling on the video sequence containing the group behavior event to obtain a video sample frame as a representative of the video event.That is, a number of frames are sparsely extracted from the video sequence according to a specified rule as a representative of the video event. The rule may be random sampling or uniform sampling according to time sequence, and the sampled video frame is marked as PsAs shown at 1 in fig. 2.
And s2, obtaining the character target characteristics of the single-frame sampling frame through the target detection network and the dimensionality reduction operation. Specifically, the features of the human target in the sampling frame are extracted and compressed. Using a target detection network, such as the Faster R-CNN (ABS/1409.1556,2014) network 2 constructed based on VGG-16(KarenSimonyan and Andrew Zisserman. Very depth dependent image networks for big-scale image reception. CoRR, abs/1409.1556,2014), sampled frames P are processed using a sample detection network, such as the Faster R-CNN (Kaoqingren, Ross B. Girshick, and Jian Sun. Fastr-CNN: firmware real-time detection with real-scale projection networks. In NIPS, pages 91-99,2015) network 2sThe character object in (1) is extracted. The characteristic extraction process is that ofsInputting the data into a target detection network for forward calculation, extracting the characteristics of the last convolution layer corresponding to the region where the character target is located, taking the characteristics as the characteristics of the character target and recording the characteristics as Fa. For the target detection network 2 based on the VGG-16 network architecture, the human target feature is a feature corresponding to the 3 rd convolution unit of the 5 th group of convolution layers in the region where the human target is located. Then use a full connection layer 3, pair FaPerforming dimensionality reduction, i.e. after processing by the full connection layer 3
Figure BDA0002251668710000041
d is
Figure BDA0002251668710000042
The dimensions of the features, as shown in fig. 2, the fully connected layer 3 processes three different frames, which are exemplary embodiments only, and actually all sampled frames. PsAfter the processing in step s2 is completed, N object features and position coordinates thereof are obtained and recorded as:
Figure BDA0002251668710000043
wherein
Figure BDA0002251668710000044
I.e. the feature after dimension reduction
Figure BDA0002251668710000045
liIs the central coordinate (x) of the area where the object is locatedi，yi)。
And s3, constructing a graph model according to the appearance and the position relation among individuals and extracting the behavior expression characteristics of the single-frame group by utilizing a graph convolution neural network. In particular, for a sampled frame PsCharacter target appearance features and position coordinates:
Figure BDA0002251668710000051
constructing a relational network graph, extracting P by using graph convolution neural networksThe expressed group behavior characteristics. The relationship network graph may be constructed based on appearance and location characteristics between pairs of people. For N person object targets, a matrix G of N x N, i.e. a relational network diagram (shown as 8 in FIG. 2), is calculated to describe their relationship to each other, wherein
Figure BDA0002251668710000052
Figure BDA0002251668710000053
And □l(li，lj) Respectively, the appearance and the position similarity are defined as follows:
Figure BDA0002251668710000054
Figure BDA0002251668710000055
the relationship of target i relative to target j is normalized using the Softmax function, defined as follows:
after the construction of the relationship network graph G between N human target objects is completed, the characteristics of the single human target are weighted and fused by using a graph convolution neural network (GCN) (ThomasN. Kipf and Max welling. semi-collaborative classification with graphical connectivity network. CoRR, abs/1609.02907,2016.). As shown in operation 4 of FIG. 2, the atlas neural network receives the dimensionality reduced features
Figure BDA0002251668710000057
And the relation network graph G (8 in FIG. 2) is used as input, and the weighted fusion characteristic of the single person target is output and is marked as Fg. By sampling frames PsAll N personal object target features FgAn averaging operation, shown as operation 5 in FIG. 2, may result in PsThe single-frame group behavior expression characteristics are recorded as
Figure BDA0002251668710000058
And s4, obtaining the group behavior expression characteristics of the whole video by fusing the multi-frame group behavior characteristics. Specifically, single-frame group behavior expression features are obtained
Figure BDA0002251668710000059
Then, through the addition operation 6, the group behavior expression characteristics of all the sampling frames are summed according to the bit to obtain the group behavior expression characteristics of the whole video, which is marked as Fv。FvAnd the classification result of the whole video can be obtained by performing behavior classification through the full connection layer 7, and the result is output. The classification loss function is defined as follows:
Figure BDA00022516687100000510
wherein, yGFor the real value of the behavior category of the video group,
Figure BDA00022516687100000511
and predicting the value of the behavior category of the model. L is1Is a cross entropy loss function.
The above is a specific implementation scheme of the group behavior identification method based on the relational graph analysis provided by the invention. This embodiment was validated on the Volleyball (Mostafa S.Ibrahim, Srikanth muradharan, ZhiweiDeng, Arash Vahdat, and group Mori.A. longitudinal deep temporal model for group recognition. in CVPR, pages 1971-. Compared with the existing model, the method provided by the invention achieves the leading detection precision, and the specific comparison result is shown in the following table. In table 1, the higher the accuracy, the better the corresponding model, and the scheme provided by the present invention achieves the leading detection level.
TABLE 1 comparison of the method of the invention with existing methods in the classification of group behaviors
Figure BDA0002251668710000061
Comparative reference:
[1]Tianmin Shu,Sinisa Todorovic,and Song-Chun Zhu.CERN:confidence-energy recurrent network for group activity recognition.In CVPR,pages 4255–4263,2017.
[2]Mostafa S.Ibrahim and Greg Mori.Hierarchical relational networksfor group activity recognition and retrieval.In ECCV,pages 742–758,2018.
[3]Timur M.Bagautdinov,Alexandre Alahi,Franc，ois Fleuret,Pascal Fua,and Silvio Savarese.Social scene understanding:End-to-end multi-person actionlocalization and collective activity recognition.In CVPR,pages 3425–3434,2017.

Claims (8)

1. a group behavior identification method based on relational graph analysis is characterized by comprising the following steps:
s1 sparse sampling is carried out on the video sequence containing the group behavior event to obtain a video sampling frame which is used as a representative of the video event;
s2, obtaining the character target characteristics of the single-frame sampling frame through a target detection network and dimensionality reduction operation;
s3, constructing a graph model according to the appearance and position relation between individuals and extracting the behavior expression characteristics of the single-frame group by utilizing a graph convolution neural network; and
s4, obtaining the group behavior expression characteristics of the whole video by fusing the multi-frame group behavior characteristics.
2. The method for group behavior recognition based on relational graph analysis according to claim 1, wherein in step S1, random sampling or uniform sampling in time order sparsely extracts a plurality of frames from the video sequence.
3. The method for group behavior recognition based on relational graph analysis according to claim 1, wherein in step S2, sampling frames P are sampled by using a target detection networksThe figure target in the system is extracted with characteristics, and the characteristic extraction process is that P issInputting the data into a target detection network for forward calculation, extracting the characteristics of the last convolution layer corresponding to the region where the character target is located, taking the characteristics as the characteristics of the character target and recording the characteristics as Fa(ii) a Then use a full connection layer, pair FaAnd performing dimensionality reduction operation.
4. The method for group behavior recognition based on relational graph analysis according to claim 1, wherein in the step S3, for the sampling frame PsThe human target appearance feature and the position coordinate of the video sample frame obtained after the processing in step S2 are:
Figure FDA0002251668700000011
constructing a relational network graph in which
Figure FDA0002251668700000012
I.e. the feature after dimension reduction
Figure FDA0002251668700000013
liIs the central coordinate (x) of the area where the object is locatedi，yi)，
Specifically, for N personal object targets, a N x N relationship network graph G is calculated, wherein
Figure FDA0002251668700000014
Figure FDA0002251668700000015
And hl(li，lj) Respectively, the appearance and the position similarity are defined as follows:
Figure FDA0002251668700000016
Figure FDA0002251668700000017
wherein (x)i，yi) The coordinates of the center of the area where the human target is located; and
the relationship of target i relative to target j is normalized using the Softmax function, defined as follows:
Figure FDA0002251668700000018
5. the method for group behavior recognition based on relational graph analysis according to claim 4, wherein in step S3, after the construction of the relational network graph among N human target objects is completed, the weighted fusion is performed on the characteristics of the individual human target objects by using the graph convolution neural network, and the graph convolution neural network receives the characteristics after the dimension reduction
Figure FDA0002251668700000019
Taking the sum relation network graph as input, outputting the weighted fusion characteristic of the single person target, and recording the characteristic as FgBy sampling the frame PsAll N personal object featuresSign FgTaking an average operation to obtain PsThe single-frame group behavior expression characteristics are recorded as
Figure FDA0002251668700000021
6. The method for identifying group behaviors based on relational graph analysis according to claim 1, wherein in step S4, the group behavior expression features of all the sampling frames are summed bit by bit through the summation operation to obtain the group behavior expression feature of the whole video, which is denoted as FvAnd FvAnd performing behavior category division through the full connection layer to obtain a classification result of the whole video and outputting the result.
7. The method of claim 6, wherein the classification loss function in the classification is defined as follows:
Figure FDA0002251668700000022
wherein, yGFor the real value of the behavior category of the video group,
Figure FDA0002251668700000023
for model behavior class prediction, L1Is a cross entropy loss function.
8. A group behavior recognition system based on relational graph analysis, which is characterized by comprising a single-frame group behavior expression feature extraction module and a multi-frame behavior expression feature fusion and classification module, and is used for realizing the group behavior recognition method according to any one of claims 1 to 7, wherein,
the single-frame group behavior expression feature extraction module is used for extracting group behavior expression features from video frames obtained by sparse sampling; and
the multi-frame behavior expression feature fusion and classification module is used for performing multi-frame fusion on the group behavior expression features extracted from a plurality of video sampling frames and constructing a classifier to classify the video behaviors.
CN201911036597.4A 2019-10-29 2019-10-29 Group behavior recognition method based on relational graph analysis Active CN110796081B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911036597.4A CN110796081B (en) 2019-10-29 2019-10-29 Group behavior recognition method based on relational graph analysis

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911036597.4A CN110796081B (en) 2019-10-29 2019-10-29 Group behavior recognition method based on relational graph analysis

Publications (2)

Publication Number Publication Date
CN110796081A true CN110796081A (en) 2020-02-14
CN110796081B CN110796081B (en) 2023-07-21

Family

ID=69441845

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911036597.4A Active CN110796081B (en) 2019-10-29 2019-10-29 Group behavior recognition method based on relational graph analysis

Country Status (1)

Country Link
CN (1) CN110796081B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112131944A (en) * 2020-08-20 2020-12-25 深圳大学 Video behavior identification method and system
US20210216777A1 (en) * 2020-01-15 2021-07-15 Drishti Technologies, Inc. Almost unsupervised cycle and action detection

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011014810A1 (en) * 2009-07-30 2011-02-03 Northwestern University Systems, methods, and apparatus for reconstruction of 3-d object morphology, position, orientation and texture using an array of tactile sensors
CN105701467A (en) * 2016-01-13 2016-06-22 河海大学常州校区 Many-people abnormal behavior identification method based on human body shape characteristic
CN106241534A (en) * 2016-06-28 2016-12-21 西安特种设备检验检测院 Many people boarding abnormal movement intelligent control method
CN106529467A (en) * 2016-11-07 2017-03-22 南京邮电大学 Group behavior identification method based on multi-feature fusion
US20170169297A1 (en) * 2015-12-09 2017-06-15 Xerox Corporation Computer-vision-based group identification
CN110222719A (en) * 2019-05-10 2019-09-10 中国科学院计算技术研究所 A kind of character recognition method and system based on multiframe audio-video converged network
CN110245603A (en) * 2019-06-12 2019-09-17 成都信息工程大学 A kind of group abnormality behavior real-time detection method

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011014810A1 (en) * 2009-07-30 2011-02-03 Northwestern University Systems, methods, and apparatus for reconstruction of 3-d object morphology, position, orientation and texture using an array of tactile sensors
US20170169297A1 (en) * 2015-12-09 2017-06-15 Xerox Corporation Computer-vision-based group identification
CN105701467A (en) * 2016-01-13 2016-06-22 河海大学常州校区 Many-people abnormal behavior identification method based on human body shape characteristic
CN106241534A (en) * 2016-06-28 2016-12-21 西安特种设备检验检测院 Many people boarding abnormal movement intelligent control method
CN106529467A (en) * 2016-11-07 2017-03-22 南京邮电大学 Group behavior identification method based on multi-feature fusion
CN110222719A (en) * 2019-05-10 2019-09-10 中国科学院计算技术研究所 A kind of character recognition method and system based on multiframe audio-video converged network
CN110245603A (en) * 2019-06-12 2019-09-17 成都信息工程大学 A kind of group abnormality behavior real-time detection method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
谭程午;夏利民;王嘉;: "基于融合特征的群体行为识别", no. 01, pages 17 - 22 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210216777A1 (en) * 2020-01-15 2021-07-15 Drishti Technologies, Inc. Almost unsupervised cycle and action detection
US11875264B2 (en) * 2020-01-15 2024-01-16 R4N63R Capital Llc Almost unsupervised cycle and action detection
CN112131944A (en) * 2020-08-20 2020-12-25 深圳大学 Video behavior identification method and system
CN112131944B (en) * 2020-08-20 2023-10-17 深圳大学 Video behavior recognition method and system

Also Published As

Publication number Publication date
CN110796081B (en) 2023-07-21

Similar Documents

Publication Publication Date Title
CN108830157B (en) Human behavior identification method based on attention mechanism and 3D convolutional neural network
CN109255284B (en) Motion trajectory-based behavior identification method of 3D convolutional neural network
CN108985252B (en) Improved image classification method of pulse depth neural network
CN111814661A (en) Human behavior identification method based on residual error-recurrent neural network
CN110826447A (en) Restaurant kitchen staff behavior identification method based on attention mechanism
Liao et al. Triplet-based deep similarity learning for person re-identification
CN113591968A (en) Infrared weak and small target detection method based on asymmetric attention feature fusion
CN112597980B (en) Brain-like gesture sequence recognition method for dynamic vision sensor
Tang et al. Selective spatiotemporal features learning for dynamic gesture recognition
CN111276240A (en) Multi-label multi-mode holographic pulse condition identification method based on graph convolution network
CN110458235B (en) Motion posture similarity comparison method in video
He et al. Human emotion recognition in video using subtraction pre-processing
CN110796081A (en) Group behavior identification method based on relational graph analysis
Rehman et al. Deep learning for video classification: A review
CN110889335A (en) Human skeleton double-person interaction behavior recognition method based on multi-channel space-time fusion network
CN113936317A (en) Priori knowledge-based facial expression recognition method
Borra et al. Face recognition based on convolutional neural network
Wang et al. Single shot multibox detector with deconvolutional region magnification procedure
Dastbaravardeh et al. Channel Attention‐Based Approach with Autoencoder Network for Human Action Recognition in Low‐Resolution Frames
CN114863572B (en) Myoelectric gesture recognition method of multi-channel heterogeneous sensor
Arif et al. Video representation by dense trajectories motion map applied to human activity recognition
CN114882590B (en) Lip reading method based on event camera multi-granularity space-time feature perception
Huang et al. Group re-identification via transferred representation and adaptive fusion
Tong et al. Unconstrained Facial expression recognition based on feature enhanced CNN and cross-layer LSTM
CN115797827A (en) ViT human body behavior identification method based on double-current network architecture

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant