CN112396018A - Badminton player foul action recognition method combining multi-modal feature analysis and neural network - Google Patents

Badminton player foul action recognition method combining multi-modal feature analysis and neural network Download PDF

Info

Publication number
CN112396018A
CN112396018A CN202011364578.7A CN202011364578A CN112396018A CN 112396018 A CN112396018 A CN 112396018A CN 202011364578 A CN202011364578 A CN 202011364578A CN 112396018 A CN112396018 A CN 112396018A
Authority
CN
China
Prior art keywords
motion
neural network
network
optical flow
player
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011364578.7A
Other languages
Chinese (zh)
Other versions
CN112396018B (en
Inventor
张刚瀚
黄国恒
程良伦
张煜乾
陈泽炯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong University of Technology
Original Assignee
Guangdong University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong University of Technology filed Critical Guangdong University of Technology
Priority to CN202011364578.7A priority Critical patent/CN112396018B/en
Publication of CN112396018A publication Critical patent/CN112396018A/en
Application granted granted Critical
Publication of CN112396018B publication Critical patent/CN112396018B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/23—Recognition of whole body movements, e.g. for sport training
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Psychiatry (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Social Psychology (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a badminton foul action recognition method combining multi-modal characteristic analysis and a neural network, which comprises the following steps: extracting figure images, motion attitude sequences and optical flow data of athletes in real time; sending the figure image into a space flow network of a double-flow network to obtain spatial characteristics of an athlete; the motion attitude sequence is transmitted into a multilayer graph convolution neural network as a directed graph to obtain attitude space-time characteristics of the athlete during motion; extracting the characteristics of each frame of optical flow data through a convolutional neural network, and then sending the optical flow data into a time relation network to obtain the optical flow motion information characteristics of athletes; and pairing the three obtained characteristics in pairs respectively to obtain three aggregation characteristics, sending the aggregation characteristics into a convolutional neural network respectively to obtain three fusion characteristics, performing weighted fusion on the fusion characteristics to obtain the final overall human body multi-modal fusion motion characteristics, and sending the final overall human body multi-modal fusion motion characteristics into a full-connection network to obtain a final action classification recognition result. The invention improves the accuracy of identifying the foul action of the athletes.

Description

Badminton player foul action recognition method combining multi-modal feature analysis and neural network
Technical Field
The invention relates to the technical field of image processing, in particular to a badminton foul action recognition method combining multi-modal feature analysis and a neural network.
Background
The badminton match rules are clearly specified, the goal is to meet the standard action, the goal cannot be intentionally delayed, the sight of the opponent cannot be blocked by the standing position, the opponent cannot be intentionally invaded into the field area in the match, and certain actions are made to block the attack of the opponent or disperse the attention of the opponent. However, sometimes these actions are relatively subtle, so that human eyes cannot observe and judge the actions carefully, and in addition, the actions of athletes frequently change in the process of a game, so that the judgment may ignore certain foul actions, thereby affecting the fairness of the game. With the improvement of computer vision technology, the machine can realize the fine analysis of videos and images, so that the machine can be used for replacing human eyes to identify whether the match actions of the athletes are violated, namely, the action of the athletes is identified to judge whether the actions of the athletes meet the specifications during the match. The current behavior recognition method mainly uses a double-flow method and 3D convolution, and also carries out behavior recognition by inputting human body gestures. However, due to the uncertainty of human behaviors, the motions of athletes during a match are complex, the motions of the athletes during a hand swing may not be obvious, and some complex motions are mixed, which may cause misjudgment of the system on the behaviors, and the detailed analysis is difficult due to the lack of information interaction caused by the use of data of a single mode.
In the prior art, a chinese patent publication No. CN110705463A discloses a method and a system for identifying human body behaviors based on a multi-mode dual-stream 3D network in 1 month and 17 days 2020, including: a depth dynamic map sequence DDIS generated based on the depth video; a pose evaluation graph sequence PEMS generated based on the RGB video; respectively inputting the depth dynamic graph sequence and the pose evaluation graph sequence into a 3D convolutional neural network, and constructing a DDIS stream and a PEMS stream to obtain respective classification results; and fusing the obtained classification results to obtain a final behavior recognition result. The patent does not combine multi-feature information, does not have feature fusion, and has low recognition accuracy.
Disclosure of Invention
The invention provides the badminton foul motion recognition method combining the multi-modal characteristic analysis and the neural network, aiming at overcoming the defects that the foul motion recognition of athletes in the prior art does not have multi-characteristic information fusion and is low in recognition accuracy.
The primary objective of the present invention is to solve the above technical problems, and the technical solution of the present invention is as follows:
a badminton player foul action recognition method combining multi-modal feature analysis and a neural network comprises the following steps:
s1: extracting figure images, motion attitude sequences and optical flow data of athletes in real time;
s2: sending the figure image into a space flow network of a double-flow network to obtain spatial characteristics of an athlete;
s3: the motion attitude sequence is transmitted into a multilayer graph convolution neural network as a directed graph to obtain attitude space-time characteristics of the athlete during motion;
s4: extracting the characteristics of each frame of optical flow data through a convolutional neural network, and then sending the optical flow data into a time relation network to obtain the optical flow motion information characteristics of athletes;
s5: pairwise matching the three characteristics obtained in the steps S1, S2 and S3 respectively to obtain three polymerization characteristics;
s6: respectively sending the three aggregation characteristics into a convolutional neural network to obtain three fusion characteristics;
s7: weighting and fusing the three fusion characteristics to obtain final overall human body multi-modal fusion motion characteristics;
s8: and sending the multi-modal fusion motion characteristics of the whole human body into a full-connection network to obtain a final action classification recognition result.
Further, in step S1, a player character image is acquired by video frame capture, a player motion pose is acquired by openpos, and player optical flow data is acquired by DenseFlow.
Further, in step S2, the character image is sent to the spatial flow network of the dual-flow network, and the spatial information of the character image is modeled to obtain the character spatial characteristics of the athlete.
Further, in step S3, the motion gesture sequence is transmitted as a directed graph into a multi-layer graph convolution neural network, and the motion gesture sequence of the athlete is modeled to obtain the motion gesture spatio-temporal features of the athlete.
Further, the motion attitude space-time characteristic is obtained through graph convolution operation, and the graph convolution operation formula is as follows:
Figure BDA0002805052770000021
wherein v isti、vtjRepresenting a joint point of a human posture, finAnd foutRepresenting input and output images, w and w' representing weights between joint points and reconstructed weights, ltiBy using articulation point vtiTo assign to other nodes a digital label that depends on the shortest path, Z, between two nodesti(. is a regularization term; b (v)ti)＝{vtj|d(vtj,vti) ≧ D }, where D is set to a constant 1, D (v)tj,vti) Is the shortest path between two joint points.
Further, the process of obtaining the optical flow motion information characteristic of the athlete in step S4 is as follows:
modeling each frame of optical flow of the optical flow sequence by using a ResNet basic network in the convolutional neural network, and then fusing features of each frame of optical flow after modeling;
sending the optical flow sequence after the feature fusion into a time relation network to group according to different frame numbers, and sequencing the sequence numbers of the optical flows in each group from small to large;
modeling each group of optical flow sequences to obtain inter-frame time relation characteristics, and then fusing the inter-frame time relation characteristics of the same group to obtain inter-segment time relation characteristics;
and adding the time relation characteristics among all the segments to obtain the optical flow motion information characteristics of the whole athlete containing the time reasoning information.
Further, in step S6, the three aggregation features are respectively sent to the convolutional neural network, each aggregation feature contains a feature pair, and the convolutional neural network models and fuses each feature pair to obtain three fusion features.
Furthermore, the full-connection network carries out action recognition and classification on the input human body multi-mode fusion motion characteristics, and judges whether the sportsman has foul actions.
Further, the process of acquiring the motion gesture sequence includes:
acquiring a moving image sequence, and passing each picture of the image sequence through a VGG19 network to obtain image characteristics;
respectively obtaining joint point confidence coefficient of each joint point of the body of the athlete and affinity vectors among the joint points according to the image characteristics;
and clustering the joint points by using the joint point confidence coefficient and the affinity vectors among the joint points and carrying out skeleton splicing to obtain the motion posture sequence of the athlete.
Further, the human image is an RGB image.
Compared with the prior art, the technical scheme of the invention has the beneficial effects that:
according to the invention, different characteristics are obtained through the human image, the motion attitude sequence and the optical flow data, and the characteristics are fused and then are recognized in the full-connection network, so that the accuracy of foul recognition is improved.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Fig. 2 is a diagram of the network architecture of the present invention.
FIG. 3 is a diagram of a time relationship network architecture according to the present invention.
Detailed Description
In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described herein, and therefore the scope of the present invention is not limited by the specific embodiments disclosed below.
Example 1
As shown in fig. 1-2, a badminton player foul action recognition method combining multi-modal feature analysis and neural network includes the following steps:
s1: extracting figure images, motion attitude sequences and optical flow data of athletes in real time;
s2: sending the figure image into a space flow network of a double-flow network to obtain spatial characteristics of an athlete;
s3: the motion attitude sequence is transmitted into a multilayer graph convolution neural network (GCN) as a directed graph to obtain attitude space-time characteristics of the athlete during motion;
s4: extracting the characteristics of each frame of optical flow data through a convolutional neural network, and then sending the optical flow data into a time relation network to obtain the optical flow motion information characteristics of athletes;
s5: pairwise matching the three characteristics obtained in the steps S1, S2 and S3 respectively to obtain three polymerization characteristics;
s6: respectively sending the three aggregation characteristics into a convolutional neural network to obtain three fusion characteristics;
s7: weighting and fusing the three fusion characteristics to obtain final overall human body multi-modal fusion motion characteristics;
s8: and sending the multi-modal fusion motion characteristics of the whole human body into a full-connection network to obtain a final action classification recognition result.
Further, in step S1, a player character image is acquired by video frame capture, a player motion pose is acquired by openpos, and player optical flow data is acquired by DenseFlow.
In a specific embodiment, a plurality of cameras can be arranged to shoot a badminton game field in multiple angles, and character images are obtained by intercepting video frames in real time.
The process of acquiring the motion attitude sequence comprises the following steps:
acquiring a moving image sequence, and passing each picture of the image sequence through a VGG19 network with 10 layers to obtain an image characteristic F;
obtaining joint point confidence coefficient of each joint point of the body of the athlete according to the image characteristics
Figure BDA0002805052770000051
And affinity vector between joint points
Figure BDA0002805052770000052
Wherein j refers to the joint point subscript, c refers to the limb subscript number;
and clustering the joint points by using the joint point confidence coefficient and the affinity vectors among the joint points and carrying out skeleton splicing to obtain the motion posture sequence of the athlete.
And transmitting the obtained motion attitude sequence as a directed graph into a multilayer graph convolution neural network, and modeling the motion attitude sequence of the athlete to obtain the motion attitude space-time characteristics of the athlete.
More specifically, the motion posture sequence of the athlete is regarded as a graph structure, the vertex of the graph is a joint point in each frame, and the vertex set of the graph is represented as V ═ V { (V })tiI 1, T, i 1, N, the edges of the graph are the connecting edges between the respective joint points in each frame and the connecting edges of the corresponding joint points between the frames, and the set of the connecting edges between the respective joint points in each frame is denoted as Es＝{vtivtjL (i, j) belongs to H, and the connection edge set of the corresponding joint points between frames is represented as EF＝{vtiv(t+1)j}. Then the framework sequence is sequenced using the following equationAnd (3) line graph convolution:
Figure BDA0002805052770000053
vti、vtjrepresenting a joint point of a human posture, finAnd foutRepresenting input and output images, w and w' representing weights between joint points and reconstructed weights, ltiBy using articulation point vtiTo assign to other nodes a digital label that depends on the shortest path, Z, between two nodesti(. is a regularization term; b (v)ti)＝{vtj|d(vtj,vti) ≧ D }, where D is set to a constant 1, D (v)tj,vti) Is the shortest path between two joint points.
The invention is provided with 9 layers of space-time graph convolution arithmetic units, and finally the movement posture space-time characteristics PF of the athlete can be output.
In the invention, optical flow data is acquired through a DenseFlow toolkit, and the specific flow is as follows: every time two pictures are input, the first picture T (x, y) is a reference image, the second picture I (x, y) is a current image, and then the following objective functions are designed, so that each corresponding point on the two registered images is as same as possible:
Figure BDA0002805052770000061
u (x, y), v (x, y) are the amount of shift for each point on the image,
Figure BDA0002805052770000062
ψ (x) is an error function. Minimizing the objective function can output the optical flow sequence data of the sportsman in the competition.
After optical flow sequence data of the athlete is acquired, the invention adopts a spatial flow network in a double-flow architecture to extract spatial features for an image frame, as shown in fig. 3, a Time Relationship Network (TRN) is used to replace a time flow network to acquire optical flow motion information features, and the process of acquiring the optical flow motion information features of the athlete is as follows:
modeling each frame of optical flow of the optical flow sequence by using a ResNet base line network in a Convolutional Neural Network (CNN), and then fusing features of each frame of optical flow after modeling;
sending the optical flow sequence after feature fusion into a time relation network to be grouped according to different frame numbers (specifically, the optical flow sequence can be grouped according to a group of 2 frames, a group of 3 frames and a group of 4 frames), and sequencing the sequence numbers of the optical flows in each group from small to large;
modeling each group of optical flow sequences to obtain inter-frame time relation characteristics, and then fusing the inter-frame time relation characteristics of the same group to obtain inter-segment time relation characteristics;
and adding the time relation characteristics among all the segments to obtain the optical flow motion information characteristics of the whole athlete containing the time reasoning information.
Taking 3 frames as an example, the temporal relationship of 3 frames can be expressed as:
Figure BDA0002805052770000063
wherein h isφAnd gθThe method is realized by using a simple multilayer perceptron; optical flow motion information characteristics MF (V) ═ T of athlete2(V)+T3(V)+T4(V)。
Further, in step S6, the three aggregation features are respectively sent to the convolutional neural network, each aggregation feature contains a feature pair, and the convolutional neural network models and fuses each feature pair to obtain three fusion features.
In a specific embodiment, the obtained human body space characteristics SF, the optical flow motion information characteristics MF and the motion posture space-time characteristics PF of the badminton are paired pairwise and aggregated to obtain three different aggregation characteristics < SF, MF >, < PF, MF > and < PF, SF >. The three aggregated features are then input into the CNN module for modeling, resulting in Fusion features Fusion1, Fusion2, and Fusion3 for the paired modal features. Then the three fusion characteristics are weighted and fused to obtain the final human body multi-modal fusion characteristic, the characteristic comprises fusion characteristics of different combinations of three modes, information complementation can be realized among different modes, thus the obtained information is richer, the robustness of the characteristic is higher,
furthermore, the full-connection network carries out action recognition and classification on the input human body multi-mode fusion motion characteristics, and judges whether the sportsman has foul actions.
The same or similar reference numerals correspond to the same or similar parts;
the terms describing positional relationships in the drawings are for illustrative purposes only and are not to be construed as limiting the patent;
it should be understood that the above-described embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.

Claims (10)

1. A badminton player foul action recognition method combining multi-modal feature analysis and a neural network is characterized by comprising the following steps:
s1: extracting figure images, motion attitude sequences and optical flow data of athletes in real time;
s2: sending the figure image into a space flow network of a double-flow network to obtain spatial characteristics of an athlete;
s3: the motion attitude sequence is transmitted into a multilayer graph convolution neural network as a directed graph to obtain attitude space-time characteristics of the athlete during motion;
s4: extracting the characteristics of each frame of optical flow data through a convolutional neural network, and then sending the optical flow data into a time relation network to obtain the optical flow motion information characteristics of athletes;
s5: pairwise matching the three characteristics obtained in the steps S1, S2 and S3 respectively to obtain three polymerization characteristics;
s6: respectively sending the three aggregation characteristics into a convolutional neural network to obtain three fusion characteristics;
s7: weighting and fusing the three fusion characteristics to obtain final overall human body multi-modal fusion motion characteristics;
s8: and sending the multi-modal fusion motion characteristics of the whole human body into a full-connection network to obtain a final action classification recognition result.
2. The method for identifying the foul action of the badminton player by combining the multi-modal feature analysis and the neural network as claimed in claim 1, wherein in step S1, images of the player' S character are obtained by video frame capturing, the motion posture of the player is obtained by openpos, and the optical flow data of the player is obtained by DenseFlow.
3. The badminton player foul motion recognition method combining multi-modal feature analysis and neural network as claimed in claim 1, wherein step S2 is to send the character image into the spatial flow network of the dual-flow network to model the spatial information of the character image and obtain the spatial character feature of the player.
4. The badminton player foul motion recognition method combining multi-modal feature analysis and neural network as claimed in claim 1, wherein step S3 is to introduce the motion gesture sequence as a directed graph into a multi-layered graph convolution neural network, and model the motion gesture sequence of the player to obtain the motion gesture spatiotemporal features of the player.
5. The badminton player foul motion recognition method combining multi-modal feature analysis and neural network as claimed in claim 4, wherein the motion attitude spatiotemporal features are obtained through graph convolution operation, and the graph convolution operation formula is as follows:
Figure FDA0002805052760000021
wherein v isti、vtjRepresenting a joint point of a human posture, finAnd foutRepresenting input and output images, w and w' representing weights between joint points and reconstructed weights, ltiBy using articulation point vtiTo assign to other nodes a digital label that depends on the shortest path, Z, between two nodesti(. is a regularization term; b (v)ti)＝{vtj|d(vtj,vti) ≧ D }, where D is set to a constant 1, D (v)tj,vti) Is the shortest path between two joint points.
6. The badminton player foul motion recognition method combining multi-modal feature analysis and neural network as claimed in claim 1, wherein the step S4 of obtaining the optical flow motion information features of the player is as follows:
modeling each frame of optical flow of the optical flow sequence by using a ResNet basic network in the convolutional neural network, and then fusing features of each frame of optical flow after modeling;
sending the optical flow sequence after the feature fusion into a time relation network to group according to different frame numbers, and sequencing the sequence numbers of the optical flows in each group from small to large;
modeling each group of optical flow sequences to obtain inter-frame time relation characteristics, and then fusing the inter-frame time relation characteristics of the same group to obtain inter-segment time relation characteristics;
and adding the time relation characteristics among all the segments to obtain the optical flow motion information characteristics of the whole athlete containing the time reasoning information.
7. The badminton player foul motion recognition method combining multi-modal feature analysis and neural network as claimed in claim 1, wherein in step S6, three aggregated features are respectively fed into the convolutional neural network, each aggregated feature contains a feature pair, and modeling and fusion of each feature pair by the convolutional neural network results in three fused features.
8. The badminton player foul action recognition method combining multi-modal feature analysis and neural network as claimed in claim 1, wherein the fully connected network performs action recognition classification on the input human multi-modal fusion motion features to judge whether the player has foul action.
9. The badminton player foul motion recognition method combining multi-modal feature analysis and neural network according to claim 1, wherein the process of obtaining the motion gesture sequence comprises:
acquiring a moving image sequence, and passing each picture of the image sequence through a VGG19 network to obtain image characteristics;
respectively obtaining joint point confidence coefficient of each joint point of the body of the athlete and affinity vectors among the joint points according to the image characteristics;
and clustering the joint points by using the joint point confidence coefficient and the affinity vectors among the joint points and carrying out skeleton splicing to obtain the motion posture sequence of the athlete.
10. The badminton player foul motion recognition method combining multi-modal feature analysis and neural network as claimed in claim 1, wherein the character image is an RGB image.
CN202011364578.7A 2020-11-27 2020-11-27 Badminton player foul action recognition method combining multi-mode feature analysis and neural network Active CN112396018B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011364578.7A CN112396018B (en) 2020-11-27 2020-11-27 Badminton player foul action recognition method combining multi-mode feature analysis and neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011364578.7A CN112396018B (en) 2020-11-27 2020-11-27 Badminton player foul action recognition method combining multi-mode feature analysis and neural network

Publications (2)

Publication Number Publication Date
CN112396018A true CN112396018A (en) 2021-02-23
CN112396018B CN112396018B (en) 2023-06-06

Family

ID=74605505

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011364578.7A Active CN112396018B (en) 2020-11-27 2020-11-27 Badminton player foul action recognition method combining multi-mode feature analysis and neural network

Country Status (1)

Country Link
CN (1) CN112396018B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113239766A (en) * 2021-04-30 2021-08-10 复旦大学 Behavior recognition method based on deep neural network and intelligent alarm device
CN113239897A (en) * 2021-06-16 2021-08-10 石家庄铁道大学 Human body action evaluation method based on space-time feature combination regression
CN117333947A (en) * 2023-10-18 2024-01-02 首都体育学院 Badminton action analysis method and system
CN117953588A (en) * 2024-03-26 2024-04-30 南昌航空大学 Badminton player action intelligent recognition method integrating scene information

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109460707A (en) * 2018-10-08 2019-03-12 华南理工大学 A kind of multi-modal action identification method based on deep neural network
CN110096950A (en) * 2019-03-20 2019-08-06 西北大学 A kind of multiple features fusion Activity recognition method based on key frame
CN110705463A (en) * 2019-09-29 2020-01-17 山东大学 Video human behavior recognition method and system based on multi-mode double-flow 3D network
CN110892408A (en) * 2017-02-07 2020-03-17 迈恩德玛泽控股股份有限公司 Systems, methods, and apparatus for stereo vision and tracking
CN111259804A (en) * 2020-01-16 2020-06-09 合肥工业大学 Multi-mode fusion sign language recognition system and method based on graph convolution

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110892408A (en) * 2017-02-07 2020-03-17 迈恩德玛泽控股股份有限公司 Systems, methods, and apparatus for stereo vision and tracking
CN109460707A (en) * 2018-10-08 2019-03-12 华南理工大学 A kind of multi-modal action identification method based on deep neural network
CN110096950A (en) * 2019-03-20 2019-08-06 西北大学 A kind of multiple features fusion Activity recognition method based on key frame
CN110705463A (en) * 2019-09-29 2020-01-17 山东大学 Video human behavior recognition method and system based on multi-mode double-flow 3D network
CN111259804A (en) * 2020-01-16 2020-06-09 合肥工业大学 Multi-mode fusion sign language recognition system and method based on graph convolution

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113239766A (en) * 2021-04-30 2021-08-10 复旦大学 Behavior recognition method based on deep neural network and intelligent alarm device
CN113239897A (en) * 2021-06-16 2021-08-10 石家庄铁道大学 Human body action evaluation method based on space-time feature combination regression
CN113239897B (en) * 2021-06-16 2023-08-18 石家庄铁道大学 Human body action evaluation method based on space-time characteristic combination regression
CN117333947A (en) * 2023-10-18 2024-01-02 首都体育学院 Badminton action analysis method and system
CN117333947B (en) * 2023-10-18 2024-05-10 首都体育学院 Badminton action analysis method and system
CN117953588A (en) * 2024-03-26 2024-04-30 南昌航空大学 Badminton player action intelligent recognition method integrating scene information

Also Published As

Publication number Publication date
CN112396018B (en) 2023-06-06

Similar Documents

Publication Publication Date Title
CN112396018A (en) Badminton player foul action recognition method combining multi-modal feature analysis and neural network
Reddy et al. Tessetrack: End-to-end learnable multi-person articulated 3d pose tracking
Cai et al. Temporal hockey action recognition via pose and optical flows
CN110472612B (en) Human behavior recognition method and electronic equipment
CN111291885A (en) Near-infrared image generation method, network generation training method and device
CN112131908A (en) Action identification method and device based on double-flow network, storage medium and equipment
CN111767866B (en) Human body model creation method and device, electronic equipment and storage medium
CN112906604A (en) Behavior identification method, device and system based on skeleton and RGB frame fusion
CN111046734A (en) Multi-modal fusion sight line estimation method based on expansion convolution
CN109961039A (en) A kind of individual&#39;s goal video method for catching and system
CN112036260A (en) Expression recognition method and system for multi-scale sub-block aggregation in natural environment
CN113221663A (en) Real-time sign language intelligent identification method, device and system
Zhang et al. Deep monocular 3d human pose estimation via cascaded dimension-lifting
Kumar et al. Human pose estimation using deep learning: review, methodologies, progress and future research directions
Hsieh et al. Online human action recognition using deep learning for indoor smart mobile robots
CN114093024A (en) Human body action recognition method, device, equipment and storage medium
US20220148296A1 (en) Method and system for symmetric recognition of handed activities
CN117152829A (en) Industrial boxing action recognition method of multi-view self-adaptive skeleton network
CN115841602A (en) Construction method and device of three-dimensional attitude estimation data set based on multiple visual angles
CN110096989A (en) Image processing method and device
Saif et al. Aggressive action estimation: a comprehensive review on neural network based human segmentation and action recognition
CN114897678A (en) Infant eye ground retina panoramic image generation, acquisition and feedback method and system
Lessa et al. SoccerKicks: a Dataset of 3D dead ball kicks reference movements for humanoid robots
Meng et al. 3D human pose estimation based on a fully connected neural network with adversarial learning prior knowledge
Ascenso Development of a non-invasive motion capture system for swimming biomechanics

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant