CN112507870A - Behavior recognition method and system through human skeleton extraction technology - Google Patents
Behavior recognition method and system through human skeleton extraction technology Download PDFInfo
- Publication number
- CN112507870A CN112507870A CN202011423108.3A CN202011423108A CN112507870A CN 112507870 A CN112507870 A CN 112507870A CN 202011423108 A CN202011423108 A CN 202011423108A CN 112507870 A CN112507870 A CN 112507870A
- Authority
- CN
- China
- Prior art keywords
- human body
- behavior
- feature
- human
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000605 extraction Methods 0.000 title claims abstract description 46
- 238000000034 method Methods 0.000 title claims abstract description 26
- 230000006399 behavior Effects 0.000 claims abstract description 167
- 230000008859 change Effects 0.000 claims abstract description 129
- 238000013528 artificial neural network Methods 0.000 claims abstract description 62
- 238000010801 machine learning Methods 0.000 claims abstract description 25
- 238000013135 deep learning Methods 0.000 claims description 15
- 239000011159 matrix material Substances 0.000 claims description 13
- 230000001133 acceleration Effects 0.000 claims description 8
- 239000000284 extract Substances 0.000 claims description 6
- 230000002159 abnormal effect Effects 0.000 claims description 3
- 230000009471 action Effects 0.000 description 10
- 238000004891 communication Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 6
- 230000036544 posture Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 210000002414 leg Anatomy 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 230000001681 protective effect Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 210000003423 ankle Anatomy 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 210000001513 elbow Anatomy 0.000 description 1
- 210000002683 foot Anatomy 0.000 description 1
- 230000005021 gait Effects 0.000 description 1
- 210000004247 hand Anatomy 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 210000003127 knee Anatomy 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003238 somatosensory effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 210000000707 wrist Anatomy 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/23—Recognition of whole body movements, e.g. for sport training
- G06V40/25—Recognition of walking or running movements, e.g. gait recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Psychiatry (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a behavior recognition method and system through a human skeleton extraction technology, and relates to the field of behavior recognition. A behavior recognition method through a human skeleton extraction technology comprises the following steps: establishing a human body model through human body characteristics and behavior characteristics, and extracting speed change characteristics of each joint point, geographical coordinate change characteristics of a human body, included angle characteristics among different joints and relative coordinate change characteristics among different joint points according to the human body model; the speed change characteristics, the geographic coordinate change, the included angle characteristics and the relative coordinate change characteristics of the human body are utilized to perform machine learning through a back propagation algorithm neural network to establish a neural network system model, and behavior recognition is performed through the neural network system model.
Description
Technical Field
The invention relates to the field of behavior recognition, in particular to a behavior recognition method and system through a human skeleton extraction technology.
Background
Extraction of human skeletons has been a very important research topic in the field of computer vision. The extraction of the information of the human skeleton has very important significance. First, the recognition of human skeleton can monitor the motion behavior of human, from which we can know the posture and behavior of human, such as: the position of the framework can be used for identifying actions of lifting hands, walking, lifting legs, running and the like of a person; secondly, the skeleton information of the human body can be used as intermediate information to help understand more complex human motion behaviors, such as: the leg skeleton information may be used for gait analysis of the person.
The human behavior recognition is the core of home accompanying, particularly for patients, old people and children, the human behavior is very complex, different behaviors can be presented for different people by the same behavior, and the effective recognition of the human behavior becomes a critical affair.
At present, in the prior art, traditional accelerometers, gyroscopes and the like are adopted for recognizing human behaviors, for example, a sensor is placed on a certain part of a human body, and the method for recognizing human behaviors is limited in behavior types, low in recognition accuracy, low in recognition speed, single in recognition behavior and poor in expandability.
Therefore, it is desirable to design a behavior recognition method and system using human skeleton extraction technology to improve the accuracy of human behavior recognition.
Disclosure of Invention
The invention aims to provide a behavior recognition method through a human skeleton extraction technology, which can meet the accompanying requirements of people and improve the accuracy of human behavior recognition.
Another object of the present invention is to provide a behavior recognition system by a human skeleton extraction technique, which can satisfy accompanying of people and improve accuracy of human behavior recognition.
The embodiment of the invention is realized by the following steps:
in a first aspect: the embodiment of the application provides a behavior identification method through a human skeleton extraction technology, which comprises the following steps: establishing a human body model through human body characteristics and behavior characteristics, and extracting speed change characteristics of each joint point, geographical coordinate change characteristics of a human body, included angle characteristics among different joints and relative coordinate change characteristics among different joint points according to the human body model; and performing machine learning by using the speed change characteristics, the geographical coordinate change, the included angle characteristics and the relative coordinate change characteristics of the human body through a back propagation algorithm neural network to establish a neural network system model, and performing behavior recognition by using the neural network system model.
In some embodiments of the present invention, the building of the human body model by the human body feature and the behavior feature comprises the following steps: acquiring human behavior original data, extracting three-dimensional coordinates of a human body for multiple times through the human behavior original data, and establishing the human body model by using the three-dimensional coordinates.
In some embodiments of the present invention, a demonstration animation of the human body model is generated according to different behavior characteristics, and the speed variation characteristic, the geographic coordinate characteristic, the angle characteristic, and the relative coordinate variation characteristic are extracted through the demonstration animation.
In some embodiments of the present invention, a kinect sensor is used to detect human behavior to obtain RGB video streams as the above-mentioned human behavior raw data.
In some embodiments of the present invention, acceleration variation characteristics of different joint points are extracted through the human body model, and the acceleration variation characteristics are added to perform machine learning through a back propagation algorithm neural network to establish the neural network system model.
In some embodiments of the present invention, the speed change feature, the geographic coordinate change feature, the included angle feature, and the relative coordinate change feature form an input matrix, the input matrix is encoded according to the behavior feature, and the encoded data is subjected to machine learning by using a back propagation algorithm neural network to establish a neural network system model.
In some embodiments of the present invention, whether a plurality of human body behaviors of the human body model are similar is determined according to the angle characteristic and the relative coordinate change characteristic, and whether each human body behavior is abnormal is analyzed according to the speed change characteristic and the geographic coordinate change of the similar plurality of human body behaviors.
In a second aspect, an embodiment of the present application provides a behavior recognition system by human skeleton extraction technology, which includes a data modeling module, a feature extraction module, a deep learning module, and a behavior recognition module, the data modeling module is used for establishing a human body model through human body characteristics and behavior characteristics, the characteristic extraction module is used for extracting speed change characteristics of each joint point, geographical coordinate change characteristics of a human body, included angle characteristics among different joints and relative coordinate change characteristics among different joint points according to the human body model, the deep learning module is used for performing machine learning to establish a neural network system model by using the speed change characteristic, the geographic coordinate change, the included angle characteristic and the relative coordinate change characteristic of the human body through a back propagation algorithm neural network, and the behavior recognition module is used for performing behavior recognition by using the neural network system model.
In some embodiments of the present invention, the data modeling module includes an animation demonstration module, the animation demonstration module generates demonstration animations of the human body model according to different behavior characteristics, and the characteristic extraction module extracts the speed variation characteristic, the geographic coordinate characteristic, the angle characteristic, and the relative coordinate variation characteristic through the demonstration animations.
In some embodiments of the present invention, the behavior recognition system using human skeleton extraction technology includes a feature coding module, where the feature coding module forms an input matrix with the speed change feature, the geographic coordinate change feature, the included angle feature, and the relative coordinate change feature, and codes the input matrix according to the behavior feature, and the coded data is machine-learned by using a back propagation algorithm neural network through the deep learning module to build a neural network system model.
Compared with the prior art, the embodiment of the invention has at least the following advantages or beneficial effects:
in a first aspect: the embodiment of the application provides a behavior identification method through a human skeleton extraction technology, which comprises the following steps: establishing a human body model through human body characteristics and behavior characteristics, and extracting speed change characteristics of each joint point, geographical coordinate change characteristics of a human body, included angle characteristics among different joints and relative coordinate change characteristics among different joint points according to the human body model; and performing machine learning by using the speed change characteristics, the geographical coordinate change, the included angle characteristics and the relative coordinate change characteristics of the human body through a back propagation algorithm neural network to establish a neural network system model, and performing behavior recognition by using the neural network system model.
With respect to the first aspect: according to the embodiment of the application, different types of human body models are established through human body characteristics and behavior characteristics, so that people of different types can be accompanied and cared for more peacefully using the human body models; the speed change characteristics of each joint point and the geographic coordinate change characteristics of the human body are extracted through the human body model, so that the monitoring of human body behaviors in different scenes is realized, the human body behaviors are conveniently predicted, and protective measures are taken in time; the included angle characteristics among different joints and the relative coordinate change characteristics among different joint points are extracted through the human body model, so that the posture and the action of human body behaviors are observed, and the recognition and the analysis are facilitated; the neural network is used for machine learning through the speed change characteristic, the geographic coordinate change characteristic, the included angle characteristic and the relative coordinate change characteristic by using a back propagation algorithm, so that a neural network system model is established, the accuracy of human behavior recognition is improved, the behavior recognition is carried out by using a deep application network system model, and the efficiency of human behavior recognition is improved.
In a second aspect, an embodiment of the present application provides a behavior recognition system by human skeleton extraction technology, which includes a data modeling module, a feature extraction module, a deep learning module, and a behavior recognition module, the data modeling module is used for establishing a human body model through human body characteristics and behavior characteristics, the characteristic extraction module is used for extracting speed change characteristics of each joint point, geographical coordinate change characteristics of a human body, included angle characteristics among different joints and relative coordinate change characteristics among different joint points according to the human body model, the deep learning module is used for performing machine learning to establish a neural network system model by using the speed change characteristic, the geographic coordinate change, the included angle characteristic and the relative coordinate change characteristic of the human body through a back propagation algorithm neural network, and the behavior recognition module is used for performing behavior recognition by using the neural network system model.
According to the embodiment of the application, the data modeling module is used for establishing the human body model by utilizing the human body characteristics and the behavior characteristics, so that people of different types can be conveniently accompanied and careless to use; the speed change characteristics of each joint point, the geographic coordinate change characteristics of the human body, the included angle characteristics of different joints and the relative coordinate change characteristics of different joint points are extracted through the characteristic extraction module according to the human body model, so that the monitoring of human body behaviors in different scenes is realized, the human body behaviors can be conveniently predicted, and protective measures can be taken in time; the speed change characteristics, the geographic coordinate change, the included angle characteristics and the relative coordinate change characteristics of the human body are utilized by the deep learning module to perform machine learning through a back propagation algorithm neural network to establish a neural network system model, so that the accuracy of human behavior identification is improved; the behavior recognition module is used for recognizing the behaviors by utilizing the neural network system model, so that the efficiency of recognizing the human body behaviors is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
FIG. 1 is a schematic flow chart of a behavior recognition method by human skeleton extraction technology according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a behavior recognition system by human skeleton extraction technology according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and the individual features of the embodiments can be combined with one another without conflict.
Example 1
Referring to fig. 1, fig. 1 is a schematic flow chart illustrating a behavior recognition method by human skeleton extraction technology according to an embodiment of the present application. Establishing a human body model through human body characteristics and behavior characteristics, and extracting speed change characteristics of each joint point, geographical coordinate change characteristics of a human body, included angle characteristics among different joints and relative coordinate change characteristics among different joint points according to the human body model; and performing machine learning by using the speed change characteristics, the geographical coordinate change, the included angle characteristics and the relative coordinate change characteristics of the human body through a back propagation algorithm neural network to establish a neural network system model, and performing behavior recognition by using the neural network system model.
In detail, the human body model is established through human body characteristics and behavior characteristics, wherein the human body characteristics can be any one or more of height, body type, age, hand length, leg length and foot length of the human body, and the behavior characteristics can be postures and actions of all parts of the human body. By establishing the human body model, virtual images of different human body characteristics and behavior characteristics can be simulated, and the behaviors of people can be conveniently identified, recorded and demonstrated.
In detail, the speed change characteristics and the geographic coordinate change characteristics of all the joint points are extracted according to the human body model, so that the speed and the positions of human body behaviors are obtained, and behavior habits and the conditions of old people, children, patients or other crowds are conveniently monitored. The geographic coordinate change characteristic can simulate the coordinate change position of a human body on the electronic map through the human body model, so that the geographic coordinate position is determined according to the moving distance and the route of the human body. Optionally, the change of the geographic coordinate is obtained by arranging a GPS positioning and displacement sensor on the human body.
In detail, the included angle characteristics among different joints and the point relative coordinate change characteristics of the different joints are extracted according to the human body model. Optionally, different joints may be identified by a human body model generated by human body characteristics, and two joint points may be obtained by a human body model generated by behavior characteristics. Judging the action amplitude of human behavior, such as joint points of elbows, wrists, knees and ankles. Optionally, the included angle feature is recorded by using the included angle formed by the joint point where the two adjacent joints coincide with the two joints respectively. Optionally, when the two extracted joints are not adjacent, the included angle characteristic can be recorded through the included angle formed by the intersection point of the extension lines of the two joints and the two joints, and the included angle can also be formed by selecting a point on the human body and respectively connecting the points of the two joints. Wherein the angle characteristic may include the angle size and range. The posture and the action of the human behavior can be obtained by utilizing the included angle characteristic between the two joints and the relative coordinate change characteristic between the two joint points, so that whether the posture and the action of the human behavior are normal or not can be conveniently judged.
In detail, a neural network system model is established by machine learning through a back propagation algorithm neural network by utilizing the speed change characteristics, the geographical coordinate change, the included angle characteristics and the relative coordinate change characteristics of the human body. Therefore, the relation among the speed change of each joint point of the human body, the geographic coordinate change of the human body action, the included angle between different joints and the relative coordinate change of different joint points is obtained, and the neural network system model is utilized to carry out behavior recognition, so that the human body behavior can be detected, the inertial behavior of the human body behavior can be analyzed, and the health and the safety of people can be maintained.
In some embodiments of the present invention, the building of the human body model by the human body feature and the behavior feature comprises the following steps: acquiring human behavior original data, extracting three-dimensional coordinates of a human body for multiple times through the human behavior original data, and establishing the human body model by using the three-dimensional coordinates.
In detail, the human body characteristics and the behavior characteristics extract three-dimensional coordinates of the human body for multiple times by acquiring human body behavior original data, so that a human body model is established according to different actions of the human body. The three-dimensional coordinates can be joint points at two ends of different joints and a plurality of continuous points of a human body peripheral connecting line, so that human body characteristics and behavior characteristics can be collected conveniently, human body models of different people are designed by utilizing the human body characteristics and the behavior characteristics, and personalized analysis and detection are carried out.
In some embodiments of the present invention, a demonstration animation of the human body model is generated according to different behavior characteristics, and the speed variation characteristic, the geographic coordinate characteristic, the angle characteristic, and the relative coordinate variation characteristic are extracted through the demonstration animation.
In detail, the demonstration animation of the human body model is generated through different behavior characteristics of the human body, and the speed change characteristics of all joint points, the geographical coordinate change characteristics of the human body, the included angle characteristics among different joints and the coordinate change characteristics of different joint points can be extracted through the demonstration animation, so that the human body behavior detection precision of the human body model is further improved.
In some embodiments of the present invention, a kinect sensor is used to detect human behavior to obtain RGB video streams as the above-mentioned human behavior raw data.
The human behavior original data detects the human behavior of a specific figure through a kinect sensor, so that three-dimensional coordinates are extracted by utilizing the collected RGB video stream, and a human body model is built by utilizing the three-dimensional coordinates. The kinect sensor is a 3D somatosensory camera, has the functions of instant dynamic capture, image identification, microphone input, voice identification, community interaction and the like, and can share pictures and information with other equipment through the Internet.
In some embodiments of the present invention, acceleration variation characteristics of different joint points are extracted through the human body model, and the acceleration variation characteristics are added to perform machine learning through a back propagation algorithm neural network to establish the neural network system model.
In detail, acceleration change characteristics of different joint points are extracted through a human body model, and the acceleration change characteristics are added into a back propagation algorithm neural network for machine learning, so that the research on human body behaviors is further deepened.
In some embodiments of the present invention, the speed change feature, the geographic coordinate change feature, the included angle feature, and the relative coordinate change feature form an input matrix, the input matrix is encoded according to the behavior feature, and the encoded data is subjected to machine learning by using a back propagation algorithm neural network to establish a neural network system model.
In detail, the speed change characteristic, the geographic coordinate change characteristic, the included angle characteristic and the row pair coordinate change characteristic form an input matrix and are coded, so that machine learning is performed according to different human behaviors, and the study on the human behaviors with the same or similar actions is facilitated.
In some embodiments of the present invention, whether a plurality of human body behaviors of the human body model are similar is determined according to the angle characteristic and the relative coordinate change characteristic, and whether each human body behavior is abnormal is analyzed according to the speed change characteristic and the geographic coordinate change of the similar plurality of human body behaviors.
In detail, the similarity of a plurality of human body behaviors of the human body model is judged through the included angle characteristic and the relative coordinate change characteristic, so that the difference of the human body behaviors is subjected to big data analysis according to a neural network system model generated by machine learning, and the accuracy of judging the human body behavior abnormity is improved.
Example 2
Referring to fig. 2, fig. 2 is a schematic diagram illustrating a behavior recognition system using a human skeleton extraction technique according to an embodiment of the present application. The behavior recognition system based on the human skeleton extraction technology comprises a data modeling module, a feature extraction module, a deep learning module and a behavior recognition module. The components can be electrically connected with each other through one or more communication buses or signal lines, so that data transmission or interaction is realized. The data modeling module is used for building a human body model through human body characteristics and behavior characteristics, the characteristic extraction module is used for extracting speed change characteristics of all joint points, geographic coordinate change characteristics of the human body, included angle characteristics among different joints and relative coordinate change characteristics among different joint points according to the human body model, the deep learning module is used for conducting machine learning through a back propagation algorithm neural network by utilizing the speed change characteristics, the geographic coordinate change, the included angle characteristics and the relative coordinate change characteristics of the human body to build a neural network system model, and the behavior recognition module is used for conducting behavior recognition through the neural network system model.
In detail, the behavior recognition system based on the human skeleton extraction technology further comprises a data acquisition module, wherein the data acquisition module is used for detecting human behaviors through a kinect sensor to acquire RGB video streams as human behavior original data and acquiring human features and behavior features through the human original data. Wherein raw data can be acquired using optical coding. The specific content of the human body characteristics and behavior characteristics obtained by the human body raw data is the same as the principle of the embodiment 1, and need not be described herein again. In detail, the data modeling module is in communication connection with the data acquisition module so as to acquire human body characteristics and behavior characteristics, and a human body model is established through the human body characteristics and the behavior characteristics. In detail, the characteristic extraction module is in communication connection with the data modeling module, and extracts the speed change characteristic of each joint point, the geographic coordinate change characteristic of a human body, the included angle characteristic between different joints and the relative coordinate change characteristic between different joint points by using a human body model. The deep learning module is in communication connection with the feature extraction module, so that machine learning is performed through a back propagation algorithm neural network according to the speed change feature, the geographic coordinate change, the included angle feature and the relative coordinate change feature, and a neural network system model is established. The behavior recognition module is in communication connection with the deep learning module, so that recognition is carried out by utilizing the neural network system model.
In some embodiments of the present invention, the data modeling module includes an animation demonstration module, the animation demonstration module generates demonstration animations of the human body model according to different behavior characteristics, and the characteristic extraction module extracts the speed variation characteristic, the geographic coordinate characteristic, the angle characteristic, and the relative coordinate variation characteristic through the demonstration animations.
In detail, the animation demonstration module is in communication connection with the data acquisition module, so that the human body characteristics and the behavior characteristics of the original data are acquired, and the demonstration animation of the human body model is generated according to different types of behavior characteristics. After a static human body model is generated according to the human body characteristics, a demonstration animation composed of a plurality of human body models is generated through the behavior characteristics. In detail, the feature extraction module is in communication connection with the animation demonstration module, so that the feature extraction module extracts speed change features, geographic coordinate features, included angle features and relative coordinate change features through demonstration of animations.
In some embodiments of the present invention, the behavior recognition system using human skeleton extraction technology includes a feature coding module, where the feature coding module forms an input matrix with the speed change feature, the geographic coordinate change feature, the included angle feature, and the relative coordinate change feature, and codes the input matrix according to the behavior feature, and the coded data is machine-learned by using a back propagation algorithm neural network through the deep learning module to build a neural network system model.
In detail, the feature coding module is in communication connection with the feature extraction module so as to obtain a speed change feature, a geographic coordinate change feature, an included angle feature and a relative coordinate change feature, and further form the speed change feature, the geographic coordinate change feature, the included angle feature and the relative coordinate change feature into an input matrix and carry out coding according to the behavior feature. In detail, the deep learning module is in communication connection with the characteristic coding module, so that the coded data are subjected to machine learning by using a back propagation algorithm neural network to establish a neural network system model.
It will be appreciated that the configuration shown in fig. 2 is merely illustrative and that a behavior recognition system via human skeleton extraction techniques may also include more or fewer components than shown in fig. 2, or have a different configuration than shown in fig. 2. The components shown in fig. 2 may be implemented in hardware, software, or a combination thereof.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
To sum up, the embodiment of the present application provides a behavior recognition method and system by human skeleton extraction technology:
according to the embodiment of the application, different types of human body models are established through human body characteristics and behavior characteristics, so that people of different types can be accompanied and cared for more peacefully using the human body models; the speed change characteristics of each joint point and the geographic coordinate change characteristics of the human body are extracted through the human body model, so that the monitoring of human body behaviors in different scenes is realized, the human body behaviors are conveniently predicted, and protective measures are taken in time; the included angle characteristics among different joints and the relative coordinate change characteristics among different joint points are extracted through the human body model, so that the posture and the action of human body behaviors are observed, and the recognition and the analysis are facilitated; the neural network is used for machine learning through the speed change characteristic, the geographic coordinate change characteristic, the included angle characteristic and the relative coordinate change characteristic by using a back propagation algorithm, so that a neural network system model is established, the accuracy of human behavior recognition is improved, the behavior recognition is carried out by using a deep application network system model, and the efficiency of human behavior recognition is improved.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
It will be evident to those skilled in the art that the present application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Claims (10)
1. A behavior recognition method through a human skeleton extraction technology is characterized by comprising the following steps: establishing a human body model through human body characteristics and behavior characteristics, and extracting speed change characteristics of each joint point, geographical coordinate change characteristics of a human body, included angle characteristics among different joints and relative coordinate change characteristics among different joint points according to the human body model; and performing machine learning by using the speed change characteristic, the geographical coordinate change, the included angle characteristic and the relative coordinate change characteristic of the human body through a back propagation algorithm neural network to establish a neural network system model, and performing behavior recognition by using the neural network system model.
2. The method for behavior recognition through human skeleton extraction technology as claimed in claim 1, wherein the step of building the human body model through the human body features and the behavior features comprises the following steps: acquiring human behavior original data, extracting three-dimensional coordinates of a human body for multiple times through the human behavior original data, and establishing the human body model by using the three-dimensional coordinates.
3. The behavior recognition method according to claim 2, wherein a demonstration animation of the human body model is generated according to different behavior features, and the speed variation feature, the geographic coordinate feature, the included angle feature, and the relative coordinate variation feature are extracted through the demonstration animation.
4. The method as claimed in claim 2, wherein a kinect sensor is used to detect human body behavior to obtain RGB video stream as the raw data of human body behavior.
5. The method as claimed in claim 1, wherein the acceleration variation characteristics of different joint points are extracted from the human body model, and the acceleration variation characteristics are added to perform machine learning by using a back propagation algorithm neural network to build the neural network system model.
6. The behavior recognition method through the human skeleton extraction technology as claimed in claim 1, wherein the speed variation feature, the geographic coordinate variation feature, the included angle feature and the relative coordinate variation feature form an input matrix and are encoded according to the behavior feature, and the encoded data are subjected to machine learning through a back propagation algorithm neural network to establish a neural network system model.
7. The behavior recognition method according to claim 1, wherein whether a plurality of human behaviors of the human body model are similar is determined according to the angle feature and the relative coordinate change feature, and whether each human behavior is abnormal is analyzed according to the speed change feature and the geographic coordinate change of the similar plurality of human behaviors.
8. A behavior recognition system by human skeleton extraction technology is characterized by comprising a data modeling module, a feature extraction module, a deep learning module and a behavior recognition module, the data modeling module is used for establishing a human body model through human body characteristics and behavior characteristics, the characteristic extraction module is used for extracting speed change characteristics of all joint points, geographic coordinate change characteristics of a human body, included angle characteristics among different joints and relative coordinate change characteristics among different joint points according to the human body model, the deep learning module is used for performing machine learning to establish a neural network system model by using the speed change characteristic, the geographic coordinate change, the included angle characteristic and the relative coordinate change characteristic of the human body through a back propagation algorithm neural network, and the behavior recognition module is used for performing behavior recognition by using the neural network system model.
9. The system of claim 8, wherein the data modeling module comprises an animation demonstration module, the animation demonstration module generates a demonstration animation of the human body model according to different behavior features, and the feature extraction module extracts the speed variation feature, the geographic coordinate feature, the included angle feature and the relative coordinate variation feature through the demonstration animation.
10. The behavior recognition system through the human skeleton extraction technology as claimed in claim 8, comprising a feature coding module, wherein the feature coding module forms the speed variation feature, the geographic coordinate variation feature, the included angle feature and the relative coordinate variation feature into an input matrix and codes the input matrix according to the behavior feature, and the coded data is subjected to machine learning through the deep learning module by using a back propagation algorithm neural network to build a neural network system model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011423108.3A CN112507870A (en) | 2020-12-08 | 2020-12-08 | Behavior recognition method and system through human skeleton extraction technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011423108.3A CN112507870A (en) | 2020-12-08 | 2020-12-08 | Behavior recognition method and system through human skeleton extraction technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112507870A true CN112507870A (en) | 2021-03-16 |
Family
ID=74971260
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011423108.3A Pending CN112507870A (en) | 2020-12-08 | 2020-12-08 | Behavior recognition method and system through human skeleton extraction technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112507870A (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104850846A (en) * | 2015-06-02 | 2015-08-19 | 深圳大学 | Human behavior recognition method and human behavior recognition system based on depth neural network |
| US20200322626A1 (en) * | 2017-12-19 | 2020-10-08 | Huawei Technologies Co., Ltd. | Image coding method, action recognition method, and action recognition apparatus |
-
2020
- 2020-12-08 CN CN202011423108.3A patent/CN112507870A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104850846A (en) * | 2015-06-02 | 2015-08-19 | 深圳大学 | Human behavior recognition method and human behavior recognition system based on depth neural network |
| US20200322626A1 (en) * | 2017-12-19 | 2020-10-08 | Huawei Technologies Co., Ltd. | Image coding method, action recognition method, and action recognition apparatus |
Non-Patent Citations (1)
| Title |
|---|
| 盛斌 等: "虚拟现实理论基础与应用开发实践", 上海交通大学出版社, pages: 47 - 57 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP4002198A1 (en) | Posture acquisition method and device, and key point coordinate positioning model training method and device | |
| CN104850846B (en) | A kind of Human bodys' response method and identifying system based on deep neural network | |
| KR102480362B1 (en) | cloud-based augmented reality | |
| Wang et al. | Geometric pose affordance: 3d human pose with scene constraints | |
| Min et al. | Support vector machine approach to fall recognition based on simplified expression of human skeleton action and fast detection of start key frame using torso angle | |
| CN107015646A (en) | The recognition methods of motion state and device | |
| KR20150039252A (en) | Apparatus and method for providing application service by using action recognition | |
| Zabulis et al. | Multicamera human detection and tracking supporting natural interaction with large-scale displays | |
| Nguyen et al. | Civil War battlefield experience: Historical event simulation using augmented reality technology | |
| CN114821004A (en) | Virtual space construction method, virtual space construction device, equipment and storage medium | |
| CN107016411B (en) | Data processing method and device | |
| KR20160049191A (en) | Wearable device | |
| Mahurkar | Integrating yolo object detection with augmented reality for ios apps | |
| CN116700471A (en) | Method and system for enhancing user experience of virtual reality system | |
| Wang et al. | Sign language recognition using Mediapipe | |
| Ludl et al. | Using simulation to improve human pose estimation for corner cases | |
| CN115546491B (en) | Fall alarm method, system, electronic equipment and storage medium | |
| CN112507870A (en) | Behavior recognition method and system through human skeleton extraction technology | |
| CN108986191B (en) | Character action generation method and device and terminal equipment | |
| Eom et al. | Data‐Driven Reconstruction of Human Locomotion Using a Single Smartphone | |
| Mastorakis | Human fall detection methodologies: from machine learning using acted data to fall modelling using myoskeletal simulation | |
| Moustakas et al. | Using modality replacement to facilitate communication between visually and hearing-impaired people | |
| CN115982452A (en) | Information recommendation method and device, electronic equipment and storage medium | |
| CN112818948B (en) | Behavior identification method based on visual attention under embedded system | |
| Narang et al. | Generating virtual avatars with personalized walking gaits using commodity hardware |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |