CN115601837A - Action recognition method and device - Google Patents

Action recognition method and device Download PDF

Info

Publication number
CN115601837A
CN115601837A CN202211297199.XA CN202211297199A CN115601837A CN 115601837 A CN115601837 A CN 115601837A CN 202211297199 A CN202211297199 A CN 202211297199A CN 115601837 A CN115601837 A CN 115601837A
Authority
CN
China
Prior art keywords
data
action
motion
target
sub
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211297199.XA
Other languages
Chinese (zh)
Inventor
杨斌
张吉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Sports Co ltd
Original Assignee
Hangzhou Sports Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Sports Co ltd filed Critical Hangzhou Sports Co ltd
Priority to CN202211297199.XA priority Critical patent/CN115601837A/en
Publication of CN115601837A publication Critical patent/CN115601837A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Human Computer Interaction (AREA)
  • Social Psychology (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

The present specification provides a motion recognition method and a device, wherein the motion recognition method includes: acquiring initial action data corresponding to an action to be identified; determining a characteristic value corresponding to the initial action data, and creating a data segmentation interval according to the characteristic value; processing the initial action data into target action data according to the data segmentation interval, and constructing action characteristics according to the target action data; and inputting the motion characteristics into a motion recognition model for processing to obtain motion information corresponding to the motion to be recognized. When the action to be recognized is recognized, the characteristic information in the initial action data can be fully combined, so that the action information of the action to be recognized can be accurately predicted by the model, and the accuracy of action recognition is effectively improved.

Description

Action recognition method and device
Technical Field
The present disclosure relates to the field of motion recognition technologies, and in particular, to a motion recognition method. The present specification also relates to a motion recognition apparatus, a computing device, and a computer-readable storage medium.
Background
With the development of internet technology, motion recognition technology is applied in more and more scenes; such as a human-computer interaction scene, a monitoring scene or an online education scene, so as to provide corresponding business services to the user by recognizing the action of the user. In the prior art, when the action of a user is identified, most of the actions are identified by adopting an image identification mode, namely, an image containing the action to be identified is collected and then the image is processed to obtain the action type; although the solution can fulfill the requirement of motion recognition, the solution cannot be implemented when the image capturing device is absent, and a large amount of computing resources are consumed from the viewpoint of image processing dimension, so an effective solution is needed to solve the above problem.
Disclosure of Invention
In view of this, the embodiments of the present disclosure provide a method for recognizing an action. The present specification also relates to an action recognition apparatus, a computing device, and a computer-readable storage medium, which are used to solve the technical problems in the prior art.
According to a first aspect of embodiments herein, there is provided a motion recognition method, including:
acquiring initial action data corresponding to an action to be identified;
determining a characteristic value corresponding to the initial action data, and creating a data segmentation interval according to the characteristic value;
processing the initial action data into target action data according to the data segmentation interval, and constructing action characteristics according to the target action data;
and inputting the motion characteristics into a motion recognition model for processing to obtain motion information corresponding to the motion to be recognized.
According to a second aspect of embodiments herein, there is provided a motion recognition apparatus including:
the data acquisition module is configured to acquire initial action data corresponding to the action to be recognized;
the creating interval module is configured to determine a characteristic value corresponding to the initial action data and create a data dividing interval according to the characteristic value;
the construction characteristic module is configured to process the initial action data into target action data according to the data segmentation interval and construct action characteristics according to the target action data;
and the model processing module is configured to input the motion characteristics into a motion recognition model for processing, and obtain motion information corresponding to the motion to be recognized.
According to a third aspect of embodiments herein, there is provided a computing device comprising:
a memory and a processor;
the memory is configured to store computer-executable instructions, and the processor is configured to implement the steps of the motion recognition method when executing the computer-executable instructions.
According to a fourth aspect of embodiments herein, there is provided a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the motion recognition method.
In the action recognition method provided by the specification, after initial action data corresponding to an action to be recognized is acquired, in order to improve the action recognition accuracy, a characteristic value corresponding to the initial action data can be determined at the moment, and then a data partition interval for processing the initial action data is created according to the characteristic value; at the moment, the initial action data can be processed into target action data according to the data segmentation interval, and then action features are constructed on the basis of the target action data; and finally, inputting the motion characteristics into a pre-trained motion recognition model for processing, so as to obtain motion information corresponding to the motion to be recognized and output by the motion recognition model. When the action to be recognized is recognized, the characteristic information in the initial action data can be fully combined, so that the action information of the action to be recognized can be accurately predicted by the model, and the accuracy of action recognition is effectively improved.
Drawings
Fig. 1 is a schematic diagram of a motion recognition method according to an embodiment of the present disclosure;
fig. 2 is a flowchart of a method for recognizing an action according to an embodiment of the present disclosure;
fig. 3 is a schematic diagram of a peak value in a motion recognition method according to an embodiment of the present disclosure
FIG. 4 is a flowchart illustrating a method for motion recognition according to an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of a motion recognition device according to an embodiment of the present disclosure;
fig. 6 is a block diagram of a computing device according to an embodiment of the present disclosure.
Detailed Description
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present description. This description may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, as those skilled in the art will be able to make and use the present disclosure without departing from the spirit and scope of the present disclosure.
The terminology used in the description of the one or more embodiments is for the purpose of describing the particular embodiments only and is not intended to be limiting of the description of the one or more embodiments. As used in one or more embodiments of the present specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present specification refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It will be understood that, although the terms first, second, etc. may be used herein in one or more embodiments to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first can also be referred to as a second and, similarly, a second can also be referred to as a first without departing from the scope of one or more embodiments of the present description. The word "if," as used herein, may be interpreted as "at … …" or "at … …" or "in response to a determination," depending on the context.
First, the noun terms referred to in one or more embodiments of the present specification are explained.
An IMU: the (inertial measurement unit) is used for measuring the three-axis attitude angle (or angular velocity) and acceleration of the object.
In the present specification, a motion recognition method is provided, and the present specification relates to a motion recognition apparatus, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.
Referring to the schematic diagram shown in fig. 1, after motion data corresponding to a motion to be recognized is collected through the wearable device 100 of the user, in order to improve the motion recognition accuracy, a feature value corresponding to initial motion data may be determined at this time, and then a data segmentation interval for processing the initial motion data is created according to the feature value; at the moment, the action data can be processed into action data capable of being subjected to action analysis according to the data division interval, and then action characteristics are constructed on the basis of the part of action data; and finally, inputting the motion characteristics into a pre-trained motion recognition model for processing, so as to obtain motion information corresponding to the motion to be recognized and output by the motion recognition model. When the action to be recognized is recognized, the characteristic information in the action data can be fully combined, so that the action information of the action to be recognized can be accurately predicted by the model, and the accuracy of action recognition is effectively improved.
Fig. 2 is a flowchart illustrating a method for recognizing an action according to an embodiment of the present disclosure, which specifically includes the following steps:
step S202, collecting initial action data corresponding to the action to be recognized.
The motion recognition method provided by the embodiment can be applied to scenes such as motion recognition of sports, motion recognition of man-machine interaction, motion recognition of online education and the like; action data of the user are collected for action analysis, so that action information of the user is determined in response to an action analysis result, and downstream services can make accurate response according to the action information. For example, in a motion recognition scene of sports, by recognizing the motion of the user, an image capturing terminal (such as a mobile phone, a tablet or an intelligent monitor) may be triggered to capture the wonderful moment of the user. Or in the action recognition scene of man-machine interaction, by recognizing the action of the user, the computer can be triggered to respond to the recognized action to provide services for the user, such as controlling game characters to release skills. In addition, or in an action recognition scene of online education, by recognizing the action of the user, the computer can be triggered to respond to the recognized action and display corresponding services through the display unit, such as displaying teaching aids through a screen.
Based on this, the action to be recognized specifically means that the user moves the head, the feet, the body, or the hands at the current moment, and the action needs to be recognized, so that the downstream service can respond to the recognized action accurately. Correspondingly, the initial motion data specifically refers to data acquired by a sensor after a user puts out a motion to be recognized, and the data corresponds to data such as acceleration, angular velocity and the like, and is used for performing motion recognition processing operation on the basis of the data.
The present embodiment is described by taking an application of the motion recognition method in a football motion recognition scene as an example, and the same or corresponding description contents may be referred to in the motion recognition process in other scenes, which is not described herein again.
Furthermore, when the initial action data corresponding to the action to be recognized is collected, the action information of the action to be recognized is analyzed from the initial action data in consideration of follow-up requirements, and therefore the covering dimension of the initial action data determines the accuracy of action recognition; therefore, in order to improve the recognition accuracy, the acceleration data and the gyroscope data may be collected to form the initial motion data, and in this embodiment, the specific implementation manner is as follows:
acquiring acceleration data and gyroscope data of a target duration through a motion sensor; and taking the acceleration data and the gyroscope data as the initial motion data.
Specifically, the motion capture sensor specifically refers to a sensor capable of capturing acceleration data and gyroscope data, and includes, but is not limited to, a sensor capable of separately capturing acceleration data, a sensor capable of separately capturing gyroscope data, or a sensor capable of simultaneously capturing acceleration data and gyroscope data, and preferably, the acceleration data and gyroscope data may be captured by an IMU sensor. It should be noted that, in order to accurately collect the motion data of the user, the motion sensor is installed on the wearable device of the user, so as to realize interaction between the sensor and the processor on the wearable device, and realize subsequent motion recognition processing operation. The wearable device includes, but is not limited to, a mobile terminal, a bracelet, a foot ring, and the like, and this embodiment is not limited in any way herein.
Based on this, when wearing the user motion of installing the wearable equipment of motion sensor, can gather the acceleration data and the gyroscope data of target duration through the motion sensor this moment, regard it as the initial action data of action of treating discernment to make things convenient for the follow-up can combine acceleration data and gyroscope data analysis to show the current action type of user. The time length of the target duration may be set according to an actual application scenario, for example, set to be 400ms, 450ms, or 600ms, and the like, which is not limited herein.
For example, a user first wears a foot ring to participate in football, in the process of the user first, initial action data with the duration of 600ms is collected through an IMU sensor installed in the foot ring, the data comprises 6-axis data, and the data are acceleration data: ACC _ x, ACC _ y and ACC _ z, and gyro data gyro _ row (rotating around the x-axis), gyro _ pitch (rotating around the y-axis) and gyro _ yaw (rotating around the z-axis) to facilitate subsequent analysis of the type of soccer action at this time by user a in conjunction with the 6-axis data.
In conclusion, the acceleration data and the gyroscope data of the target duration are collected through the motion sensor and are used as the initial motion data of the motion to be recognized, so that the motion to be recognized can be conveniently and accurately recognized subsequently, and the precision of motion recognition is guaranteed.
Step S204, determining a characteristic value corresponding to the initial action data, and creating a data segmentation interval according to the characteristic value.
Specifically, after the initial motion data corresponding to the motion to be recognized is collected, further, it is considered that the motion to be recognized needs to be analyzed in combination with the motion recognition model subsequently, and the initial motion data is motion data information corresponding to a period of time, so that the initial motion data not only includes data capable of analyzing the motion information of the motion to be recognized, but also includes motion data corresponding to some slight motions after the motion to be recognized is completed.
The characteristic value specifically refers to a data peak value corresponding to the initial motion data, that is, a maximum value of the acceleration or the angular velocity of the motion to be recognized at a certain time within a time interval corresponding to the initial motion data, and the maximum value can fully represent the motion variation amplitude of the motion to be recognized. For example, when a user takes off a jump, the user gradually rises along the z-axis acceleration in the process from the starting moment of the jump to the highest point and then falls to the ground from the highest point, and after the user reaches a certain point, the user gradually falls along the time, a peak value corresponding to the acceleration data is generated in the process, the peak value can represent that the motion amplitude of the user in the z-axis direction is large, and the operation of action recognition processing can be completed by combining an action recognition model subsequently. Therefore, by determining the characteristic value of the initial action data, the interval with the largest motion amplitude change of the action to be recognized, namely the data segmentation interval, can be conveniently determined on the basis of the characteristic value, so that the target action data can be conveniently intercepted from the initial action data subsequently and is used for analyzing the action type of the action to be recognized. Correspondingly, the data division interval specifically refers to an interval for processing the initial motion data, for intercepting the motion data that can represent the motion to be recognized most, and belongs to the time interval.
Based on this, after the initial action data corresponding to the action to be recognized is collected, the characteristic value corresponding to the initial action data can be determined at this time, and then a data segmentation interval capable of segmenting the initial action data is created by combining the characteristic value, so that the action data capable of representing the action to be recognized can be conveniently intercepted from the initial action data, and then the processing operation of action recognition can be completed by combining a model.
In practical application, the initial action data is considered to contain action data of multiple dimensions, and the action data of each dimension represents the action change condition of a user; in general, no matter how the motion to be recognized of the user changes, the motion data of a certain dimension changes, and at this time, the motion data of the dimension may be selected as the reference motion data, and the motion information of the motion to be recognized may be preliminarily determined based on the reference motion data, so that the motion information may be preliminarily located in a short time, and the length of the motion recognition cycle may be reduced, in this embodiment, the specific implementation manner is as follows:
extracting reference motion data from the initial motion data; determining a reference amplitude value corresponding to the reference motion data; executing the step S204 when the reference amplitude is greater than a preset amplitude threshold; and under the condition that the reference amplitude is smaller than or equal to a preset amplitude threshold, determining preliminary action information of the action to be recognized based on the reference action data.
Specifically, the reference motion data specifically refers to data that can represent any motion to be recognized in the initial motion data, and the determination of the reference motion data may be selected according to an actual application scenario, which is not limited in this embodiment. For example, when a soccer player runs on a field, if the foot motion force and the rotation angular velocity are small, the data of the axis rotating around the y axis in the gyroscope data can be selected as the reference motion data, which can fully represent that the motion type of the soccer player at the current moment is the dribbling. Correspondingly, the reference amplitude specifically refers to a maximum absolute value of the instantaneous occurrence of the reference motion data in a period; correspondingly, the preset amplitude threshold specifically refers to a threshold for preliminarily identifying the motion information of the motion to be identified, and the size of the preset amplitude threshold may be set according to an actual application scenario, which is not limited herein.
Based on the above, after the initial action data corresponding to the action to be recognized is acquired, as the initial action data comprises axis data of multiple dimensions, the reference action data capable of representing the action to be recognized can be extracted from the initial action data according to an action recognition scene, and the reference amplitude corresponding to the reference action data is determined; comparing the reference amplitude with a preset amplitude threshold value; if the reference amplitude is smaller than or equal to the preset amplitude threshold value, the action to be recognized is similar to the basic action, and then the preliminary action information of the action to be recognized can be directly determined based on the reference action data; if the reference amplitude is greater than the preset amplitude threshold, which indicates that the difference between the motion to be recognized and the basic motion is large, step S204 may be executed to complete the motion recognition processing operation subsequently by combining with the motion recognition model.
In summary, by performing preliminary motion recognition on the motion to be recognized in a manner of determining the reference amplitude value by the reference motion data, the motion information of the motion to be recognized can be directly determined when the motion belongs to the basic motion, so that the motion recognition period can be effectively reduced, and a downstream service scene can be quickly responded to.
Further, when the determined reference amplitude is greater than the preset amplitude threshold, it is described that the motion to be recognized is a non-basic motion, at this time, the motion type needs to be analyzed based on the initial motion data, and since the basic motion data can sufficiently represent the motion condition of the motion to be recognized, the characteristic value can be determined based on the reference motion data, and the data partition interval is created, in this embodiment, a specific implementation manner is as follows:
determining a reference characteristic value corresponding to the reference motion data, and taking the reference characteristic value as the characteristic value corresponding to the initial motion data; and determining a section creation strategy corresponding to the reference action data, and creating the data segmentation section according to the section creation strategy and the reference characteristic value.
Specifically, the reference characteristic value specifically refers to a data peak value corresponding to the reference motion data, that is, a maximum value reached by an angular velocity of the motion to be recognized around the y-axis rotation angle at a certain time in a time interval corresponding to the reference motion data; correspondingly, the interval creation strategy specifically refers to a strategy for creating data partition intervals by using the reference characteristic value as a parameter and combining preset interval partition information.
Based on this, when it is determined that the identification processing of the action to be identified needs to be completed by combining with the action identification model, at this time, the reference characteristic value corresponding to the reference action data can be determined on the basis of the reference action data, then the reference characteristic value is used as the characteristic value corresponding to the initial action data, then the interval creation strategy corresponding to the reference action data is determined, and finally the data division interval capable of processing the sub-action data of each dimension in the initial action data is created by combining the reference characteristic value and the interval creation strategy, so that the processing operation from the initial action data to the target action data can be completed subsequently.
In practical application, the determination of the reference motion data may be selected according to actual requirements in consideration of different motion types in different scenes, and accordingly, the characteristic value and the interval creation policy are also controlled by the reference motion data selected according to the scene requirements, and during specific implementation, dynamic adjustment and selection may be performed according to the actual requirements, which is not limited herein.
According to the above example, after the 6-axis data (initial motion data) corresponding to the user A is collected, when the kicking is considered, the hand motion strength and the rotation angular velocity are small in the process of running without a ball, so that gyro _ pitch rotating around the y axis in the 6-axis data can be selected as reference motion data, and then the reference amplitude corresponding to the gyro _ pitch is determined; at this time, the reference amplitude is compared with the amplitude threshold THR1, if the reference amplitude is smaller than the amplitude threshold THR1, it is indicated that the movement amplitude of the user nail is smaller, and it can be determined that the action of the user nail at the current moment is dribbling.
Further, if the value is greater than the predetermined value, it indicates that the motion amplitude of the user a is relatively large, at this time, it may be determined that the peak point p of the data gyro _ pitch is a reference point, that is, in a curve of an angular velocity rotating around the y axis within a time interval constructed based on the data gyro _ pitch, a signal peak point p is selected as the reference point, and then a data partition interval is created according to an interval creation policy [ p-datalen/2,p + ((datalen/2) -1) ], since the sensor acquires initial motion data of 600ms, the data partition interval is determined to be [ p-300, p +299] in combination with the peak point p and the interval creation policy corresponding to the data gyro _ pitch, so that the acceleration data and the gyroscope data may be subsequently subjected to data partition according to [ p-300, p +299], respectively, so as to complete the motion identification processing operation of the user a subsequent combination with the football motion identification model.
In summary, the accuracy of the determined data segmentation interval can be ensured by determining the reference characteristic value based on the reference motion data and creating the data segmentation interval, so that the initial motion data can be conveniently and accurately segmented subsequently, and the motion data which can fully represent the user within the target duration can be extracted for subsequent analysis of the motion to be recognized.
And step S206, processing the initial motion data into target motion data according to the data segmentation interval, and constructing motion characteristics according to the target motion data.
Specifically, after the data division section is determined based on the feature value, further, it is considered that the initial motion data includes sub-motion data of a plurality of dimensions, and each sub-motion data can represent a change condition of the motion to be recognized in the dimension of the acceleration or the angular velocity, so after the data division section is determined, the initial motion data may be divided according to the data division section, so that a data segment with a low degree of correlation with the motion to be recognized in the initial motion data is deleted, and the remaining data segment is used as the target motion data. Further, after the target action data capable of representing the action type of the action to be recognized is determined, since the target action data includes action data capable of representing the action type of the action to be recognized in multiple dimensions, in order to enable the action recognition model to complete the action recognition of the action to be recognized in combination with the target action data, the action features can be constructed on the basis of the target action data, so that data information of the action to be recognized in multiple dimensions can be represented through the action features.
The target action data specifically refers to action data which are left after data pieces with low degree of correlation with the action to be recognized are removed after the initial action data are segmented according to the data segmentation intervals; correspondingly, the action characteristics specifically refer to a matrix constructed based on the target action data, wherein the matrix comprises expressions of the target action data in different characteristic dimensions, and the action data of all dimensions are integrated, so that the action recognition model can accurately recognize the action on the basis.
In practical application, when an action feature is constructed based on target action data, because the target action data includes sub-target action data with different dimensions, and the different sub-target action data can represent action change conditions of an action to be identified in different dimensions, before the action feature is constructed, the target action data including sub-target action data with multiple dimensions can be obtained after the sub-action data with each dimension is subjected to segmentation processing through a data segmentation interval, and in this embodiment, the specific implementation manner is as follows:
extracting a plurality of sub-motion data from the initial motion data; respectively carrying out segmentation processing on each sub-action data according to the data segmentation interval to obtain sub-target action data corresponding to each sub-action data; and determining the target action data based on the sub-target action data corresponding to each sub-action data.
Specifically, the sub-motion data specifically refers to motion data corresponding to different dimensions in the initial motion data, including but not limited to 3-axis acceleration data corresponding to acceleration dimensions, 3-axis angular velocity data corresponding to gyroscope dimensions, and the like; correspondingly, the sub-target motion data specifically refers to target motion data corresponding to different dimensions obtained after each sub-motion data is subjected to segmentation processing according to the data segmentation interval.
It should be noted that, because the data partition interval is created based on the reference feature value corresponding to the reference motion data, the data partition interval created based on the reference feature value should correspond to the reference motion data, and because the reference motion data can more fully represent the motion condition of the motion to be recognized, when the sub-motion data of other dimensions are partitioned, the data partition interval corresponding to the reference motion data can be processed, and the data partition interval corresponding to each sub-motion data is not needed to be used for completion, so that the features corresponding to each sub-target motion data can be fused into the motion features, thereby ensuring that the motion recognition model can accurately recognize the motion information of the motion to be recognized on the basis of the motion features.
Based on this, after the data partition interval is determined, a plurality of pieces of sub-motion data can be extracted from the initial motion data, and then each piece of sub-motion data is subjected to data partition processing according to the data partition interval, that is, a piece of data is cut out from the sub-motion data according to the data partition interval, and is used as sub-target motion data corresponding to each piece of sub-motion data, and the sub-target motion data is integrated into target motion data, so that motion features can be conveniently constructed on the basis of the sub-target motion data.
Following the above example, the data division interval is determined to be [ p-300, p +299] based on the peak value p corresponding to the data gyro _ pitch and the interval creation strategy]Thereafter, further, interval [ p-300, p +299] may now be partitioned according to data]For the acceleration data: ACC _ x, ACC _ y and ACC _ z, and gyroscope data gyro _ row, gyro _ pitch and gyro _ yaw are respectively subjected to data segmentation processing, namely corresponding intervals [ p-300, p +299] are intercepted from acceleration data ACC _ x, ACC _ y and ACC _ z]To obtain target acceleration data ACC _ x j ，ACC_y j And ACC _ z j (ii) a And intercepting corresponding intervals [ p-300, p +299] in gyro data gyro _ row, gyro _ pitch and gyro _ yaw]To obtain target gyroscope data gyro _ row j ，gyro_pitch j And gyro _ yaw j And the action recognition of the user A is conveniently carried out by combining the target acceleration data and the target gyroscope data.
In summary, by respectively performing segmentation processing on each sub-action data in the initial action data according to the data segmentation interval, redundant data with a low degree of correlation with the action to be recognized in the sub-action data of each dimension can be deleted on the basis of the same interval, so that action recognition processing operation can be completed by combining sub-target action data of each dimension in the following process.
Furthermore, after the target action data is obtained, in order to sufficiently express the action condition of the action to be recognized in each dimension through the action feature so as to improve the recognition accuracy of the action recognition model, a wider action representation dimension can be realized by constructing the associated action feature and the cross action feature corresponding to the target action data so as to obtain the action feature that sufficiently represents the action to be recognized, in this embodiment, the specific implementation manner is as follows from step S2062 to step S2066.
Step S2062, determining a target feature value corresponding to the target motion data.
Step S2064, relevant action characteristics and cross action characteristics corresponding to the target action data are constructed according to the target characteristic values.
Specifically, the target characteristic value specifically refers to a peak value, an amplitude value, a valley value and/or a flat value and the like corresponding to the target action data; correspondingly, the associated action characteristics specifically refer to characteristics of each sub-target action data in the target action data, including but not limited to unimodal characteristics, inter-peak characteristics and the like corresponding to the sub-target action data; the unimodal feature refers to a relevant feature corresponding to a single peak in a curve corresponding to the sub-target action data, such as a peak width feature, an amplitude mean square error, a peak height feature and/or a kurtosis value; the inter-peak characteristics refer to correlation characteristics corresponding to two or more peaks in a curve corresponding to the sub-target motion data, such as inter-peak width characteristics and/or inter-peak ratio characteristics. Correspondingly, the cross action feature specifically refers to a cross feature between any two sub-target action data in the target action data, including but not limited to a peak time difference, a peak ratio and/or a peak width ratio between any two sub-target action data; the peak value time difference is a difference value between times corresponding to respective peak values of any two sub-target action data, and the peak value ratio is a ratio of the respective peak values of any two sub-target action data; the peak width ratio specifically refers to a ratio of peak widths of any two pieces of sub-target motion data.
In practical application, besides the above interdependence relationship, the associated action feature and the cross action feature may also determine features of other dimensions based on the sub-target action data according to a practical application scenario, so as to ensure the richness of the associated action feature and the cross action feature, thereby improving the subsequent model identification accuracy, which is not limited herein.
Further, since the target action data includes sub-target action data of each dimension, it is necessary to construct associated action features and cross action features for each sub-target action data, so as to obtain action features fusing all the representations, and in this embodiment, the specific implementation manner is as follows:
(1) The construction of the associated action characteristics comprises the following steps: determining a sub-target characteristic value corresponding to each sub-target action data contained in the target action data according to the target characteristic value; calculating sub-associated action characteristics of each sub-target action data corresponding to a preset associated characteristic dimension according to the sub-target characteristic values; and fusing the sub-associated action characteristics of each sub-target action data to obtain the associated action characteristics.
Specifically, the sub-target characteristic values specifically refer to a peak value, an amplitude value, a valley value and/or a flat value and the like corresponding to each sub-target action data; correspondingly, the preset associated feature dimension specifically refers to a dimension for calculating feature expression corresponding to the sub-target action data, and includes but is not limited to a unimodal feature dimension and/or an inter-peak feature dimension; accordingly, the sub-associated action characteristics specifically refer to characteristics of the sub-target action data itself, including but not limited to unimodal characteristics and/or inter-peak characteristics, and the like. Wherein unimodal features include, but are not limited to, peak width features, amplitude mean square error, peak height features, and/or kurtosis values, among others; the peak-to-peak characteristics include, but are not limited to, peak-to-peak width characteristics and/or peak-to-peak ratio characteristics, and the like.
Based on this, after the target characteristic value corresponding to the target action data is obtained, the sub-target characteristic value corresponding to each sub-target action data can be determined according to the target characteristic value, then the sub-association action characteristic corresponding to each sub-target action data in the preset association characteristic dimension is calculated according to the sub-target characteristic value, namely the sub-association action characteristic corresponding to the corresponding sub-target action data in the preset association characteristic dimension is calculated based on the sub-target characteristic value, and finally the sub-association action characteristics of each sub-target action data are fused, so that the association action characteristic corresponding to the target action data can be obtained.
In summary, the associated action features are calculated by taking the sub-target action data as a unit and are associated with the preset associated feature dimensions, so that the associated action features can fully represent the features of the target action data, and the action features capable of fully representing the action to be identified can be conveniently constructed by combining the cross action features.
(2) Constructing the cross action characteristics: the method comprises the following steps: calculating sub-cross action characteristics corresponding to preset cross characteristic dimensions between any two sub-target action data according to the sub-target characteristic values; and fusing the sub-cross action features between any two sub-target action data to obtain the cross action feature.
Specifically, the preset cross feature dimension specifically refers to a dimension for calculating a cross feature expression between any two pieces of sub-target action data, and includes, but is not limited to, a peak time difference feature dimension, a peak ratio feature dimension, and/or a peak width ratio feature dimension; correspondingly, the sub-cross action feature specifically refers to a cross feature that exists between any two sub-target action data, and includes, but is not limited to, a peak time difference feature, a peak ratio feature, and/or a peak width ratio feature. The peak time difference characteristic refers to a difference between times corresponding to respective peaks of any two sub-target motion data, the peak ratio characteristic refers to a ratio of respective peaks of any two sub-target motion data, and the peak width ratio characteristic specifically refers to a ratio of respective peak widths of any two sub-target motion data.
Based on this, after the sub-target feature values are obtained, the sub-cross action features corresponding to any two sub-target action data in the preset cross feature dimension can be calculated according to the sub-target feature values, that is, the sub-cross action features corresponding to the preset cross feature dimension between the two corresponding sub-target action data are calculated based on the any two sub-target feature values, and finally the sub-cross action features between any two sub-target action data are fused, so that the cross action features corresponding to the target action data can be obtained.
In summary, the cross action features are calculated by taking the sub-target action data as a unit and are all associated with the preset cross feature dimension, so that an association relationship between any two sub-target action data can be established and embodied in a feature expression mode, and the influence between the sub-target action data can be conveniently balanced by combining an action recognition model in the following process, thereby improving the action recognition accuracy.
Step S2066, fusing the associated action features and the cross action features to obtain the action features.
Based on this, after obtaining the associated action features and the cross action features corresponding to the target action data, in order to meet the input requirements of the subsequent action recognition model, the associated action features and the cross action features may be fused at this time to obtain the action features, so as to implement the subsequent model prediction processing process.
Along with the above example, the target acceleration data ACC _ x is obtained j ，ACC_y j And ACC _ z j (ii) a And target gyroscope data gyro _ row j ，gyro_pitch j And gyro _ yaw j Then, further, the peak value and the amplitude value corresponding to each axis data can be determined at this time; and then calculating the corresponding unimodal characteristic and the corresponding inter-peak characteristic according to the peak value and the amplitude value corresponding to each axis data.
Referring to the schematic diagram of FIG. 3, the axis data gyro _ pitch is shown j Explaining the calculation process of the sub-associated action characteristics, calculating axis data gyro _ pitch based on the peak value corresponding to the p point and the amplitude corresponding to the data j Corresponding unimodal feature, i.e. the axis data gyro _ pitch j Corresponding peak width characteristic wF1, amplitude mean square deviation std, peak height characteristic Hpeak and kurtosis value (a kurtosis coefficient in statistics); wherein the peak width feature wF1= W1/0.707hpeak, w1 denotes the peak width, and 0.707Hpeak denotes the peak height position at which the peak width feature is calculated, i.e., the height of the vertical axis; the peak height feature Hpeak _ gy = (Hp 1+ Hp 2)/2, hp1 represents a peak, and Hp2 represents a trough. Computing axis data gyro _ pitch simultaneously j Corresponding inter-peak features, i.e. the axis data gyro _ pitch j Corresponding toPeak-to-peak width characteristic tW1 and peak-to-peak ratio characteristic rpeak; wherein the peak-to-peak ratio characteristic rpeak = Hpeak1/Hpeak2, where Hpeak1 denotes the first peak and Hpeak2 denotes the second peak. Obtaining axis data gyro _ pitch based on the above calculation j Corresponding sub-associated action features.
Further, calculating a peak time difference, a peak ratio and a peak width ratio between any two axis data according to the peak value and the amplitude value corresponding to each axis data; with axis data gyro _ pitch j And gyro _ row j The calculation process of the sub-cross action feature is explained. Based on axis data gyro _ pitch j And gyro _ row j Calculating a time difference tsW between two peaks corresponding to the peak value and the amplitude value corresponding to the data, wherein the time difference tsW = Tpeak _ gy-Tpeak _ gx, and Tpeak _ gy represents axis data gyro _ pitch j The peak value of (a), tpeak _ gx represents the axis data gyro _ row j The time node corresponding to the peak value of (a); peak ratio rpeak _ gyx = Hpeak _ gy/Hpeak _ gx, where Hpeak _ gy and Hpeak _ gx represent same time axis data gyro _ pitch j And gyro _ row j A peak value of (d); peak-width ratio rwF _ gyx = wF1_ gy/wF2_ gx, where wF1_ gy represents axis data gyro _ pitch j wF2_ gx denotes axis data gyro _ row j Peak width of (d). Obtaining axis data gyro _ pitch based on the above calculation j And gyro _ row j Sub-cross action features in between.
Accordingly, the calculation processes of the sub-associated action features and the sub-intersection action features of other axis data can be referred to the above description, and this embodiment is not described in detail herein. That is, axis data gyro _ yaw can also be calculated j And gyro _ pitch j Is equal to or greater than the peak ratio rpeak _ gzy = Hpeak _ gz/Hpeak _ gy, where Hpeak _ gz and Hpeak _ gy represent the same time axis data gyro _ raw j And gyro _ pitch j A peak value of (d); axis data ACC _ z j And ACC _ y j Peak ratio rpeak _ azy = Hpeak _ az/Hpeak _ ay, where Hpeak _ az and Hpeak _ ay represent the same time-of-day axis data ACC _ z j And ACC _ y j The peak value of (c).
And finally, fusing the sub-associated action features and the sub-cross action features respectively corresponding to the axis data to obtain action features corresponding to action changes of the user A in the ball kicking process, so as to conveniently obtain the ball kicking action type of the user A at the current moment based on the action features.
In conclusion, the action features corresponding to the target action data are constructed by combining the preset associated feature dimensions and the cross feature dimensions, data representation of multiple dimensions is fully combined, action information of the action to be recognized is fully expressed in the action features, and therefore when the action recognition model performs action recognition, recognition accuracy can be guaranteed.
And S208, inputting the motion characteristics into a motion recognition model for processing to obtain motion information corresponding to the motion to be recognized.
Specifically, after the motion characteristics corresponding to the motion to be recognized are obtained, the motion characteristics may be further input to a motion recognition model trained in advance for processing, so as to obtain motion information of the motion to be recognized according to a recognition result of the recognition model. The action information specifically refers to information representing the action type of the action to be recognized, and the action information is different under different recognition scenes; in a football action recognition scene, after the action to be recognized is recognized, the action information includes but is not limited to kicking a ball, shooting a goal, carrying a ball and/or running without a ball, etc.; or in the basketball action recognition scene, after the action to be recognized is recognized, the action information includes, but is not limited to, dribbling, shooting, putting and/or passing and the like.
The action recognition model is a model capable of recognizing the action to be recognized, and the recognition result is set according to requirements in the training process. Correspondingly, the action recognition model may be implemented by using an SVM (support vector machine), a random forest, or a logistic regression, and may be selected according to an actual application scenario in specific implementation, which is not limited herein.
In practical applications, in order to improve the recognition accuracy of the motion recognition model, the motion recognition model needs to be trained sufficiently in a training phase, so that an accurate prediction can be made in response to a motion to be recognized in an application phase, in this embodiment, the motion recognition model is trained as follows:
acquiring sample action data, and determining a sample characteristic value corresponding to the sample action data; creating a sample data segmentation interval according to the sample characteristic value, and processing the sample action data into target sample action data according to the sample data segmentation interval; constructing sample action characteristics according to the target sample action data, inputting the sample action characteristics to an initial action recognition model for processing, and obtaining predicted action information; and adjusting parameters of the initial action recognition model according to sample action information corresponding to the sample action data and the predicted action information until the action recognition model meeting the training stop condition is obtained.
Specifically, the training stopping condition specifically refers to a condition that the training of the initial recognition model is stopped after the current recognition capability of the initial recognition model can meet the use requirement, and includes but is not limited to model iteration times or loss value comparison; after the number of times of training iteration of the initial motion recognition model reaches a set number of times, the training of the initial motion recognition model can be stopped, and the result of the last iteration training can be used as the motion recognition model; or after each training of the initial motion recognition model is completed, calculating a loss value of the initial motion model by combining a preset loss function, stopping training the initial motion recognition model under the condition that the loss value is smaller than a preset loss value threshold, and taking the model after the last parameter adjustment as the motion recognition model.
According to the above example, after the action characteristics corresponding to the action of the user A at the current moment are obtained, the action characteristics can be input into a football action recognition model which is trained in advance and can recognize football actions, and the action of the user A at the current moment is determined to be a shooting action according to the recognition result.
In addition, after the action information corresponding to the action to be recognized is obtained, the downstream service can make a corresponding response according to the action information so as to provide corresponding service for the user. For example, in a sports action recognition scene, after action information of an action to be recognized of a user is obtained, the action information can represent that the current user is a wonderful sport moment, such as shooting when playing basketball, shooting when playing football, or jumping when kicking a goal, or jumping when jumping far away, and a collection request can be sent to video collection equipment or image collection equipment according to the identified action information so as to collect the sport action at the current moment through the video collection equipment or the image collection equipment; or the mark information corresponding to the action information at the current moment is added in the collected video or image, so that the user can quickly find the content with higher importance degree for browsing in the subsequent watching stage, and the use experience of the user is improved.
In specific implementation, after obtaining the action information of the action to be recognized, corresponding processing operation may be performed in response to the action information according to the actual application environment, and this embodiment is not limited herein.
In the action recognition method provided by the specification, after initial action data corresponding to an action to be recognized is collected, in order to improve the action recognition accuracy, a characteristic value corresponding to the initial action data can be determined, and then a data partition interval for processing the initial action data is created according to the characteristic value; at the moment, the initial action data can be processed into target action data according to the data division interval, and then action characteristics are constructed on the basis of the target action data; and finally, inputting the motion characteristics into a pre-trained motion recognition model for processing, so as to obtain motion information corresponding to the motion to be recognized and output by the motion recognition model. When the action to be recognized is recognized, the characteristic information in the initial action data can be fully combined, so that the action information of the action to be recognized can be accurately predicted by the model, and the accuracy of action recognition is effectively improved.
The following will further describe the motion recognition method with reference to fig. 4 by taking an application of the motion recognition method provided in the present specification to a basketball motion recognition scenario as an example. Fig. 4 shows a processing flow chart of an action recognition method provided in an embodiment of the present specification, which specifically includes the following steps:
and S402, acquiring acceleration data and gyroscope data of the target duration corresponding to the user through the motion sensor.
And S404, determining reference gyroscope data according to the gyroscope data, and determining a reference amplitude corresponding to the reference gyroscope data.
Step S406, judging whether the reference amplitude is smaller than a preset amplitude threshold value; if yes, go to step S408; if not, go to step S410.
In step S408, it is determined that the motion of the user at the current time is the dribbling motion.
Step S410, determining a reference peak value corresponding to the reference gyroscope data, and creating a data division interval according to the reference peak value and an interval creation strategy.
Step S412, performing segmentation processing on the gyroscope data and the acceleration data according to the data segmentation intervals, to obtain target acceleration data and target gyroscope data.
Step S414, determining an acceleration data peak value corresponding to the target acceleration data, and determining a gyroscope data peak value corresponding to the target gyroscope data.
And step S416, constructing the associated action characteristic and the cross action characteristic corresponding to the acceleration data according to the acceleration data peak value, and constructing the associated action characteristic and the cross action characteristic corresponding to the gyroscope data according to the gyroscope data peak value.
And step S418, fusing the association action characteristic and the cross action characteristic corresponding to the acceleration data and the association action characteristic and the cross action characteristic corresponding to the gyroscope data to obtain the action characteristic.
Step S420, inputting the motion characteristics into the basketball motion recognition model trained in advance, and processing the motion characteristics to obtain the basketball motion information of the user at the current moment.
Specifically, acceleration data and gyroscope data of a user corresponding to a target duration are acquired through an IMU sensor mounted on a bracelet worn by the user, then an amplitude corresponding to y-axis data in the gyroscope data is determined, and whether the amplitude is smaller than a preset amplitude threshold value is judged; and if so, determining that the action of the user at the current moment is the running action without the ball. If not, determining a peak value corresponding to the y-axis data, and then creating a data segmentation interval according to the peak value.
Further, each axis data included in the acceleration data and the gyroscope data is respectively segmented according to the data segmentation interval to obtain target acceleration data and target gyroscope data; determining peak values corresponding to all axis data contained in the target acceleration data and the target gyroscope data, constructing sub-associated action characteristics corresponding to all axis data and sub-cross action characteristics between any two axis data according to the peak values, fusing all the sub-associated action characteristics and the sub-cross action characteristics to obtain action characteristics corresponding to the user, and finally inputting the action characteristics to a basketball action recognition model for processing to obtain the action of the user at the current moment as a shooting action.
Furthermore, a marking request is created according to the shooting action, and video marking information is sent to the mobile phone communicated with the bracelet, so that description information of the shooting action generated by the user at the current moment is marked in the video recorded by the mobile phone, and the user can conveniently and quickly view video clips of the shooting action when browsing the video in the follow-up process.
In summary, after acquiring initial motion data corresponding to a motion to be recognized, in order to improve motion recognition accuracy, a feature value corresponding to the initial motion data may be determined, and then a data partition interval for processing the initial motion data is created according to the feature value; at the moment, the initial action data can be processed into target action data according to the data segmentation interval, and then action features are constructed on the basis of the target action data; and finally, inputting the motion characteristics into a pre-trained motion recognition model for processing, so as to obtain motion information corresponding to the motion to be recognized and output by the motion recognition model. When the action to be recognized is recognized, the characteristic information in the initial action data can be fully combined, so that the action information of the action to be recognized can be accurately predicted by the model, and the accuracy of action recognition is effectively improved.
Corresponding to the above method embodiment, the present specification further provides an embodiment of a motion recognition apparatus, and fig. 5 shows a schematic structural diagram of a motion recognition apparatus provided in an embodiment of the present specification. As shown in fig. 5, the apparatus includes:
a data collecting module 502 configured to collect initial motion data corresponding to a motion to be recognized;
a creating interval module 504 configured to determine a feature value corresponding to the initial motion data, and create a data dividing interval according to the feature value;
a build feature module 506 configured to process the initial motion data into target motion data according to the data partition interval, and build a motion feature according to the target motion data;
and the model processing module 508 is configured to input the motion characteristics into a motion recognition model for processing, and obtain motion information corresponding to the motion to be recognized.
In an optional embodiment, the motion recognition apparatus further includes:
a determine magnitude module configured to extract baseline motion data in the initial motion data; determining a reference amplitude value corresponding to the reference motion data; the create interval module 504 is run if the baseline amplitude is greater than a preset amplitude threshold.
In an optional embodiment, the create interval module 504 is further configured to:
determining a reference characteristic value corresponding to the reference motion data, and taking the reference characteristic value as the characteristic value corresponding to the initial motion data; and determining a section creation strategy corresponding to the reference action data, and creating the data segmentation section according to the section creation strategy and the reference characteristic value.
In an alternative embodiment, the build feature module 506 is further configured to:
determining a target characteristic value corresponding to the target action data; constructing associated action characteristics and cross action characteristics corresponding to the target action data according to the target characteristic values; and fusing the associated action features and the cross action features to obtain the action features.
In an optional embodiment, the associated action feature building includes:
determining a sub-target characteristic value corresponding to each sub-target action data contained in the target action data according to the target characteristic value; calculating sub-associated action characteristics of each sub-target action data corresponding to a preset associated characteristic dimension according to the sub-target characteristic values; and fusing the sub-associated action characteristics of each sub-target action data to obtain the associated action characteristics.
In an optional embodiment, the cross-action feature construction includes:
calculating sub-cross action characteristics corresponding to preset cross characteristic dimensions between any two sub-target action data according to the sub-target characteristic values; and fusing the sub-cross action features between any two sub-target action data to obtain the cross action feature.
In an alternative embodiment, the build feature module 506 is further configured to:
extracting a plurality of sub-action data from the initial action data; respectively carrying out segmentation processing on each sub-action data according to the data segmentation interval to obtain sub-target action data corresponding to each sub-action data; and determining the target action data based on the sub-target action data corresponding to each sub-action data.
In an optional embodiment, the data acquisition module 502 is further configured to:
acquiring acceleration data and gyroscope data of a target duration through a motion sensor; and taking the acceleration data and the gyroscope data as the initial motion data.
In an alternative embodiment, the motion recognition model is trained by:
acquiring sample action data, and determining a sample characteristic value corresponding to the sample action data; creating a sample data segmentation interval according to the sample characteristic value, and processing the sample action data into target sample action data according to the sample data segmentation interval; constructing sample action characteristics according to the target sample action data, inputting the sample action characteristics to an initial action recognition model for processing, and obtaining predicted action information; and adjusting parameters of the initial action recognition model according to sample action information corresponding to the sample action data and the predicted action information until the action recognition model meeting the training stopping condition is obtained.
After the initial action data corresponding to the action to be recognized is acquired, in order to improve the action recognition accuracy, the action recognition device provided by the specification can determine a characteristic value corresponding to the initial action data, and then create a data partition interval for processing the initial action data according to the characteristic value; at the moment, the initial action data can be processed into target action data according to the data division interval, and then action characteristics are constructed on the basis of the target action data; and finally, inputting the motion characteristics into a pre-trained motion recognition model for processing, so as to obtain motion information corresponding to the motion to be recognized and output by the motion recognition model. When the action to be recognized is recognized, the characteristic information in the initial action data can be fully combined, so that the action information of the action to be recognized can be accurately predicted by the model, and the accuracy of action recognition is effectively improved.
The above is a schematic scheme of a motion recognition apparatus of the present embodiment. It should be noted that the technical solution of the motion recognition device is the same as that of the motion recognition method, and for details that are not described in detail in the technical solution of the motion recognition device, reference may be made to the description of the technical solution of the motion recognition method.
Fig. 6 illustrates a block diagram of a computing device 600 provided according to an embodiment of the present description. The components of the computing device 600 include, but are not limited to, a memory 610 and a processor 620. The processor 620 is coupled to the memory 610 via a bus 630 and a database 650 is used to store data.
Computing device 600 also includes access device 640, access device 640 enabling computing device 600 to communicate via one or more networks 660. Examples of such networks include the Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the internet. Access device 640 may include one or more of any type of network interface (e.g., a Network Interface Card (NIC)) whether wired or wireless, such as an IEEE802.11 Wireless Local Area Network (WLAN) wireless interface, a worldwide interoperability for microwave access (Wi-MAX) interface, an ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a bluetooth interface, a Near Field Communication (NFC) interface, and so forth.
In one embodiment of the present description, the above-described components of computing device 600, as well as other components not shown in FIG. 6, may also be connected to each other, such as by a bus. It should be understood that the block diagram of the computing device architecture shown in FIG. 6 is for purposes of example only and is not limiting as to the scope of the present description. Those skilled in the art may add or replace other components as desired.
Computing device 600 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), mobile phone (e.g., smartphone), wearable computing device (e.g., smartwatch, smartglasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. Computing device 600 may also be a mobile or stationary server. Wherein the processor 620 is configured to implement the steps of the motion recognition method when executing the computer-executable instructions.
The above is an illustrative scheme of a computing device of the present embodiment. It should be noted that the technical solution of the computing device and the technical solution of the motion recognition method belong to the same concept, and for details that are not described in detail in the technical solution of the computing device, reference may be made to the description of the technical solution of the motion recognition method.
An embodiment of the present specification also provides a computer-readable storage medium storing computer instructions, which when executed by a processor, are used for the motion recognition method.
The above is an illustrative scheme of a computer-readable storage medium of the present embodiment. It should be noted that the technical solution of the storage medium belongs to the same concept as the technical solution of the above-mentioned motion recognition method, and for details that are not described in detail in the technical solution of the storage medium, reference may be made to the description of the technical solution of the above-mentioned motion recognition method.
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The computer instructions comprise computer program code which may be in the form of source code, object code, an executable file or some intermediate form, or the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, etc. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
It should be noted that, for the sake of simplicity, the foregoing method embodiments are described as a series of acts or combinations, but those skilled in the art should understand that the present disclosure is not limited by the described order of acts, as some steps may be performed in other orders or simultaneously according to the present disclosure. Further, those skilled in the art should also appreciate that the embodiments described in this specification are preferred embodiments and that acts and modules referred to are not necessarily required for this description.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
The preferred embodiments of the present specification disclosed above are intended only to aid in the description of the specification. Alternative embodiments are not exhaustive and do not limit the invention to the precise embodiments described. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the specification and its practical application, to thereby enable others skilled in the art to best understand the specification and its practical application. The specification is limited only by the claims and their full scope and equivalents.

Claims (12)

1. A motion recognition method, comprising:
acquiring initial action data corresponding to an action to be identified;
determining a characteristic value corresponding to the initial action data, and creating a data segmentation interval according to the characteristic value;
processing the initial action data into target action data according to the data segmentation interval, and constructing action characteristics according to the target action data;
and inputting the motion characteristics into a motion recognition model for processing to obtain motion information corresponding to the motion to be recognized.
2. The method of claim 1, wherein before the step of determining the characteristic value corresponding to the initial motion data is performed, the method further comprises:
extracting reference motion data from the initial motion data;
determining a reference amplitude value corresponding to the reference motion data;
and under the condition that the reference amplitude is larger than a preset amplitude threshold value, executing the step of determining the characteristic value corresponding to the initial action data.
3. The method of claim 2, wherein the determining the feature value corresponding to the initial motion data comprises:
determining a reference characteristic value corresponding to the reference motion data, and taking the reference characteristic value as the characteristic value corresponding to the initial motion data;
correspondingly, the creating of the data partition interval according to the feature value includes:
and determining a section creation strategy corresponding to the reference action data, and creating the data segmentation section according to the section creation strategy and the reference characteristic value.
4. The method of claim 1, wherein said constructing motion features from said target motion data comprises:
determining a target characteristic value corresponding to the target action data;
constructing associated action characteristics and cross action characteristics corresponding to the target action data according to the target characteristic values;
and fusing the associated action features and the cross action features to obtain the action features.
5. The method of claim 4, wherein the associated action feature is constructed by:
determining a sub-target characteristic value corresponding to each sub-target action data contained in the target action data according to the target characteristic value;
calculating sub-associated action characteristics of each sub-target action data corresponding to a preset associated characteristic dimension according to the sub-target characteristic values;
and fusing the sub-associated action characteristics of each sub-target action data to obtain the associated action characteristics.
6. The method of claim 5, wherein the cross-action feature is constructed by:
calculating sub-cross action characteristics corresponding to preset cross characteristic dimensions between any two sub-target action data according to the sub-target characteristic values;
and fusing the sub-cross action features between any two sub-target action data to obtain the cross action feature.
7. The method of claim 1, wherein the processing the initial motion data into target motion data according to the data partition interval comprises:
extracting a plurality of sub-action data from the initial action data;
respectively carrying out segmentation processing on each sub-action data according to the data segmentation interval to obtain sub-target action data corresponding to each sub-action data;
and determining the target action data based on the sub-target action data corresponding to each sub-action data.
8. The method according to any one of claims 1 to 7, wherein the collecting initial motion data corresponding to the motion to be recognized comprises:
acquiring acceleration data and gyroscope data of a target duration through a motion sensor;
and taking the acceleration data and the gyroscope data as the initial motion data.
9. The method according to any of claims 1-7, wherein the motion recognition model is trained by:
acquiring sample action data, and determining a sample characteristic value corresponding to the sample action data;
creating a sample data segmentation interval according to the sample characteristic value, and processing the sample action data into target sample action data according to the sample data segmentation interval;
constructing sample action characteristics according to the target sample action data, inputting the sample action characteristics to an initial action recognition model for processing, and obtaining predicted action information;
and adjusting parameters of the initial action recognition model according to sample action information corresponding to the sample action data and the predicted action information until the action recognition model meeting the training stopping condition is obtained.
10. An action recognition device, comprising:
the data acquisition module is configured to acquire initial action data corresponding to the action to be recognized;
the creating interval module is configured to determine a characteristic value corresponding to the initial action data and create a data dividing interval according to the characteristic value;
the construction characteristic module is configured to process the initial action data into target action data according to the data segmentation interval and construct action characteristics according to the target action data;
and the model processing module is configured to input the motion characteristics into a motion recognition model for processing, and obtain motion information corresponding to the motion to be recognized.
11. A computing device, comprising:
a memory and a processor;
the memory is configured to store computer-executable instructions, and the processor is configured to implement the steps of the motion recognition method according to any one of claims 1 to 9 when executing the computer-executable instructions.
12. A computer-readable storage medium storing computer instructions, which when executed by a processor, implement the steps of the motion recognition method of any one of claims 1 to 9.
CN202211297199.XA 2022-10-21 2022-10-21 Action recognition method and device Pending CN115601837A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211297199.XA CN115601837A (en) 2022-10-21 2022-10-21 Action recognition method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211297199.XA CN115601837A (en) 2022-10-21 2022-10-21 Action recognition method and device

Publications (1)

Publication Number Publication Date
CN115601837A true CN115601837A (en) 2023-01-13

Family

ID=84849011

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211297199.XA Pending CN115601837A (en) 2022-10-21 2022-10-21 Action recognition method and device

Country Status (1)

Country Link
CN (1) CN115601837A (en)

Similar Documents

Publication Publication Date Title
CN109905772B (en) Video clip query method, device, computer equipment and storage medium
CN108256433B (en) Motion attitude assessment method and system
WO2020233464A1 (en) Model training method and apparatus, storage medium, and device
EP3243554B1 (en) Information processing device, information processing method, and program
CN108304758B (en) Face characteristic point tracking method and device
JP6684883B2 (en) Method and system for providing camera effects
JP6082101B2 (en) Body motion scoring device, dance scoring device, karaoke device, and game device
US20160225410A1 (en) Action camera content management system
KR102241414B1 (en) Electronic device for providing a feedback for a specivic motion using a machine learning model a and machine learning model and method for operating thereof
CN103514432A (en) Method, device and computer program product for extracting facial features
KR20160059844A (en) Method for training using movement trajectory analysis and apparatus therefor
EP3088058A1 (en) Analysis device, recording medium, and analysis method
CN110766081A (en) Interface image detection method, model training method and related device
CN109740567A (en) Key point location model training method, localization method, device and equipment
CN113487709A (en) Special effect display method and device, computer equipment and storage medium
CN111738769A (en) Video processing method and device
CN112102157A (en) Video face changing method, electronic device and computer readable storage medium
Hammam et al. Real-time multiple spatiotemporal action localization and prediction approach using deep learning
CN114513694A (en) Scoring determination method and device, electronic equipment and storage medium
EP4198772A1 (en) Method and device for making music recommendation
CN115601837A (en) Action recognition method and device
US20230390603A1 (en) Exercise improvement instruction device, exercise improvement instruction method, and exercise improvement instruction program
CN114298018A (en) Video title generation method and device and storage medium
CN113992976A (en) Video playing method, device, equipment and computer storage medium
Chakraborty et al. A site entropy rate and degree centrality based algorithm for image co-segmentation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination