CN108764176A - A kind of action sequence recognition methods, system and equipment and storage medium - Google Patents
A kind of action sequence recognition methods, system and equipment and storage medium Download PDFInfo
- Publication number
- CN108764176A CN108764176A CN201810550608.XA CN201810550608A CN108764176A CN 108764176 A CN108764176 A CN 108764176A CN 201810550608 A CN201810550608 A CN 201810550608A CN 108764176 A CN108764176 A CN 108764176A
- Authority
- CN
- China
- Prior art keywords
- action sequence
- sample
- neural networks
- lstm neural
- identified
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
This application discloses a kind of action sequence recognition methods, system and equipment and computer readable storage medium, this method to include：Raw image data is obtained, and pretreatment operation is carried out to the raw image data and obtains sample to be identified；Feature extraction is carried out to each frame image in the sample to be identified, obtains feature vector；Described eigenvector is inputted in the LSTM neural networks that training is completed, obtains action sequence recognition result.Action sequence recognition methods provided by the present application, pretreatment operation is carried out to raw image data first, obtain sample to be identified, ensure that each frame image in sample to be identified is all of the same size and discrimination, feature extraction is carried out to each frame image again, and in the LSTM neural networks for the feature vector input training completion for obtaining extraction, to obtain action sequence recognition result.Raw image data is handled by feature extraction, then is combined with LSTM neural networks, the discrimination of action sequence is improved.
Description
Technical field
This application involves technical field of image processing, more specifically to a kind of action sequence recognition methods, system and
Equipment and a kind of computer readable storage medium.
Background technology
In recent years, the research of human action identification is paid high attention to by industrial quarters, in video monitoring, game and machine
There is important application in the fields such as people.However efficiently action recognition algorithm is very challenging：First, different mobile speed
Degree leads to the fluctuation of the same action in time；Secondly, it is many action have similitude, such as it is high throwing and wave；Most
Afterwards, different people also results in the difficulty of identification in the difference of height, figure etc..Action sequence identification side in the prior art
Method generally directly using image data as the input of deep learning network, however directly uses raw video picture data meeting
Influence the discrimination of final action sequence.
Therefore, how to improve the discrimination of action sequence is those skilled in the art's problem to be solved.
Invention content
The application's is designed to provide a kind of action sequence recognition methods, system and equipment and a kind of computer-readable deposits
Storage media improves the discrimination of action sequence.
To achieve the above object, this application provides a kind of action sequence recognition methods, including：
Raw image data is obtained, and pretreatment operation is carried out to the raw image data and obtains sample to be identified；
Feature extraction is carried out to each frame image in the sample to be identified, obtains feature vector；
Described eigenvector is inputted in the LSTM neural networks that training is completed, obtains action sequence recognition result.
Wherein, it is described obtain action sequence recognition result after, further include：
The average recognition rate of the sample to be identified is calculated according to the discrimination of each frame image.
Wherein, the pretreatment operation includes any one of turning operation, down-sampling operation or cutting operation or several
Combination.
Wherein, before described eigenvector being inputted in the LSTM neural networks that training is completed, further include：
Training sample is obtained, and feature extraction is carried out to each frame image in the training sample, obtains training characteristics
Vector；
The training feature vector is inputted in LSTM neural networks, and adjusts the key parameter of the LSTM neural networks
Until the discrimination of LSTM neural networks output reaches preset value, to obtain the LSTM neural networks of training completion.
Wherein, the key parameter of the LSTM neural networks is adjusted, including：
The key parameter of the LSTM neural networks is adjusted using cross validation method and pair-wise algorithms.
Wherein, the propagated forward algorithm of the LSTM neural networks includes two level derivatives of the cell to the time.
Wherein, feature extraction is carried out to each frame image in the sample to be identified, obtains feature vector, including：
S321：Target image is determined from the sample to be identified, and obtains the previous frame image of target image and latter
Frame image；
S322：The target image, the previous frame image and a later frame image are formed into the target image pair
The candidate image answered, and the point cloud by characteristic value in the candidate image more than feature preset value is determined as key point；
S323：The HOPC features of all key points are extracted, and the target image is determined according to all HOPC features
Feature vector；
S321-S323 is repeated until all images in the sample to be identified are all extracted and completed.
To achieve the above object, this application provides a kind of action sequence identifying systems, including：
Acquisition module for obtaining raw image data, and carries out pretreatment operation to the raw image data and obtains
Sample to be identified；
Extraction module obtains feature vector for carrying out feature extraction to each frame image in the sample to be identified；
Input module obtains action sequence for inputting described eigenvector in the LSTM neural networks that training is completed
Recognition result.
To achieve the above object, this application provides a kind of action sequence identification equipments, including：
Memory, for storing computer program；
Processor is realized when for executing the computer program such as the step of above-mentioned action sequence recognition methods.
To achieve the above object, this application provides a kind of computer readable storage medium, the computer-readable storages
It is stored with computer program on medium, such as above-mentioned action sequence recognition methods is realized when the computer program is executed by processor
The step of.
By above scheme it is found that a kind of action sequence recognition methods provided by the present application, including：Obtain original image number
According to, and pretreatment operation is carried out to the raw image data and obtains sample to be identified；To each in the sample to be identified
Frame image carries out feature extraction, obtains feature vector；Described eigenvector is inputted in the LSTM neural networks that training is completed, is obtained
To action sequence recognition result.
Action sequence recognition methods provided by the present application carries out pretreatment operation to raw image data first, is waited for
It identifies sample, ensures that each frame image in sample to be identified is all of the same size and discrimination, then to each frame image
In the LSTM neural networks that the feature vector input training for carrying out feature extraction, and extraction being obtained is completed, to be acted
Recognition sequence result.Raw image data is handled by feature extraction, then is combined with LSTM neural networks, action sequence is improved
Discrimination.Disclosed herein as well is a kind of action sequence identifying system and equipment and a kind of computer readable storage mediums, together
Sample can realize above-mentioned technique effect.
Description of the drawings
In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, to embodiment or will show below
There is attached drawing needed in technology description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of application for those of ordinary skill in the art without creative efforts, can be with
Obtain other attached drawings according to these attached drawings.
Fig. 1 is a kind of flow chart of action sequence recognition methods disclosed in the embodiment of the present application；
Fig. 2 is a kind of structure chart of LSTM neural networks；
Fig. 3 is the flow chart of another action sequence recognition methods disclosed in the embodiment of the present application；
Fig. 4 is a kind of structure chart of action sequence identifying system disclosed in the embodiment of the present application；
Fig. 5 is a kind of structure chart of action sequence identification equipment disclosed in the embodiment of the present application；
Fig. 6 is the structure chart of another action sequence identification equipment disclosed in the embodiment of the present application.
Specific implementation mode
Below in conjunction with the attached drawing in the embodiment of the present application, technical solutions in the embodiments of the present application carries out clear, complete
Site preparation describes, it is clear that described embodiments are only a part of embodiments of the present application, instead of all the embodiments.It is based on
Embodiment in the application, those of ordinary skill in the art are obtained every other without creative efforts
Embodiment shall fall in the protection scope of this application.
The embodiment of the present application discloses a kind of action sequence recognition methods, improves the discrimination of action sequence.
Referring to Fig. 1, a kind of flow chart of action sequence recognition methods disclosed in the embodiment of the present application, as shown in Figure 1, packet
It includes：
S101：Raw image data is obtained, and pretreatment operation is carried out to the raw image data and obtains sample to be identified
This；
In specific implementation, it needs to carry out pretreatment operation to the raw image data after obtaining raw image data, i.e.,
Sample to be identified is obtained after enhancing operation, the present embodiment is not defined specific pretreatment operation, those skilled in the art
It can flexibly be selected according to actual conditions.As a preferred implementation manner, pretreatment operation herein include turning operation, under
Any one of sampling operation or cutting operation or several combinations.Wherein, down-sampling operation is i.e. between a sample sequence
It is primary every the sampling of several sample values.
S102：Feature extraction is carried out to each frame image in the sample to be identified, obtains feature vector；
In specific implementation, in order to improve the action sequence discrimination of neural network output, in needing to the sample identified
Each frame image carries out feature extraction, obtains feature vector.The present embodiment does not equally limit the specific features of extraction and feature carries
The concrete mode taken, those skilled in the art can flexibly select according to actual conditions.For example, each frame image can be extracted
HOPC (Chinese names：The histogram of principal component, full name in English：Histogram of Principal Component) feature,
Specific feature extracting method will describe in detail in next embodiment.
S103：Described eigenvector is inputted in the LSTM neural networks that training is completed, obtains action sequence recognition result.
In specific implementation, the feature vector of each frame image is inputted into the LSTM (Chinese names that training is completed：Shot and long term
Memory network, full name in English：Network-Long Short Term Memory Network) in neural network, to be moved
Make sequence recognition result.
As a preferred implementation manner, as shown in Fig. 2, the propagated forward algorithm of LSTM neural networks herein includes
Two level derivatives of the cell to the time.It can derive that corresponding back-propagating is calculated by propagated forward algorithm those skilled in the art
Method, since there are cell in propagated forward algorithm and Back Propagation Algorithm to the two level derivative of time, can be fine preserve it is dynamic
The temporal information for making sequence avoids the time misalignment of recognition result.
It is understood that this step gives tacit consent to the trained completion of LSTM neural networks, the instruction of above-mentioned LSTM neural networks
Practicing process is specially：Training sample is obtained, and feature extraction is carried out to each frame image in the training sample, is trained
Feature vector；The training feature vector is inputted in LSTM neural networks, and adjusts the crucial ginseng of the LSTM neural networks
Number is until the discrimination of LSTM neural networks output reaches preset value, to obtain the LSTM neural networks of training completion.
It should be noted that each frame image in above-mentioned training sample passes through pretreatment operation, i.e., turning operation, under
Sampling operation and cutting operation etc..The feature of training sample extraction and the mode of feature extraction and above-described sample to be identified
The feature of extraction is identical with the mode of feature extraction.The training feature vector extracted is inputted in LSTM neural networks, is adjusted
The key parameter of LSTM neural networks is until the discrimination of LSTM neural networks output reaches preset value.As a kind of preferred
Embodiment can utilize cross validation method and pair-wise algorithms to adjust key parameter.Referring herein to key parameter can
With include Batch_video (the video number inputted every time), Batch_frame (video frame number that each video includes),
Epoch (numbers of the primary all data of training), learning rate or learning rate decaying etc..Herein not to the initial of above-mentioned key parameter
Value is specifically limited, and those skilled in the art can be flexibly arranged according to actual conditions, for example, Batch_video=6,
Batch_frame=24, epoch=5000~8000, learning rate Learning_rate=0.1, learning rate decaying lr_decay
=0.1/1000 time.
It is understood that after above-mentioned LSTM neural networks are completed in training, test sample can also be utilized to training
The LSTM neural networks of completion are tested, specifically, the average recognition rate of all picture frames in test sample is calculated, to obtain
The action recognition accuracy rate of the LSTM neural networks.
It is understood that obtaining action sequence recognition result in this step not only including the action sequence at identification, also
It may include discrimination, that is, calculate the discrimination of each frame image, and sample to be identified is calculated according to the discrimination of each frame image
This average recognition rate.
Action sequence recognition methods provided by the embodiments of the present application carries out pretreatment operation to raw image data first,
Sample to be identified is obtained, ensures that each frame image in sample to be identified is all of the same size and discrimination, then to each
Frame image carries out feature extraction, and in the LSTM neural networks for the feature vector input training completion that extraction is obtained, to
To action sequence recognition result.Raw image data is handled by feature extraction, then is combined with LSTM neural networks, is improved dynamic
Make the discrimination of sequence.
The embodiment of the present application discloses a kind of action sequence recognition methods, and relative to a upper embodiment, the present embodiment is to skill
Art scheme has made further instruction and optimization.Specifically：
Referring to Fig. 3, the flow chart of another kind action sequence recognition methods provided by the embodiments of the present application, as shown in figure 3, packet
It includes：
S301：Raw image data is obtained, and pretreatment operation is carried out to the raw image data and obtains sample to be identified
This；
S321：Target image is determined from the sample to be identified, and obtains the previous frame image of target image and latter
Frame image；
S322：The target image, the previous frame image and a later frame image are formed into the target image pair
The candidate image answered, and the point cloud by characteristic value in the candidate image more than feature preset value is determined as key point；
S323：The HOPC features of all key points are extracted, and the target image is determined according to all HOPC features
Feature vector；
In the present embodiment, to the sample identified in the HOPC features of each frame image extract.First to be identified
Target image is chosen in all picture frames in sample, and it is to wait to merge target image with its previous frame image and a later frame image
Image is selected, and determines the key point of candidate image, key point herein is chosen from the point cloud in candidate image, selected characteristic value
More than feature preset value point cloud as key point, feature preset value those skilled in the art herein can be according to feature extraction
The precision needed is configured.
It is understood that in specific implementation, can also determine the point cloud of all picture frames first, remerge target figure
As the point cloud with its previous frame image and a later frame image, key point finally is chosen from cloud, the HOPC for extracting key point is special
Sign.It is, of course, also possible to determine the key point of all picture frames first according to preset rules, target image and its former frame are remerged
The key point of image and a later frame image is finally extracted the HOPC features of all key points, is not specifically limited herein.
For example, by taking a-th cloud of F frames as an example, merge F-1, F, the point cloud of F+1 frames；Using a as the centre of sphere, r is radius,
Ask the covariance matrix and its feature vector of a；By in eigenvector projection to 20 face bodies, the HOPC features of a are finally obtained；Frame F
HOPC feature vectors be to connect the HOPC features of all key points.
S324：Judge whether all images in the sample to be identified complete by extraction, if so, into S303,
If it is not, then entering S321；
S303：Described eigenvector is inputted in the LSTM neural networks that training is completed, obtains action sequence recognition result.
A kind of action sequence identifying system provided by the embodiments of the present application is introduced below, it is described below a kind of dynamic
Making sequence identifying system can be cross-referenced with a kind of above-described action sequence recognition methods.
Referring to Fig. 4, a kind of structure chart of action sequence identifying system provided by the embodiments of the present application, as shown in figure 4, packet
It includes：
Acquisition module 401 for obtaining raw image data, and carries out pretreatment operation to the raw image data and obtains
To sample to be identified；
Extraction module 402, for in the sample to be identified each frame image carry out feature extraction, obtain feature to
Amount；
Input module 403 obtains action sequence for inputting described eigenvector in the LSTM neural networks that training is completed
Row recognition result.
Action sequence identifying system provided by the embodiments of the present application carries out pretreatment operation to raw image data first,
Sample to be identified is obtained, ensures that each frame image in sample to be identified is all of the same size and discrimination, then to each
Frame image carries out feature extraction, and in the LSTM neural networks for the feature vector input training completion that extraction is obtained, to
To action sequence recognition result.Raw image data is handled by feature extraction, then is combined with LSTM neural networks, is improved dynamic
Make the discrimination of sequence.
On the basis of the above embodiments, further include as a preferred implementation manner,：
Computing module, the average identification for calculating the sample to be identified according to the discrimination of each frame image
Rate.
On the basis of the above embodiments, as a preferred implementation manner, the pretreatment operation include turning operation,
Any one of down-sampling operation or cutting operation or several combinations.
On the basis of the above embodiments, further include as a preferred implementation manner,：
Training sample processing module is carried out for obtaining training sample, and to each frame image in the training sample
Feature extraction obtains training feature vector；
Training module for inputting the training feature vector in LSTM neural networks, and adjusts the LSTM nerves
The key parameter of network is until the discrimination of LSTM neural networks output reaches preset value, to obtain the LSTM of training completion
Neural network.
On the basis of the above embodiments, the training module is specially by the instruction as a preferred implementation manner,
Practice in feature vector input LSTM neural networks, and the LSTM god is adjusted using cross validation method and pair-wise algorithms
Key parameter through network is until the discrimination of LSTM neural networks output reaches preset value, to obtain training completion
The module of LSTM neural networks.
On the basis of the above embodiments, the propagated forward of the LSTM neural networks as a preferred implementation manner,
Algorithm includes two level derivatives of the cell to the time.
On the basis of the above embodiments, the extraction module 402 includes as a preferred implementation manner,：
Determination unit for determining target image from the sample to be identified, and obtains the former frame figure of target image
Picture and a later frame image；
Screening unit, for the target image, the previous frame image and a later frame image to be formed the mesh
The corresponding candidate image of logo image, and the point cloud by characteristic value in the candidate image more than feature preset value is determined as key
Point；
Extraction unit, the HOPC features for extracting all key points, and the mesh is determined according to all HOPC features
The feature vector of logo image；The workflow of the determination unit is restarted up to all images in the sample to be identified
All extraction is completed.
Present invention also provides a kind of action sequence identification equipments, referring to Fig. 5, a kind of action provided by the embodiments of the present application
The structure chart of recognition sequence equipment, as shown in figure 5, including：
Memory 100, for storing computer program；
Processor 200, may be implemented the step of above-described embodiment is provided when for executing the computer program.
Specifically, memory 100 includes non-volatile memory medium, built-in storage.The non-volatile memory medium stores
Have operating system and computer-readable instruction, the built-in storage be non-volatile memory medium in operating system and computer can
The operation of reading instruction provides environment.Processor 200 provides calculating and control ability for action sequence identification equipment, is deposited described in execution
When the computer program preserved in reservoir 100, following steps may be implemented：Raw image data is obtained, and to the original graph
Sample to be identified is obtained as data carry out pretreatment operation；Feature is carried out to each frame image in the sample to be identified to carry
It takes, obtains feature vector；Described eigenvector is inputted in the LSTM neural networks that training is completed, obtains action sequence identification knot
Fruit.
The embodiment of the present application carries out pretreatment operation to raw image data first, obtains sample to be identified, ensures to wait knowing
Each frame image very in this is all of the same size and discrimination, then carries out feature extraction to each frame image, and will
It extracts in the LSTM neural networks that obtained feature vector input training is completed, to obtain action sequence recognition result.Pass through
Feature extraction handles raw image data, then is combined with LSTM neural networks, and the discrimination of action sequence is improved.
Preferably, it when the processor 200 executes the computer subprogram preserved in the memory 100, may be implemented
Following steps：The average recognition rate of the sample to be identified is calculated according to the discrimination of each frame image.
Preferably, it when the processor 200 executes the computer subprogram preserved in the memory 100, may be implemented
Following steps：Training sample is obtained, and feature extraction is carried out to each frame image in the training sample, obtains training characteristics
Vector；
Preferably, it when the processor 200 executes the computer subprogram preserved in the memory 100, may be implemented
Following steps：The training feature vector is inputted in LSTM neural networks, and adjusts the crucial ginseng of the LSTM neural networks
Number is until the discrimination of LSTM neural networks output reaches preset value, to obtain the LSTM neural networks of training completion.
Preferably, it when the processor 200 executes the computer subprogram preserved in the memory 100, may be implemented
Following steps：The key parameter of the LSTM neural networks is adjusted using cross validation method and pair-wise algorithms.
Preferably, it when the processor 200 executes the computer subprogram preserved in the memory 100, may be implemented
Following steps：Target image is determined from the sample to be identified, and obtains the previous frame image and a later frame figure of target image
Picture；The target image, the previous frame image and a later frame image are formed into the corresponding candidate figure of the target image
Picture, and the point cloud by characteristic value in the candidate image more than feature preset value is determined as key point；Extract all key points
HOPC features, and determine according to all HOPC features the feature vector of the target image；It repeats the above steps until described
All images in sample to be identified all complete by extraction.
On the basis of the above embodiments, preferably, referring to Fig. 6, the action sequence identification equipment is also
Including：
Input interface 300 is connected with processor 200, computer program, parameter and instruction for obtaining external importing,
It is preserved into memory 100 through the control of processor 200.The input interface 300 can be connected with input unit, and it is manual to receive user
The parameter of input or instruction.The input unit can be the touch layer covered on display screen, can also be to be arranged in terminal enclosure
Button, trace ball or Trackpad, can also be keyboard, Trackpad or mouse etc..Specifically, in the present embodiment, Ke Yitong
It crosses input interface 300 and inputs LSTM neural network models etc..
Display unit 400 is connected with processor 200, the data sent for video-stream processor 200.The display unit 400
Can be display screen, liquid crystal display or the electric ink display screen etc. in PC machine.It, can be with specifically, in the present embodiment
The action sequence recognition result etc. of sample to be identified is shown by display unit 400.
The network port 500 is connected with processor 200, for being communicatively coupled with external each terminal device.The communication link
The communication technology used by connecing can be cable communicating technology or wireless communication technique, and such as mobile high definition chained technology (MHL) leads to
It is blue with universal serial bus (USB), high-definition media interface (HDMI), adopting wireless fidelity technology (WiFi), Bluetooth Communication Technology, low-power consumption
The tooth communication technology, the communication technology etc. based on IEEE802.11s.Specifically, in the present embodiment, the network port can be passed through
500 import LSTM neural network models etc. to processor 200.
Video collector 600 is connected with processor 200, for obtaining video data, then sends video data to place
It manages device 200 and carries out Data Analysis Services, handling result can be sent to display unit 400 and shown by subsequent processor 200,
Or be transmitted to processor 100 and preserved, or preset data receiver end can be sent to by the network port 500
End.Specifically, in the present embodiment, sample, training sample and test sample to be identified etc. can be obtained with video collector 600.
Present invention also provides a kind of computer readable storage medium, which may include：USB flash disk, mobile hard disk,
Read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic
The various media that can store program code such as dish or CD.Computer program, the calculating are stored on the storage medium
Machine program realizes following steps when being executed by processor：Raw image data is obtained, and the raw image data is carried out pre-
Processing operation obtains sample to be identified；To in the sample to be identified each frame image carry out feature extraction, obtain feature to
Amount；Described eigenvector is inputted in the LSTM neural networks that training is completed, obtains action sequence recognition result.
The embodiment of the present application carries out pretreatment operation to raw image data first, obtains sample to be identified, ensures to wait knowing
Each frame image very in this is all of the same size and discrimination, then carries out feature extraction to each frame image, and will
It extracts in the LSTM neural networks that obtained feature vector input training is completed, to obtain action sequence recognition result.Pass through
Feature extraction handles raw image data, then is combined with LSTM neural networks, and the discrimination of action sequence is improved.
Preferably, when the computer subprogram stored in the computer readable storage medium is executed by processor, specifically
Following steps may be implemented：The average recognition rate of the sample to be identified is calculated according to the discrimination of each frame image.
Preferably, when the computer subprogram stored in the computer readable storage medium is executed by processor, specifically
Following steps may be implemented：Training sample is obtained, and feature extraction is carried out to each frame image in the training sample, is obtained
Training feature vector；
Preferably, when the computer subprogram stored in the computer readable storage medium is executed by processor, specifically
Following steps may be implemented：The training feature vector is inputted in LSTM neural networks, and adjusts the LSTM neural networks
Key parameter until the LSTM neural networks output discrimination reach preset value, with obtain training completion LSTM nerve
Network.
Preferably, when the computer subprogram stored in the computer readable storage medium is executed by processor, specifically
Following steps may be implemented：The crucial ginseng of the LSTM neural networks is adjusted using cross validation method and pair-wise algorithms
Number.
Preferably, when the computer subprogram stored in the computer readable storage medium is executed by processor, specifically
Following steps may be implemented：From the sample to be identified determine target image, and obtain target image previous frame image and
A later frame image；The target image, the previous frame image and a later frame image are formed the target image to correspond to
Candidate image, and by characteristic value in the candidate image be more than feature preset value point cloud be determined as key point；Extraction is all
The HOPC features of key point, and determine according to all HOPC features the feature vector of the target image；It repeats the above steps
Until all images in the sample to be identified all complete by extraction.
Each embodiment is described by the way of progressive in this specification, the highlights of each of the examples are with other
The difference of embodiment, just to refer each other for identical similar portion between each embodiment.
The foregoing description of the disclosed embodiments enables professional and technical personnel in the field to realize or use the application.
Various modifications to these embodiments will be apparent to those skilled in the art, as defined herein
General Principle can in other embodiments be realized in the case where not departing from spirit herein or range.Therefore, the application
It is not intended to be limited to the embodiments shown herein, and is to fit to and the principles and novel features disclosed herein phase one
The widest range caused.
Each embodiment is described by the way of progressive in specification, the highlights of each of the examples are with other realities
Apply the difference of example, just to refer each other for identical similar portion between each embodiment.For system disclosed in embodiment
Speech, since it is corresponded to the methods disclosed in the examples, so description is fairly simple, related place is referring to method part illustration
?.It should be pointed out that for those skilled in the art, under the premise of not departing from the application principle, also
Can to the application, some improvement and modification can also be carried out, these improvement and modification also fall into the application scope of the claims
It is interior.
It should also be noted that, in the present specification, relational terms such as first and second and the like be used merely to by
One entity or operation are distinguished with another entity or operation, without necessarily requiring or implying these entities or operation
Between there are any actual relationship or orders.Moreover, the terms "include", "comprise" or its any other variant meaning
Covering non-exclusive inclusion, so that the process, method, article or equipment including a series of elements includes not only that
A little elements, but also include other elements that are not explicitly listed, or further include for this process, method, article or
The intrinsic element of equipment.In the absence of more restrictions, the element limited by sentence "including a ...", is not arranged
Except there is also other identical elements in the process, method, article or apparatus that includes the element.
Claims (10)
1. a kind of action sequence recognition methods, which is characterized in that including：
Raw image data is obtained, and pretreatment operation is carried out to the raw image data and obtains sample to be identified；
Feature extraction is carried out to each frame image in the sample to be identified, obtains feature vector；
Described eigenvector is inputted in the LSTM neural networks that training is completed, obtains action sequence recognition result.
2. action sequence recognition methods according to claim 1, which is characterized in that it is described obtain action sequence recognition result it
Afterwards, further include：
The average recognition rate of the sample to be identified is calculated according to the discrimination of each frame image.
3. action sequence recognition methods according to claim 1, which is characterized in that the pretreatment operation includes that overturning is grasped
Any one of work, down-sampling operation or cutting operation or several combinations.
4. action sequence recognition methods according to claim 1, which is characterized in that described eigenvector is inputted training and is completed
LSTM neural networks in before, further include：
Training sample is obtained, and feature extraction is carried out to each frame image in the training sample, obtains training feature vector；
By the training feature vector input LSTM neural networks in, and adjust the LSTM neural networks key parameter until
The discrimination of the LSTM neural networks output reaches preset value, to obtain the LSTM neural networks of training completion.
5. action sequence recognition methods according to claim 4, which is characterized in that adjust the key of the LSTM neural networks
Parameter, including：
The key parameter of the LSTM neural networks is adjusted using cross validation method and pair-wise algorithms.
6. action sequence recognition methods according to claim 4, which is characterized in that the propagated forward of the LSTM neural networks
Algorithm includes two level derivatives of the cell to the time.
7. according to any one of the claim 1-6 action sequence recognition methods, which is characterized in that in the sample to be identified
Each frame image carry out feature extraction, obtain feature vector, including：
S321：Target image is determined from the sample to be identified, and obtains the previous frame image and a later frame figure of target image
Picture；
S322：It is corresponding that the target image, the previous frame image and a later frame image are formed into the target image
Candidate image, and the point cloud by characteristic value in the candidate image more than feature preset value is determined as key point；
S323：The HOPC features of all key points are extracted, and determine the feature of the target image according to all HOPC features
Vector；
S321-S323 is repeated until all images in the sample to be identified are all extracted and completed.
8. a kind of action sequence identifying system, which is characterized in that including：
Acquisition module for obtaining raw image data, and carries out pretreatment operation to the raw image data and obtains waiting knowing
Very originally；
Extraction module obtains feature vector for carrying out feature extraction to each frame image in the sample to be identified；
Input module obtains action sequence identification for inputting described eigenvector in the LSTM neural networks that training is completed
As a result.
9. a kind of action sequence identification equipment, which is characterized in that including：
Memory, for storing computer program；
Processor realizes the action sequence identification side as described in any one of claim 1 to 7 when for executing the computer program
The step of method.
10. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium
Program realizes the action sequence recognition methods as described in any one of claim 1 to 7 when the computer program is executed by processor
The step of.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810550608.XA CN108764176A (en) | 2018-05-31 | 2018-05-31 | A kind of action sequence recognition methods, system and equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810550608.XA CN108764176A (en) | 2018-05-31 | 2018-05-31 | A kind of action sequence recognition methods, system and equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108764176A true CN108764176A (en) | 2018-11-06 |
Family
ID=64001506
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810550608.XA Withdrawn CN108764176A (en) | 2018-05-31 | 2018-05-31 | A kind of action sequence recognition methods, system and equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108764176A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109902729A (en) * | 2019-02-18 | 2019-06-18 | 清华大学 | Behavior prediction method and device based on sequence state evolution |
| CN110070052A (en) * | 2019-04-24 | 2019-07-30 | 广东工业大学 | A kind of robot control method based on mankind's demonstration video, device and equipment |
| CN110070029A (en) * | 2019-04-17 | 2019-07-30 | 北京易达图灵科技有限公司 | A kind of gait recognition method and device |
| CN110765967A (en) * | 2019-10-30 | 2020-02-07 | 腾讯科技（深圳）有限公司 | Action recognition method based on artificial intelligence and related device |
| CN111325289A (en) * | 2020-03-18 | 2020-06-23 | 中国科学院深圳先进技术研究院 | Behavior recognition method, device, equipment and medium |
| CN111444895A (en) * | 2020-05-08 | 2020-07-24 | 商汤集团有限公司 | Video processing method and device, electronic equipment and storage medium |
| CN111783650A (en) * | 2020-06-30 | 2020-10-16 | 北京百度网讯科技有限公司 | Model training method, action recognition method, device, equipment and storage medium |
| WO2020252923A1 (en) * | 2019-06-18 | 2020-12-24 | 平安科技（深圳）有限公司 | Sample data processing method and apparatus, computer apparatus, and storage medium |
| CN112580577A (en) * | 2020-12-28 | 2021-03-30 | 出门问问(苏州)信息科技有限公司 | Training method and device for generating speaker image based on face key points |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104504365A (en) * | 2014-11-24 | 2015-04-08 | 闻泰通讯股份有限公司 | System and method for smiling face recognition in video sequence |
| CN105787963A (en) * | 2016-02-26 | 2016-07-20 | 浪潮软件股份有限公司 | Video target tracking method and device |
| CN107451552A (en) * | 2017-07-25 | 2017-12-08 | 北京联合大学 | A kind of gesture identification method based on 3D CNN and convolution LSTM |
-
2018
- 2018-05-31 CN CN201810550608.XA patent/CN108764176A/en not_active Withdrawn
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104504365A (en) * | 2014-11-24 | 2015-04-08 | 闻泰通讯股份有限公司 | System and method for smiling face recognition in video sequence |
| CN105787963A (en) * | 2016-02-26 | 2016-07-20 | 浪潮软件股份有限公司 | Video target tracking method and device |
| CN107451552A (en) * | 2017-07-25 | 2017-12-08 | 北京联合大学 | A kind of gesture identification method based on 3D CNN and convolution LSTM |
Non-Patent Citations (3)
| Title |
|---|
| HOSSEIN RAHMANI 等，: "Histogram of Oriented Principal Components for Cross-View Action Recognition", 《IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE》 * |
| VIVEK VEERIAH 等，: "Differential Recurrent Neural Networks for Action Recognition", 《2015 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV)》 * |
| 于成龙，: "基于视频的人体行为识别关键技术研究", 《中国博士学位论文全文数据库信息科技辑》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109902729A (en) * | 2019-02-18 | 2019-06-18 | 清华大学 | Behavior prediction method and device based on sequence state evolution |
| CN110070029A (en) * | 2019-04-17 | 2019-07-30 | 北京易达图灵科技有限公司 | A kind of gait recognition method and device |
| CN110070052A (en) * | 2019-04-24 | 2019-07-30 | 广东工业大学 | A kind of robot control method based on mankind's demonstration video, device and equipment |
| WO2020252923A1 (en) * | 2019-06-18 | 2020-12-24 | 平安科技（深圳）有限公司 | Sample data processing method and apparatus, computer apparatus, and storage medium |
| CN110765967A (en) * | 2019-10-30 | 2020-02-07 | 腾讯科技（深圳）有限公司 | Action recognition method based on artificial intelligence and related device |
| CN110765967B (en) * | 2019-10-30 | 2022-04-22 | 腾讯科技（深圳）有限公司 | Action recognition method based on artificial intelligence and related device |
| CN111325289A (en) * | 2020-03-18 | 2020-06-23 | 中国科学院深圳先进技术研究院 | Behavior recognition method, device, equipment and medium |
| CN111444895A (en) * | 2020-05-08 | 2020-07-24 | 商汤集团有限公司 | Video processing method and device, electronic equipment and storage medium |
| CN111444895B (en) * | 2020-05-08 | 2024-04-19 | 商汤集团有限公司 | Video processing method, device, electronic equipment and storage medium |
| CN111783650A (en) * | 2020-06-30 | 2020-10-16 | 北京百度网讯科技有限公司 | Model training method, action recognition method, device, equipment and storage medium |
| CN112580577A (en) * | 2020-12-28 | 2021-03-30 | 出门问问(苏州)信息科技有限公司 | Training method and device for generating speaker image based on face key points |
| CN112580577B (en) * | 2020-12-28 | 2023-06-30 | 出门问问(苏州)信息科技有限公司 | Training method and device for generating speaker image based on facial key points |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108764176A (en) | A kind of action sequence recognition methods, system and equipment and storage medium | |
| CN108830235B (en) | Method and apparatus for generating information | |
| CN108154105B (en) | Underwater biological detection and identification method and device, server and terminal equipment | |
| CN108229489A (en) | Crucial point prediction, network training, image processing method, device and electronic equipment | |
| CN110909630B (en) | Abnormal game video detection method and device | |
| US10719693B2 (en) | Method and apparatus for outputting information of object relationship | |
| CN110362494B (en) | Method for displaying microservice state information, model training method and related device | |
| CN110175502A (en) | A kind of backbone Cobb angle measuring method, device, readable storage medium storing program for executing and terminal device | |
| CN108229478A (en) | Image, semantic segmentation and training method and device, electronic equipment, storage medium and program | |
| CN110930296B (en) | Image processing method, device, equipment and storage medium | |
| AU2011254040B2 (en) | Method, apparatus and system for determining a saliency map for an input image | |
| JP2021532434A (en) | Face feature extraction model Training method, face feature extraction method, device, equipment and storage medium | |
| CN108229262B (en) | Pornographic video detection method and device | |
| US11669990B2 (en) | Object area measurement method, electronic device and storage medium | |
| KR101955919B1 (en) | Method and program for providing tht region-of-interest in image by deep-learing algorithm | |
| WO2020181706A1 (en) | Plant species identification method and apparatus | |
| CN110197183A (en) | A kind of method, apparatus and computer equipment of Image Blind denoising | |
| CN105979283A (en) | Video transcoding method and device | |
| CN113191478A (en) | Training method, device and system of neural network model | |
| CN114037003A (en) | Question-answer model training method and device and electronic equipment | |
| CN108921138B (en) | Method and apparatus for generating information | |
| CN109241930B (en) | Method and apparatus for processing eyebrow image | |
| CN111062914B (en) | Method, apparatus, electronic device and computer readable medium for acquiring facial image | |
| CN116912187A (en) | Image generation model training and image generation method, device, equipment and medium | |
| CN110490065B (en) | Face recognition method and device, storage medium and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20181106 |