CN104239119B - A kind of method and system that training on electric power emulation is realized based on kinect - Google Patents
A kind of method and system that training on electric power emulation is realized based on kinect Download PDFInfo
- Publication number
- CN104239119B CN104239119B CN201410428230.8A CN201410428230A CN104239119B CN 104239119 B CN104239119 B CN 104239119B CN 201410428230 A CN201410428230 A CN 201410428230A CN 104239119 B CN104239119 B CN 104239119B
- Authority
- CN
- China
- Prior art keywords
- arm
- action
- kinect
- training
- emulation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The invention discloses a kind of method and system that training on electric power emulation is realized based on kinect, this method includes：The movement posture information of start-up is obtained based on kinect；The method being combined using template matches and Dynamic Programming goes out the action classification of start-up come Dynamic Recognition；The context of the training on electric power emulation corresponding to action classification and action classification is described using finite state machine, the transfer of simulation status is completed using the action classification of start-up as transfer function；The logic change of finite states machine control emulation behavior is described based on XML, and is shown by three dimensional virtual technique.The method that the embodiment of the present invention strengthens training on electric power experience sense using kinect, training on electric power personnel can be made come the virtual member in operational simulation, start-up is had sensation on the spot in person by limb action.
Description
Technical field
The present invention relates to Simulating technique in Electric Power System field, and in particular to a kind of to realize that training on electric power emulates based on kinect
Method and system.
Background technology
The experience sense of enhancing user is always a direction of training on electric power emulation technology development, it now is possible to makes trainer
Member roams in the three-dimensional scenics such as transformer station, but needs constantly to move or click on mouse, or operation keyboard in training
To be control effectively to computer, or certain handheld device is needed, these all reduce the experience sense of trainer.With
Kinect appearance, body-sensing technology can be utilized to obtain the depth information of start-up, by the identification to limbs joint, to manage
The action for solving start-up is intended to, so as to be operated effectively to electric analog system.
Kinect is a posture sensing input equipment developed by Microsoft, and it is mainly by main camera, one
Depth transducer, one group of microphone and a motor are formed.Kinect possesses dynamic seizure immediately, image identification, microphone
The multiple functions such as input, speech recognition, community interactive.In order to excavate Kinect bigger potentiality, Microsoft has issued
Kinect PC drivings and programming SDK, so as to attract more developers to carry out the game and application based on Kinect
Exploitation.Therefore many novel applications are arisen at the historic moment, such as manipulation of virtual mirror, robot, virtual guitar.Wherein have suitable one
Certain applications combine virtual reality technology or augmented reality.These applications have been related to military, medical treatment, amusement, religion
Educate and the various fields such as train, but on not being employed also in training on electric power emulation field, its reason is that training on electric power is related to
To many equipment and working specification of power system.In consideration of it, Kinect, which is applied in training on electric power emulation, can also run into class
As problem, so the emulation for needing to combine in training on electric power to working specification plays the advantage of Kinect technologies as far as possible.
The content of the invention
Based on the deficiency of existing training on electric power analogue system, by (being used cooperatively base with the use of some traditional algorithms
In the method such as template matching method of template, dynamic programming；And with the use of the method based on grammer, such as finite state machine skill
Art etc.), make start-up that there is good experience sense.
The invention provides a kind of method that training on electric power emulation is realized based on kinect, comprise the following steps：
The movement posture information of start-up is obtained based on kinect；
The method being combined using template matches and Dynamic Programming goes out the action classification of start-up come Dynamic Recognition；
The context of the training on electric power emulation corresponding to action classification and action classification is subject to using finite state machine
Description, the transfer of simulation status is completed using the action classification of start-up as transfer function；
The logic change of finite states machine control emulation behavior is described based on XML, and is shown by three dimensional virtual technique
Come.
The movement posture information that start-up is obtained based on kinect is included：
The pretreatment of the limbs nodal information obtained to kinect, segmentation, the extraction of boundary profile and limbs node
Identification, the position sequence information and direction of motion vector of action are obtained from continuous multiple frames.
The action classification that the method being combined using template matches and Dynamic Programming goes out start-up come Dynamic Recognition includes：
The identification of finger movement is carried out using the pattern matching algorithm of dynamic time warping；
The direction of motion is obtained from finger position twice, and the direction of motion is normalized.
It is described using finite state machine by corresponding to action classification and action classification training on electric power emulate context come
Described, the transfer step that simulation status is completed using the action classification of start-up as transfer function is specially：
According to the definition for determining finite state machine:AD=(S, ∑, δ, K, F), wherein:
S：The state of the nonvoid set of state, referred to start-up, and the execution of corresponding emulation behavior；
∑：Corresponding character set, referred to emulation training human behavior, are showed with three dimensional virtual technique；
δ:State transition function, S × ∑ → S single-value mapping；
δ(Si, a)=Sj, Si, Sj∈ S, referred to emulation training personnel a behavior outcome, with three dimensional virtual technique come
Show；A ∈ ∑s, refer to a certain behavior of emulation training personnel；
K：Original state；；
F：Final state；；
Transfer function δ includes the action of two legs and two-arm.
Accordingly, the embodiment of the present invention additionally provides a kind of system that training on electric power emulation is realized based on kinect, including：
Movement posture information acquisition module, for obtaining the movement posture information of start-up based on kinect；
Action classification identification module, the method for being combined using template matches and Dynamic Programming go out training come Dynamic Recognition
The action classification of instruction personnel；
Shift module is emulated, for the training on electric power corresponding to action classification and action classification being imitated using finite state machine
Genuine context is described, and the transfer of simulation status is completed using the action classification of start-up as transfer function；
Display module is emulated, for being changed using XML language to describe the logic of finite states machine control emulation behavior, and
Shown by three dimensional virtual technique.
The movement posture identification module is used for the pretreatment, segmentation, side of the limbs nodal information obtained to kinect
The extraction of boundary's profile and the identification of limbs node, the position sequence information and direction of motion arrow of action are obtained from continuous multiple frames
Amount.
The action classification identification module is used to carry out finger movement using the pattern matching algorithm of dynamic time warping
Identification；And the direction of motion is obtained from finger position twice, and the direction of motion is normalized.
Combination of embodiment of the present invention training on electric power emulates the characteristics of, typical template matches and dynamic programming algorithm are utilized
The action obtained to kinect is identified, while emulation behavior is controlled using finite state machine technology, and is being emulated
State machine is described with XML language in prediction scheme, so not only increases the experience sense of training on electric power, is also enhanced this imitative
The flexibility really applied.Strengthen the method for training on electric power experience sense using kinect, training on electric power personnel can be made to pass through limbs
The virtual member that action comes in operational simulation, makes start-up have sensation on the spot in person.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is the required accompanying drawing used in technology description to be briefly described, it should be apparent that, drawings in the following description are only this
Some embodiments of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can be with
Other accompanying drawings are obtained according to these accompanying drawings.
Fig. 1 is the method flow diagram that training on electric power emulation is realized based on kinect in the embodiment of the present invention；
Fig. 2 is the method state diagram based on kinect enhancing training on electric power experience senses in the embodiment of the present invention；
Fig. 3 is the system structure diagram that training on electric power emulation is realized based on kinect in the embodiment of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation describes, it is clear that described embodiment is only part of the embodiment of the present invention, rather than whole embodiments.It is based on
Embodiment in the present invention, those of ordinary skill in the art are obtained all other under the premise of creative work is not made
Embodiment, belong to the scope of protection of the invention.
In order to make full use of Kinect body-sensing technology, it is necessary to which a variety of intelligent identification technologies are added in being emulated in training on electric power
With combination.The present invention identifies the action message acquired in Kinect using template matches and dynamic programming techniques, and according to pre-
The description of finite state machine in case, the state of the action control state machine identified is shifted, complete the control to emulating behavior.
To reach above-mentioned purpose, it is necessary to the limbs nodal information that is obtained to Kinect carries out static segmentation and identification, then
With reference to the continuity of action in time, using template matching method and dynamic programming techniques are combined to identify the one of start-up
Individual action is intended to.These actions are applied in training, also finite state machine need to be utilized in simulation scenario before emulation starts
To be intended to be described to the everything of start-up, the action body of start-up is controlled by state machine after simulation run
Test.
Fig. 1 shows the method flow diagram that training on electric power emulation is realized based on kinect in the embodiment of the present invention, including
Following steps：
S101, the movement posture information based on kinect acquisition start-ups；
Static analysis first is made to the kinect limbs nodal informations obtained, including pre-processes, split, the extraction of border wheel Guo
And the identification of limbs node；Further according to the time course of action, the position of action is obtained in the continuous multiple frames obtained from kinect
Sequence information, thus sequence construct direction of motion vector.Advised finally by this direction of motion vector using template matches and dynamic
Draw the method being combined and carry out the action classification that Dynamic Recognition goes out start-up.
Fig. 2 shows the method state diagram based on kinect enhancing training on electric power experience senses, according to Kinect
The 3D positions for the real-time bone node that forwindows sdk are provided, a certain moment can obtain the position of node, so as to know
Angle and relative position between node.In the case of continuous real-time capture, the motion vector of node can be obtained.Again by
The motion vector of artis come identify the action of start-up indicate.These identification process are included to two legs, two-arm and two hands
Identify, but according to the needs of Simulated training, the identification of leg and arm need to only be judged to lift leg, receive leg, carry arm, drop arm, left swing arm
With right swing arm；And the identification to palm then includes changing hands, clench fist, stretch out the bending of the five fingers and each finger and stretch.
Identification to above-mentioned limb action is all divided into following static identification and two steps of Dynamic Recognition.Known with gesture
Exemplified by not, static state identification key step includes pretreatment, Hand Gesture Segmentation, gestures detection, feature extraction；Dynamic process includes the time
The optimization of sequence and template matches etc..Pretreatment based on Kinect dynamic hand gesture recognitions refers in dynamic gesture training and gesture
Before identification, the depth information flow data and bone nodal information flow data obtained from Kinect is pre-processed, need
The hand to be identified is split from whole data field, to reduce follow-up amount of calculation.Complete and then use in Hand Gesture Segmentation
Border-Following algorithms carry out contour detecting processing to segmented picture.
Border-Following algorithms comprise the concrete steps that setpoint color threshold value first, when less than the threshold value, the pixel
Point is arranged to white.When more than the threshold value, the pixel is configured to certain predefined color (as red).Then distinguish
Each pixel in scanned picture successively from different directions, when the pixel of the left and right adjoined therewith two is different value, then the picture
Vegetarian refreshments is boundary point, so as to find out the profile of image.
Finger tip is distinguished with K-curvature algorithms again, the point on image outline is sequentially stored into array, utilizes K-
Curvature algorithms calculate the approximate curvature at any point.I.e. to the J point point (J) on any profile, point (J) is calculated
Between vector and point (J) and point (J+K) between point (J-K) vectorial angle whether within the specific limits, from
And judge whether it is finger tip.K takes 20 herein, the scope for judging angle be 0 to 55 degree between., can if selected angle is excessive
Causing hand, that section is erroneously identified as finger tip close to arm；And if selected angle is too small, cause when finger is shorter
Cannot be discernable by its.
Because the Kinect frame per second of depth information stream provided is 30FPS, thus even for 2 seconds kind gesture and
Speech, the frame captured in real time constitute a time series close to 60 frames.To same gesture, different people or different time
If completing, the length of the time series is all not quite similar, while continues frame number in a certain direction and be also not quite similar.This is just
Cause to train and identify that difficulty increases, thus be necessary to optimize the length of time series, remove unnecessary information, protect
The characteristic vector for marking the gesture characteristic is stayed, i.e., sampling processing is carried out to the vector after handling after filtering.
S102, the method being combined using template matches and Dynamic Programming go out the action class of start-up come Dynamic Recognition
Not；
The process of dynamic hand gesture recognition is to extract some features to match the action of gesture, present invention selection position and fortune
Dynamic direction vector is as characteristic quantity.Position feature be gesture in each point arrive starting point distance, direction vector for each point with it is previous
The direction of motion that individual point is formed is vectorial (being typically normalized), because position and speed are all scalar, it is impossible to correctly retouch
State the order of hands movement track.And in the case of position is as feature, same gesture, when actuating range is of different sizes, value is different,
So less desirable selection.And speed is for same track, the speed that different people waves the different time is different, no
Gesture can be marked well.So characteristic value is finally used as using direction of motion vector.
Template matching algorithm be it is wherein the simplest realize, that is, establish a template, allow the characteristic of each action
Classification is carried out with template, gesture is differentiated by both similarities, because the time of same gesture has with short, causes to know
Not rate is relatively low.Dynamic time warping (DTW) algorithm need to be used for this, it is a kind of pattern with non-linear normalizing effect
With algorithm, by using to characteristic signal elongate or shorten until consistent with the length of mode standard, to cause and template
Preferably match.Because the application background of the present invention is the experience sense of enhancing start-up, letter can be carried out to the action of operation
Single fixation, therefore the identification of finger movement is carried out using DTW algorithms.
The direction of motion is obtained from finger position twice, and the direction of motion is normalized, finger Y-coordinate below
Y-coordinate less than previous finger represents upward, value 1；Finger Y-coordinate below be more than previous finger Y-coordinate represent to
Under, value is -1；Within the specific limits, then direction value is 0 to the Y-coordinate of finger position twice.The motion sequence of such a finger
(0 ..., 1,1,1 ..., 1,0) or (0 ..., -1, -1, -1 ..., -1,0) is classified as, then the former finger is upward, and the latter represents
Finger is downward.If five fingers are simultaneously downward, then it represents that clench fist；Then represent to stretch the palm upwards simultaneously.The motion sequence of arm is
(0 ..., 1,1,1 ..., 1,0) carry arm is represented；Drop arm is represented for (0 ..., -1, -1, -1 ..., -1,0).The motion sequence of leg is
(0 ..., 1,1,1 ..., 1,0) lift leg is represented；Represent that leg is upright for (0 ..., -1, -1, -1 ..., -1,0).
Equally judge the direction of motion with the difference of the X-axis coordinate of front and rear two positions, by taking right arm as an example, arm X below
The X-coordinate that coordinate is less than previous arm represents inwardly to wave, value 1；The Y that arm Y-coordinate below is more than previous arm is sat
Mark represents outwards to wave, and value is -1；Within the specific limits, then direction value is 0 to the X-coordinate of arm position twice.Such right hand
The motion sequence of arm is (0 ..., 1,1, ..., 1,0) or (0 ..., -1, -1, -1 ..., -1,0), then the former right arm to
Inside wave, the latter represents that right arm is outwards waved.Represent spinning movement with the front and rear wheel Guo area of the left hand palm, area diminish for
1, area becomes greatly -1, and difference in areas is 0 within the specific limits, then motion sequence is that (0 ..., 1,1,1 ..., 1,0) represents rotation,
Motion sequence is (0 ..., -1, -1, -1 ..., -1,0) temporarily without definition.
S103, using finite state machine by corresponding to action classification and action classification training on electric power emulate context come
Described, the transfer of simulation status is completed using the action classification of start-up as transfer function；
The context that the module combines emulation is intended to understand the action of start-up, and promotes emulation according to this intention
Process.Finite state machine is the powerful for describing context grammer, and this module is used as conversion letter using the action of start-up
Number carrys out the transfer of completion status.
According to the definition for determining finite state machine:AD=(S, ∑, δ, K, F), wherein:
S：The state of the nonvoid set of state, referred to start-up, and the execution of corresponding emulation behavior；
∑：Corresponding character set, referred to emulation training human behavior, are showed with three dimensional virtual technique；
δ:State transition function, S × ∑ → S single-value mapping；
δ(Si, a)=Sj, Si, Sj∈ S, referred to emulation training personnel a behavior outcome, with three dimensional virtual technique come
Show；A ∈ ∑s, refer to a certain behavior of emulation training personnel；
K：Original state；；
F：Final state；；
Transfer function δ includes the action of two legs and two-arm, specially lifts leg, vertical leg, carry arm, drop arm, left swing arm and the right side
Swing arm；And two hand action, specially change hands, clench fist, stretch out the bending of the five fingers and each finger and stretch.
Character set ∑ is that transfer function δ behavior shows, include it is forward, upright, squat down, turn round, bending over, carry arm, drop
Arm, the bending changing hands, clench fist, stretching out the five fingers and each finger and stretch.Transfer function δ is changed for state of a control, is such as used
The left-hand rotation or right-hand rotation of arm control showing for start-up's behavior representated by ∑, can represent start-up with left-hand rotation arm
Turn left, or represent that start-up turns right with right-hand rotation arm.
By taking the overhaul of the equipments process in Fig. 2 as an example, start-up needs first to go at equipment, include it is forward, turn, be straight
Stand, squat down and the several states of operation." lift leg " this limb action makes to move ahead and erectility all goes to forward traveling；" arm
This limb action of translation " makes forward traveling go to turn condition；" vertical leg " this limb action makes to move ahead or turn condition turns
Move on to erectility；" arm moves down " this limb action makes erectility go to the state of squatting down, and " arm moves up " this limbs move
Work makes the state of squatting down go to erectility；" change hands, clench fist, stretching out the bending of the five fingers and each finger and stretch " these palm actions
Start-up is set to enter job state, and untill terminating.
S104, the logic change for describing based on XML finite states machine control emulation behavior, and pass through three dimensional virtual technique exhibition
Show to come.
The operation of start-up is described as an artificial tasks in simulation scenario, is related to when completing the operation
To some action behaviors of start-up, one finite state machine of logical constitution between these behaviors, therefore the module is in prediction scheme
It is middle to describe the logic between this behavior with XML language.
Changed in advance with XML to describe the logic of finite states machine control emulation behavior in emulation case, first to each operation
Task is described：
<Task>Task name</ task>
The state set that the task includes is described as：
Wherein state name includes：Move ahead, be upright, squat down, turn round, bending over, carry arm, drop arm, change hands, clench fist, stretching out the five fingers
And each finger bending and stretch.Action name includes：Lift leg, vertical leg, carry arm, drop arm, left swing arm and right swing arm；And two hand
Change hands, clench fist, stretching out the bending of the five fingers and each finger and stretch.
The job task representated by finite state machine is described with XML in simulation scenario, makes start-up in emulation
Operation behaviour editable.
Accordingly, Fig. 3 also show the system knot that training on electric power emulation is realized based on kinect in the embodiment of the present invention
Structure schematic diagram, the system include：
Movement posture information acquisition module, for obtaining the movement posture information of start-up based on kinect；
Action classification identification module, the method for being combined using template matches and Dynamic Programming go out training come Dynamic Recognition
The action classification of instruction personnel；
Shift module is emulated, for the training on electric power corresponding to action classification and action classification being imitated using finite state machine
Genuine context is described, and the transfer of simulation status is completed using the action classification of start-up as transfer function；
Display module is emulated, for being changed using XML language to describe the logic of finite states machine control emulation behavior, and
Shown by three dimensional virtual technique.
It should be noted that the movement posture identification module is used for the pre- place of the limbs nodal information obtained to kinect
Reason, segmentation, the extraction of boundary profile and the identification of limbs node, the position sequence information and fortune of action are obtained from continuous multiple frames
Dynamic direction vector.
It should be noted that the action classification identification module is used to carry out using the pattern matching algorithm of dynamic time warping
The identification of finger movement；And the direction of motion is obtained from finger position twice, and the direction of motion is normalized.Action
Gesture recognition module first makees static analysis to the kinect limbs nodal informations obtained, including pre-processes, splits, boundary profile
Extraction and the identification of limbs node；Further according to the time course of action, calculated in the continuous multiple frames obtained from kinect dynamic
The position sequence information of work, thus sequence construct go out direction of motion vector, its vector element be 0,1 and -1, represent respectively it is motionless,
It is moved upwardly or downwardly；Due to including 0 position in vector of number and 0 element in this direction of motion vector with template not to the utmost
It is identical, therefore further employ template matches and method that Dynamic Programming is combined ignores the influence of above-mentioned 0 element, so as to dynamic
Identify to state the action classification of start-up.Emulation shift module understands the action of start-up with reference to the context of emulation
It is intended to, and the operation of emulation is promoted according to this intention；It describes context grammer using finite state machine, with start-up
Action carry out the transfer of completion status as transfer function.Action classification identification module makees limbs using the kinect information obtained
Node is identified, and the position sequence information and direction of motion vector of action are obtained from continuous multiframe, because action is fast every time
Degree is all not quite similar, and the position in direction of motion vector comprising 0 number and 0 element in vector is not quite similar with template, according to
The method that this vector is combined using template matches and Dynamic Programming goes out the action classification of start-up come Dynamic Recognition.Emulation exhibition
Show in module and describe the job task representated by finite state machine with XML in simulation scenario, make start-up in emulation
Operation behaviour editable.By obtaining the status information of start-up based on kinect, and it is subject to action recognition, so as to by having
Limit state machine and realize emulation conversion, and be input in three-dimensional and shown based on XML, so as to increase the body-sensing of start-up
Degree.
To sum up, combination of embodiment of the present invention training on electric power emulates the characteristics of, typical template matches and dynamic is utilized to advise
The action that the method for calculating is obtained to kinect is identified, while emulation behavior is controlled using finite state machine technology, and
State machine is described with XML language in simulation scenario, the experience sense of training on electric power is so not only increased, also enhances
The flexibility of this Simulation Application.
One of ordinary skill in the art will appreciate that all or part of step in the various methods of above-described embodiment is can
To instruct the hardware of correlation to complete by program, the program can be stored in a computer-readable recording medium, storage
Medium can include：Read-only storage (ROM, Read Only Memory), random access memory (RAM, Random
Access Memory), disk or CD etc..
The embodiment of the present invention is provided above realizes that the method and system of training on electric power emulation are carried out based on kinect
It is discussed in detail, specific case used herein is set forth to the principle and embodiment of the present invention, above example
Explanation be only intended to help understand the present invention method and its core concept；Meanwhile for those of ordinary skill in the art,
According to the thought of the present invention, there will be changes in specific embodiments and applications, in summary, in this specification
Appearance should not be construed as limiting the invention.
Claims (5)
- A kind of 1. method that training on electric power emulation is realized based on kinect, it is characterised in that comprise the following steps：The movement posture information of start-up is obtained based on kinect, including：There is provided according to Kinect for windows sdk Real-time bone node 3D positions, a certain moment obtains the position of node, so as to know angle between node and relative Position；In the case of continuous real-time capture, the motion vector of node is obtained；Training is identified by the motion vector of artis again The action instruction of instruction personnel；These identification process include identifying two legs, two-arm and two hands, according to the needs of Simulated training, The identification of leg and arm need to only be judged to lift leg, receive leg, carry arm, drop arm, left swing arm and right swing arm；Identification to palm then includes turning Hand, clench fist, stretch out the bending of the five fingers and each finger and stretch；And including：The depth information flow data and bone nodal information flow data obtained from Kinect is pre-processed with And contour detecting processing is carried out to segmented picture using Border-Following algorithms, then with K-curvature Algorithm distinguishes finger tip, the point on image outline is sequentially stored into array, any point is calculated using K-curvature algorithms Approximate curvature；The length of time series of same gesture is optimized, retains the characteristic vector of mark gesture characteristic；The method being combined using template matches and Dynamic Programming goes out the action classification of start-up come Dynamic Recognition, using dynamic Time alignment DTW algorithms, by using to characteristic signal elongate or shorten until consistent with the length of mode standard, so that Obtain and preferably matched with template, according to the time course of action, action is calculated in the continuous multiple frames obtained from kinect Position sequence information, thus sequence construct go out direction of motion vector, its vector element be 0,1 and -1, represent respectively it is motionless, upwards Or move downward；The context of the training on electric power emulation corresponding to action classification and action classification is described using finite state machine, The transfer of simulation status is completed using the action classification of start-up as transfer function, according to determining for determination finite state machine Justice:AD=(S, ∑, δ, K, F), wherein：S：The state of the nonvoid set of state, referred to start-up, and the execution of corresponding emulation behavior；∑：Corresponding character set, referred to emulation training human behavior, are showed with three dimensional virtual technique；δ:State transition function, S × ∑ → S single-value mapping；δ(Si, a)=Sj, Si, Sj∈ S, referred to emulation training personnel a behavior outcome, are showed with three dimensional virtual technique； A ∈ ∑s, refer to a certain behavior of emulation training personnel；K：Original state；F：Final state；Transfer function δ includes the action of two legs and two-arm, specially lifts leg, vertical leg, carry arm, drop arm, left swing arm and right swing arm； And two hand action, specially change hands, clench fist, stretch out the bending of the five fingers and each finger and stretch；Character set ∑ is that transfer function δ behavior shows, include it is forward, upright, squat down, turn round, bending over, carry arm, drop arm, turn Hand, the bending clenched fist, stretch out the five fingers and each finger and stretch；Transfer function δ is changed for state of a control, and the start-up representated by ∑ is controlled with the left-hand rotation or right-hand rotation of arm Behavior shows, and represents that start-up turns left with left-hand rotation arm, or represent that start-up turns right with right-hand rotation arm；The logic change of finite states machine control emulation behavior is described based on XML, and is shown by three dimensional virtual technique, is wrapped Include：Each job task is described, description content is as follows：<Task>Task name</ task>The state set that the task includes is described as：Wherein state name includes：Move ahead, be upright, squat down, turn round, bending over, carry arm, drop arm, change hands, clench fist, stretching out the five fingers and each The bending of finger and stretch；Action name includes：Lift leg, vertical leg, carry arm, drop arm, left swing arm and right swing arm；And two hand turn Hand, the bending clenched fist, stretch out the five fingers and each finger and stretch.
- 2. the method for training on electric power emulation is realized based on kinect as claimed in claim 1, it is characterised in that described to be based on The movement posture information that kinect obtains start-up includes：Pretreatment, segmentation, the extraction of boundary profile and the identification of limbs node of the limbs nodal information obtained to kinect, The position sequence information and direction of motion vector of action are obtained from continuous multiple frames.
- 3. the method for training on electric power emulation is realized based on kinect as claimed in claim 2, it is characterised in that using template With carrying out Dynamic Recognition with the method that Dynamic Programming is combined and go out the action classification of start-up to include：The identification of finger movement is carried out using the pattern matching algorithm of dynamic time warping；The direction of motion is obtained from finger position twice, and the direction of motion is normalized.
- A kind of 4. system that training on electric power emulation is realized based on kinect, it is characterised in that including：Movement posture information acquisition module, for obtaining the movement posture information of start-up based on kinect, including：Including： According to the 3D positions of the Kinect for windows sdk real-time bone nodes provided, a certain moment obtains the position of node, So as to know the angle and relative position between node；In the case of continuous real-time capture, the motion vector of node is obtained； Identify that the action of start-up indicates by the motion vector of artis again；These identification process include to two legs, two-arm and Two hands identify, according to the needs of Simulated training, the identification of leg and arm need to only be judged to lift leg, receive leg, carry arm, drop arm, left swing Arm and right swing arm；Identification to palm then includes changing hands, and clenches fist, stretchs out the bending of the five fingers and each finger and stretch；And including：The depth information flow data and bone nodal information flow data obtained from Kinect is pre-processed with And contour detecting processing is carried out to segmented picture using Border-Following algorithms, then with K-curvature Algorithm distinguishes finger tip, the point on image outline is sequentially stored into array, any point is calculated using K-curvature algorithms Approximate curvature；The length of time series of same gesture is optimized, retains the characteristic vector of mark gesture characteristic；Action classification identification module, the method for being combined using template matches and Dynamic Programming go out trainer come Dynamic Recognition The action classification of member, using dynamic time warping DTW algorithms, by using characteristic signal elongate or shorten until with mark The length of quasi-mode is consistent, preferably to match with template, according to the time course of action, and the company obtained from kinect The position sequence information of action is calculated in continuous multiframe, thus sequence construct goes out direction of motion vector, and its vector element is 0,1 With -1, represent motionless, be moved upwardly or downwardly respectively；Shift module is emulated, for the training on electric power corresponding to action classification and action classification is emulated using finite state machine Context is described, and the transfer of simulation status is completed using the action classification of start-up as transfer function, according to true Determine the definition of finite state machine:AD=(S, ∑, δ, K, F), wherein：S：The state of the nonvoid set of state, referred to start-up, and the execution of corresponding emulation behavior；∑：Corresponding character set, referred to emulation training human behavior, are showed with three dimensional virtual technique；δ:State transition function, S × ∑ → S single-value mapping；δ(Si, a)=Sj, Si, Sj∈ S, referred to emulation training personnel a behavior outcome, are showed with three dimensional virtual technique； A ∈ ∑s, refer to a certain behavior of emulation training personnel；K：Original state；F：Final state；Transfer function δ includes the action of two legs and two-arm, specially lifts leg, vertical leg, carry arm, drop arm, left swing arm and right swing arm； And two hand action, specially change hands, clench fist, stretch out the bending of the five fingers and each finger and stretch；Character set ∑ is that transfer function δ behavior shows, include it is forward, upright, squat down, turn round, bending over, carry arm, drop arm, turn Hand, the bending clenched fist, stretch out the five fingers and each finger and stretch；Transfer function δ is changed for state of a control, and the start-up representated by ∑ is controlled with the left-hand rotation or right-hand rotation of arm Behavior shows, and represents that start-up turns left with left-hand rotation arm, or represent that start-up turns right with right-hand rotation arm；Display module is emulated, for being changed using XML language to describe the logic of finite states machine control emulation behavior, and is passed through Three dimensional virtual technique is shown, including：Each job task is described, description content is as follows：<Task>Task name</ task>The state set that the task includes is described as：Wherein state name includes：Move ahead, be upright, squat down, turn round, bending over, carry arm, drop arm, change hands, clench fist, stretching out the five fingers and each The bending of finger and stretch；Action name includes：Lift leg, vertical leg, carry arm, drop arm, left swing arm and right swing arm；And two hand turn Hand, the bending clenched fist, stretch out the five fingers and each finger and stretch.
- 5. the system of training on electric power emulation is realized based on kinect as claimed in claim 4, it is characterised in that the action class Other identification module carries out the identification of finger movement using the pattern matching algorithm of dynamic time warping；And from finger position twice The direction of motion is obtained, and the direction of motion is normalized.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410428230.8A CN104239119B (en) | 2014-08-27 | 2014-08-27 | A kind of method and system that training on electric power emulation is realized based on kinect |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410428230.8A CN104239119B (en) | 2014-08-27 | 2014-08-27 | A kind of method and system that training on electric power emulation is realized based on kinect |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104239119A CN104239119A (en) | 2014-12-24 |
| CN104239119B true CN104239119B (en) | 2017-11-21 |
Family
ID=52227254
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410428230.8A Active CN104239119B (en) | 2014-08-27 | 2014-08-27 | A kind of method and system that training on electric power emulation is realized based on kinect |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104239119B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104575150B (en) * | 2015-01-15 | 2016-11-09 | 广东电网有限责任公司教育培训评价中心 | The method and apparatus of many people online cooperation and system for electric analog training |
| CN104915011A (en) * | 2015-06-28 | 2015-09-16 | 合肥金诺数码科技股份有限公司 | Open environment gesture interaction game system |
| CN106607910B (en) * | 2015-10-22 | 2019-03-22 | 中国科学院深圳先进技术研究院 | A kind of robot imitates method in real time |
| CN110221686B (en) * | 2019-04-29 | 2020-08-04 | 浙江大学 | Mine resource management-oriented gesture interaction system and method based on color images |
| CN111028637B (en) * | 2019-12-27 | 2022-01-25 | 广东电网有限责任公司培训与评价中心 | Live-line work training method and system based on electric field visualization |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1924899A (en) * | 2006-09-26 | 2007-03-07 | 福建榕基软件开发有限公司 | Precise location method of QR code image symbol region at complex background |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102163382B (en) * | 2011-04-29 | 2013-07-31 | 国网电力科学研究院武汉南瑞有限责任公司 | Live working helmet type simulation training system and method |
| CN103529944B (en) * | 2013-10-17 | 2016-06-15 | 合肥金诺数码科技股份有限公司 | A kind of human motion recognition method based on Kinect |
-
2014
- 2014-08-27 CN CN201410428230.8A patent/CN104239119B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1924899A (en) * | 2006-09-26 | 2007-03-07 | 福建榕基软件开发有限公司 | Precise location method of QR code image symbol region at complex background |
Non-Patent Citations (1)
| Title |
|---|
| 一种基于Kinect的虚拟现实姿态交互工具;鲁明等;《系统仿真学报》;20130930;第25卷(第9期);全文，图3，图5和图6 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104239119A (en) | 2014-12-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10664060B2 (en) | Multimodal input-based interaction method and device | |
| CN105868715B (en) | Gesture recognition method and device and gesture learning system | |
| Plouffe et al. | Static and dynamic hand gesture recognition in depth data using dynamic time warping | |
| Trejo et al. | Recognition of yoga poses through an interactive system with kinect device | |
| CN104239119B (en) | A kind of method and system that training on electric power emulation is realized based on kinect | |
| CN103941866B (en) | Three-dimensional gesture recognizing method based on Kinect depth image | |
| Barros et al. | A dynamic gesture recognition and prediction system using the convexity approach | |
| CN108256421A (en) | A kind of dynamic gesture sequence real-time identification method, system and device | |
| CN104331164B (en) | A kind of gesture motion smoothing processing method of the similarity threshold analysis based on gesture identification | |
| CN110020633A (en) | Training method, image-recognizing method and the device of gesture recognition model | |
| Shin et al. | Gesture recognition using Bezier curves for visualization navigation from registered 3-D data | |
| CN107450714A (en) | Man-machine interaction support test system based on augmented reality and image recognition | |
| CN102222431A (en) | Hand language translator based on machine | |
| CN110073369A (en) | The unsupervised learning technology of time difference model | |
| Mohanty et al. | Nrityabodha: towards understanding Indian classical dance using a deep learning approach | |
| CN104866824A (en) | Manual alphabet identification method based on Leap Motion | |
| Karthick et al. | Transforming Indian sign language into text using leap motion | |
| CN105917356A (en) | Contour-based classification of objects | |
| Jais et al. | A review on gesture recognition using Kinect | |
| CN105373810A (en) | Method and system for building action recognition model | |
| CN112699857A (en) | Living body verification method and device based on human face posture and electronic equipment | |
| Dan et al. | Survey on hand gesture recognition approaches | |
| Azad et al. | Real-time human-computer interaction based on face and hand gesture recognition | |
| CN107292295A (en) | Hand Gesture Segmentation method and device | |
| CN104484034B (en) | A kind of gesture motion primitive transition frames localization method based on gesture identification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C53 | Correction of patent for invention or patent application | ||
| CB02 | Change of applicant information |
Address after: 510000, No. two, 333, Guang Shan Road, Tianhe District, Guangdong, Guangzhou Applicant after: EDUCATION AND TRAINING ASSESSMENT CENTER, GUANGDONG POWER GRID CORP. Address before: 510000, No. two, 333, Guang Shan Road, Tianhe District, Guangdong, Guangzhou Applicant before: GPGC Training and Evaluation Center |
|
| COR | Change of bibliographic data |
Free format text: CORRECT: APPLICANT; FROM: GUANGDONG POWER GRID CORPORATION TRAINING AND EVALUATION CENTER TO: GUANGDONG POWER GRID LLC TRAINING AND EVALUATION CENTER |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |