CN110490173A - A kind of intelligent behaviour scoring system based on 3D body-sensing model - Google Patents
A kind of intelligent behaviour scoring system based on 3D body-sensing model Download PDFInfo
- Publication number
- CN110490173A CN110490173A CN201910798879.1A CN201910798879A CN110490173A CN 110490173 A CN110490173 A CN 110490173A CN 201910798879 A CN201910798879 A CN 201910798879A CN 110490173 A CN110490173 A CN 110490173A
- Authority
- CN
- China
- Prior art keywords
- value
- coordinate
- user
- abs
- necessary condition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/40—Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/23—Recognition of whole body movements, e.g. for sport training
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Multimedia (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
A kind of intelligent behaviour scoring system based on 3d body-sensing model, with the preparation method of 3D body-sensing coordinate data, the action data of the teacher of training and learning will be related to acting as master pattern, and scaling is obtained by the same attitude data of teacher and user's figure, marking is compared in the coordinate data of master pattern is forced and user is aligned coordinate data and user action, it is lower than the corresponding standard of timestamp and user's frame picture of threshold values to wherein frame score value, processing comparison is carried out, where allowing user intuitively to understand the difference of itself movement and standard.
Description
Technical field
The present invention relates to 3D body-sensing technology fields, and in particular to a kind of intelligent behaviour marking system based on 3d body-sensing model
System.
Background technique
3D body-sensing technology is that more the technical field in forward position and hot spot, a large amount of International Technology companies and marketing enterprises are equal now
Done a large amount of technical research investment in the field 3D, scientific and technological achievement also emerge rapidly in large numbersBamboo shoots after a spring rain like it is numerous and confused old existing, such as: body feeling interaction is set
Standby infrared imaging principle is unanimously used for the active infrared photography technology in night vision system；The excellent polarisation solid of 3D effect is aobvious
Show technology.Gesture identification is converted to the technology etc. of user instruction, and application is confined to game or instruction input mostly.Rare to have
In Yoga, dancing, wushu etc. be related to the application of the education sector of body shape and movement, therefore, by 3D body-sensing technology into
One step is expanded to be related to the technology of the auto-scoring that movement aspect learns and intelligent correction guidance and have ground-breaking application value.
Summary of the invention
In view of content illustrated by background technique, the technical program is based on 3D body-sensing, for being related to kinesics
The dancing of habit, Yoga, wushu, the technical solution of the automation marking of gymnastics training and intelligent correction guidance, i.e. intelligent behaviour are beaten
Subsystem, specific content are as follows:
The acquisition of skeleton joint coordinates obtains user images and video, according to human body by dedicated video-photographic equipment
The algorithm of bone identification obtains the 3D coordinate of target object bone key point, and in order to identify the movement of human body, the technical program is adopted
It is human joint points identification basis with KINECT skeletal joint point, by coordinate transform using SpineBase on vertebra as benchmark 0
Point, other 24 artis are respectively 1, SpineMid；2, Neck ；3 ,Head； 4, ShoulderLeft；5,
ElbowLeft ；6 , WristLef ；7,HandLeft； 8,ShoulderRight；9,ElbowRight；10,
WristRight； 11,HandRight； 12,HipLeft 13 ,KneeLeft 14,AnkleLeft ；15,FootLeft；
16 ,HipRight ；17, KneeRight；18, AnkleRight； 19, FootRight ； 20 ,
SpineShoulder；21,HandTipLeft；22,ThumbLeft； 23 ,HandTipRight；24,ThumbRight；Make
Body joint point coordinate and related data are obtained with the specified technical method of GetJointKinectPosition () and relevant interface.
Standard is attentioned the identification of stance: the coordinate of monitoring skeleton artis, and the relative position between node is set
Decision condition is attentioned the judgement result of stance by the verifying acquisition standard of the condition.
Necessary condition one: the x coordinate of vertebra basic point SpineBase, difference SpineMid, SpineShoulder,
The x coordinate in the joint Neck, Head, which compares, calculates difference, the z coordinate of same vertebra basic point SpineBase, difference SpineMid,
The z coordinate in the joint SpineShoulder, Neck, Head, which compares, calculates difference, sets a threshold values F, the difference it is absolute
Value is respectively less than F value, then the necessary condition is set up；
Necessary condition two: the x, y, z coordinate of vertebra basic point SpineBase, respectively with ShoulderLeft, ElbowLeft,
WristLef、HandLeft、ShoulderRight、ElbowRight、WristRight、HandRight、HipLeft、
KneeLeft、AnkleLeft、FootLeft、HipRight、 KneeRight、 AnkleRight、FootRight、
It is poor that the corresponding x, y, z coordinate of HandTipLeft, ThumbLeft, HandTipRight, ThumbRight artis compares calculation
Value, i.e. coordinate x difference, y difference, z difference, and the coordinate relative difference between joint is calculated, and be all differences setting one
The algorithmic approach that a corresponding threshold values and necessary condition determine calculates and determines whether necessary condition is true and determines whether mark
Standard is attentioned stance.
By the technical solution of the acquisition of skeleton joint coordinates, after operation and technical treatment, criterion is vertical
Just stance is further, is 0 point of benchmark in coordinate system by SpineBase on vertebra, i.e., the coordinate of SpineBase is 0,0,0
The three-dimensional coordinate in each joint of personage, the serial number of artis is represented with the following table number of coordinate value in the frame frequency at a time point
The corresponding artis, then obtain coordinate data、、、......；、、、......；、、、......；Preferably, the necessary condition for stance of attentioning is set are as follows:
Necessary condition 1:Abs() < 12；
Necessary condition 2:Abs() < 12；
Necessary condition 3:Abs() < 12；
Necessary condition 4:Abs() < 12；
Necessary condition 5:Abs() < 16；
Necessary condition 6:Abs() < 16；
Necessary condition 7:Abs() < 16；
Necessary condition 8:Abs() < 12；
Necessary condition 9:Abs()-Abs() < 8；
Necessary condition 10:Abs()-Abs() < 18；
Necessary condition 11:Abs()-Abs() < 18；
Necessary condition 12:<-53；
Necessary condition 13:<-53；
Necessary condition 14:Abs()-Abs() < 8；
Necessary condition 15:Abs()-Abs() < 8；
Necessary condition 16:Abs() < 16；
Necessary condition 17:Abs() < 16；
Necessary condition 18: - <68；
By when obtain joint coordinates numerical value and pass sequentially through the verifying of above-mentioned condition, if it is all genuine as a result, if set and reflected
The human body penetrated is that standard is attentioned stance.
Posture stops command response module, guides good user's sense to obtain body shape movement marking and error correction
By avoiding being mingled with when motion test keyboard and mouse order as necessary operating procedure, gesture motion and limb action
Interactive order be easy and marking acts content mix itself, interfere with each other, therefore, invention creates postures to stop
The technical method of command response is stayed, i.e., scheduled duration is stopped as the input of operating system using the monitoring given pose time and ordered
It enabling, posture order detection module is set, monitoring meets instruction body joint point coordinate, when the coordinate meets the position of command triggers,
It then monitors and is kept for the duration of the posture equal or exceed preset subcommand time threshold values, then trigger the corresponding order of the posture
Operation.
Posture stops a kind of embodiment of command response: system opens joint coordinates monitoring, obtains the seat of each artis in real time
Mark, wherein command module determines whether each coordinate meets the position of trigger command, eligible -10 <<10 and -10<<10；and -10<<10 and -10< <10；When by the coordinate of HandLeft, HandRight pass to posture life
Enable detection module.
Posture order detection module obtains in joint coordinates, it is preferable that Abs()-Abs(When) < 18；According to's
The corresponding residence parameter progress add-one operation of value section setting: -10 <Parameter A is corresponded to when < 10;Abs(-When) < 10 pair
Answer parameter B；Abs(-Parameter C is corresponded to when) < 10；Abs(-Parameter D is corresponded to when) < 10；And so on；Others are stopped
Staying parameter assignment is 0；The corresponding residence parameter of the data being connected on posture order detection module is verified simultaneously, and if this
Secondary corresponding residence parameter is identical, different then this corresponding residence parameter amendment is assigned a value of 1；Residence time threshold values is set,
Program event process corresponding to each residence parameter is set, when residence parameter is greater than or equal to the threshold values, triggering is corresponded to
Command program event, realize system response user command operation.
Complete the basis of the above technical method, systems inspection to standard is attentioned stance, start a master pattern record or
One marking test process, specific technical solution are as follows:
Mode standard, marking mode are set in systems first, and creation project records the module of the background audio information of project.
Master pattern is created, according to the audio rhythm for mp4, rm format recorded in project, duration t0, including but not limited to
Dancing, gymnastics, wushu, Yoga teacher, coach, the standard stance attentioned, system are attentioned purple identification of standing by above-mentioned standard
Method monitoring, monitor validation criteria attention position after, starting open posture stop command response module, system acquisition starts to order
When enabling, prompt action demonstration starts, and the teacher, coach start to demonstrate standard operation one time, simultaneity factor real-time monitoring institute
The coordinate in the joint that is captured in different time points of teacher, coach is stated, and internal system is recorded, establishes the master die of the project
Type.
The pressure of master pattern is aligned, it is contemplated that demonstrator is fat or thin in user and master pattern, difference of height figure, will
The data of movement world coordinates and the data of student's world coordinates of teacher directly compare in master pattern, miss by a mile, do not have
The value of reference, therefore, the present invention calculate separately world coordinates according to demonstrator in master pattern and the same standard gestures of user
In system, the scaling correction factor of x, y, z, and by the data of x, y, z all in master pattern, all by scaling correction factor
Be modified, the master pattern being aligned, the dynamic coordinate data under user's posture, then with the actual movement coordinate of user
Comparing obtains score value, and particular technique method is as follows:
It is attentioned the technical method of the judgement of stance by above-mentioned standard, attentions and stood under appearance in standard, by recycle ratio to respectively
In 24 joint coordinates for obtaining user and master pattern, the maximum x value of user, maximum y value, maximum z value, minimum x
Value, minimum y value, minimum z value, maximum x value in corresponding master pattern record, maximum y value, maximum z value
, minimum x value, minimum y value, minimum z value。
Wherein a, b, c points are that master pattern is forced to snap to the scaling alignment system of the world coordinates x, y, z of user's figure
Number, by master pattern record in x respectively multiplied by a, y respectively multiplied by b, z respectively multiplied by c, obtain 0 toAll timestamps pair
The data group answered、 、、......；、、、......；、、、......；Do not support subscript variable formula program language implement when, set and using value as
0 to t*24 integer transition variable as subscript variable parameter.
User by standard attention stance system confirmation after and system monitoring to posture stop respond module start to order
When enabling, start corresponding motion test, system obtains the user's joint coordinates for crying out timestamp in real time, and is recorded in system, knot
The duration t1 of the test is recorded when beam, when t1 is greater than t0, " your rhythm of action is slower than standard in system interface display reminding
When t1-t0 ", t1 are less than or equal to t0, at system interface display reminding " your rhythm of action t0-t1 faster than standard ".Simultaneously will
The timestamp of data carries out pressure alignment in user coordinates data group, i.e.,
By the amendment of the timestamp where coordinate value, force compression equal with t0 the duration t1 of user's test action.To become
Amount t come represent 0 arrive t0 timestamp, then t time point it is corresponding force alignment after master pattern and time scale modification after
User's test video in frame frequency image in, the coordinate data of personage's node tests view to distinguish master pattern and user
The difference of frequency coordinate, for the x, y, z of the coordinate under user's test respectively with 1 combination, i.e., x1, y1, z1 represent coordinate parameters, will mark
Quasi- model coordinate still represents corresponding coordinate parameters with x, y, z, then the score value for being in the mean difference of both t time points coordinate
Are as follows:
One system-frame score value qualification determination threshold values H is set, it willIn addition record lower than H is stored in an array R, wait
System is called；The corresponding inequality of all t is carried out accumulative be superimposed and average, so that it may obtain execution user and standard it
Between equal difference value, calculation formula is as follows:
It, will by above-mentioned algorithmResult be shown in user interface as one of appraisal result, and set grade: poor, close
Lattice, good, outstanding threshold values, and result score value is determined which threshold values section to show corresponding grade in.
In above-mentioned statistics scoring algorithm, although avoiding the positive and negative of different coordinate differences from liquidating using absolute value, it is possible to make
Obtaining sum of the deviations is zero, cannot really reflect coordinate sample error, but the algorithm of accumulated absolute values cannot measure the discrete of numerical value
Degree, more stable deviation by a small margin is easy to be close with operation result the case where deviation big rise and fall, therefore the technical program
Further user is allowed to understand the offset that oneself the result of the action is compared with standard come supplementary peg count score value using the algorithm based on variance
The corresponding master pattern and the revised user's test video of duration forced after alignment of degree, first acquisition t time point
In frame frequency image in, the coordinate data of personage's node, obtaining t time point mapped according to these coordinate datas, to be based on variance former
The score value calculated is managed, wherein to force corresponding coordinate value in the master pattern after alignment to be set as being averaged in variance calculating
Value, the score value formula at t time point are as follows:
One system-frame score value qualification determination threshold values G is set, it willIn addition record lower than G is stored in an array S, wait
System is called:
WhereinValue is bigger, and the offset fluctuation for indicating user action and standard is very big, and an absolute number is difficult to user
Result based on variance operation is converted to hundred-mark system in order to which the cognition further with user integrates with by cognition of the image to error
Marking be easier to be easily accepted by a user and understand, so setting one threshold values K,Value is greater than K value, direct 0 point, on the contrary to be less than K
When value, the score value of hundred-mark system is converted to by following algorithm:
Wherein F is the error deviation score value of hundred-mark system, the result of F value is pushed to user interface, as average difference algorithm score value
A secondary outcome, give the reference of measurement fluctuation size score value one of deviation of movement one.
It is intuitive that user is guided to correct mistake movement, this technology side in order to further let the user know that the biggish movement of mistake
Case further by figure action in the lower frame of score value extract and the frame of master pattern correspondent time in figure action ratio
It is right, allow user intuitively understanding movement defect where, facilitate user's improvement act, efficiently study to movement main points, specific implementation
It is as follows:
In step 1, array R, S recorded in above-mentioned marking technical method, by loop computation, the parameter of R, S intersection is obtained
Member, and these members are stored in new array Q.
Step 2, action error picture extract, and sequential read out member's timestamp numerical value in Q array, and by the timestamp
Frame is corresponded in corresponding master pattern and user's test video and goes static background process, and static judgement is arranged in video pre-filtering
Software is obtained the unit conversion pixel of video frame picture by threshold values U first.
Specific implementation: the unit that software obtains is T orange red, and the value of T/15 is pixel p ixel value.
Further by the frame data of acquisition map pixel RGB numerical value extract, and with member's time in current Q
The RGB for stabbing the former frame position pixel is compared, and the R/G/B in RGB is subtracted each other respectively, and by the sum of results added divided by 3
As a result, and static state decision threshold K be compared, less than threshold values then by the rgb value of the pixel be assigned a value of three fixed value u1,
U2, u3, obtaining fixed value described in the best visual effect through overtesting is preferably 166,166,166, filters out and compares with former frame
There is no moving the part with image change, including the typical skeleton point not changed.
Preferably, the value of static decision threshold U is 11.
Further, optimize decision threshold algorithm, using specified span siding-to-siding block length, determine, and the operation that will determine result
Prolong the range of the raw front and back span value half to before changing.A kind of embodiment: frame object,
From x be 0 to.Width value, gap periods length are k, carry out following operation:
From y be 0 to.Height value, gap periods length are k, carry out following operation:
Set point of the coordinate as current x, y value in present frame picture, corresponding RGB numerical value is respectively.R、.G、.B, the previous timestamp of the corresponding present frame of x, y value corresponds to the point in frame picture, corresponding RGB numerical value is respectively.R、.G、.B, V value is calculated according to following formula；
V is less than 0The RGB of the position (X, Y) is set as 166,166, and will be with x difference or with y difference in positive and negative k/2 value area
Between location point RGB be set as 166,166,166.
Further progress y is incremented by 1 circulation, then carries out the circulation that x is incremented by 1, the processing of the frame picture.
Algorithm above, arithmetic speed improves speed several times than case-by-point algorithm, and when using unit orange red, 15 orange reds are equal to 1 pixel, this
It means that if with specifying a pixel to need to repeat 15*15=225 time as unit of orange red on the screen！Because 0 orange red to 14 orange reds its
Real is all 1 pixel, so (x, y) is that coordinate is signified when x is assigned a value of 0 to 14, y is assigned a value of 0 to 14, when z is assigned a value of 0 to 14
Point be all the same pixel (0,0) on screen in fact；It therefore is step-length in 0 orange red, 15 orange reds, 30 orange reds, 45 orange reds ... with 15
It successively draws a little, so that it may avoid multiple duplicate operation, but with span value k, further under the premise of fidelity operation result,
It is preferred that increasing span, it is compared with the sampling method of equiblibrium mass distribution,；Computing load is effectively reduced, the speed of calculation process is improved
Degree, and unnecessary static background is shielded, further increase operation efficiency.
By the processing of the above technical method, obtain member's timestamp numerical value in Q array respectively correspond master pattern and
Frame picture is corresponded in user video treated picture, is recorded in system.One movement score value test image analysis circle is set
A progress bar is arranged using 0 to t0 value as section in face；And it is member's timestamp numerical value in Q array is enterprising in the progress bar
The corresponding mark of row, the mark is corresponding with the picture that action error compares, and the photo that corresponding master pattern is handled with
The photo handled in user video separately compares placement, and user clicks mark, and interface different location shows two processing
Playing function button is arranged in picture afterwards, then shows corresponding two picture of mark at set time intervals.
Detailed description of the invention
Fig. 1 is a kind of overall logic frame diagram of intelligent behaviour scoring system based on 3D body-sensing model.
Fig. 2 is the schematic diagram of 3D body-sensing skeleton joint coordinates point.
Especially statement: " embodiment " etc. described in the present specification refers to the specific spy for combining embodiment description
Sign, element or feature include in the embodiment of the application generality description.There is table of the same race in multiple places in the description
It states and non-limiting refer in particular to is the same embodiment.That is, in conjunction with any embodiment describe a specific features, element or
When person's feature, what is advocated is to realize that this feature, element or feature are contained in the present invention in conjunction with other embodiments
In the scope of the claims for applying for protection；Embodiment is multiple explanatory embodiments referring to logical architecture of the present invention and thinking
Invention has been described, but scope of protection of the present invention is not limited thereto, and those skilled in the art are in the technology of the present invention
Can be designed that a lot of other modification and implementations under solution framework, can to technical solution want point transformation combination/or
Layout carries out a variety of non-intrinsically safe variations and modifications, and to those skilled in the art, other purposes also will be apparent,
The unsubstantiality change or replacement of implementation can be readily occurred in, these modifications and implementations will fall in principle model disclosed in the present application
Within enclosing and being spiritual.
Claims (5)
1. a kind of the step of intelligent behaviour scoring system based on 3D body-sensing model, feature includes and to be known as: by dedicated
Video-photographic equipment, obtain user images and video, it is crucial to obtain target object bone according to the algorithm of skeleton identification
The 3D coordinate of point；
Standard is attentioned the identification of stance: the coordinate of monitoring skeleton artis, and the relative position between node is set and is determined
Condition is attentioned the judgement result of stance by the verifying acquisition standard of the condition；
Necessary condition one: the x coordinate of vertebra basic point SpineBase, respectively SpineMid, SpineShoulder, Neck,
The x coordinate in the joint Head, which compares, calculates difference, the z coordinate of same vertebra basic point SpineBase, difference SpineMid,
The z coordinate in the joint SpineShoulder, Neck, Head, which compares, calculates difference, sets a threshold values F, the difference it is absolute
Value is respectively less than F value, then the necessary condition is set up；
Necessary condition two: the x, y, z coordinate of vertebra basic point SpineBase, respectively with ShoulderLeft, ElbowLeft,
WristLef、HandLeft、ShoulderRight、ElbowRight、WristRight、HandRight、HipLeft、
KneeLeft、AnkleLeft、FootLeft、HipRight、 KneeRight、 AnkleRight、FootRight、
It is poor that the corresponding x, y, z coordinate of HandTipLeft, ThumbLeft, HandTipRight, ThumbRight artis compares calculation
Value, i.e. coordinate x difference, y difference, z difference, and the coordinate relative difference between joint is calculated, and be all differences setting one
The algorithmic approach that a corresponding threshold values and necessary condition determine calculates and determines whether necessary condition is true and determines whether mark
Standard is attentioned stance；
Posture stops command response module, stops scheduled duration as the input of operating system using the monitoring given pose time and orders
It enabling, posture order detection module is set, monitoring meets instruction body joint point coordinate, when the coordinate meets the position of command triggers,
It then monitors and is kept for the duration of the posture equal or exceed preset subcommand time threshold values, then trigger the corresponding order of the posture
Operation；
Systems inspection is attentioned stance to standard, starts a master pattern record or a marking test process；
Mode standard, marking mode are set in systems first, and creation project records the module of the background audio information of project；
Creation master pattern is including but not limited to waved according to the audio rhythm for mp4, rm format recorded in project, duration t0
It steps, gymnastics, wushu, Yoga teacher, coach, the standard stance attentioned, system is attentioned by above-mentioned standard stands what purple identified
Method monitoring, after monitoring validation criteria attention position, starting opens posture and stops command response module, and system obtains initiation command
When, prompt action demonstration starts, and the teacher, coach start to demonstrate standard operation one time, described in simultaneity factor real-time monitoring
The coordinate in the joint that is captured in different time points of teacher, coach, and internal system is recorded, establish the master die of the project
Type；
The pressure of master pattern is aligned, and according to demonstrator in master pattern and the same standard gestures of user, calculates separately world's seat
In mark system, the scaling correction factor of x, y, z, and by the data of x, y, z all in master pattern, all pass through scaling amendment system
Number is modified, the master pattern being aligned, the dynamic coordinate data under user's posture, then is sat with the actual movement of user
Comparing is marked, score value is obtained, particular technique method is as follows:
It is attentioned the technical method of the judgement of stance by above-mentioned standard, attentions and stood under appearance in standard, by recycle ratio to respectively
In 24 joint coordinates for obtaining user and master pattern, the maximum x value of user, maximum y value, maximum z value, minimum x
Value, minimum y value, minimum z value, maximum x value in corresponding master pattern record, maximum y value, maximum z value、
Minimum x value, minimum y value, minimum z value；
Wherein a, b, c points are that master pattern is forced to snap to the scaling alignment coefficient of the world coordinates x, y, z of user's figure, will
X in master pattern record respectively multiplied by a, y respectively multiplied by b, z respectively multiplied by c, obtain 0 toThe corresponding number of all timestamps
According to group；
Further, by figure action in the lower frame of score value extract and the frame of master pattern correspondent time in personage it is dynamic
Compare, allow user intuitively understanding movement defect where.
2. a kind of the step of intelligent behaviour scoring system based on 3D body-sensing model as described in claim 1, feature includes and
Be known as: criterion is attentioned the further of stance, is 0 point of benchmark in coordinate system by SpineBase on vertebra, i.e.,
The coordinate of SpineBase be 0,0,0 in the frame frequency at a time point each joint of personage three-dimensional coordinate, under coordinate value
The serial number that table number represents artis corresponds to the artis, then obtains coordinate data、、、......；、、、......；、、、......；Preferably, the necessary condition for stance of attentioning is set are as follows:
Necessary condition 1:Abs() < 12；
Necessary condition 2:Abs() < 12；
Necessary condition 3:Abs() < 12；
Necessary condition 4:Abs() < 12；
Necessary condition 5:Abs() < 16；
Necessary condition 6:Abs() < 16；
Necessary condition 7:Abs() < 16；
Necessary condition 8:Abs() < 12；
Necessary condition 9:Abs()-Abs() < 8；
Necessary condition 10:Abs()-Abs() < 18；
Necessary condition 11:Abs()-Abs() < 18；
Necessary condition 12:<-53；
Necessary condition 13:<-53；
Necessary condition 14:Abs()-Abs() < 8；
Necessary condition 15:Abs()-Abs() < 8；
Necessary condition 16:Abs() < 16；
Necessary condition 17:Abs() < 16；
Necessary condition 18: -<68；
By when obtain joint coordinates numerical value and pass sequentially through the verifying of above-mentioned condition, if it is all genuine as a result, if set and reflected
The human body penetrated is set as standard and attentions stance.
3. a kind of the step of intelligent behaviour scoring system based on 3D body-sensing model as described in claim 1, feature includes and
Be known as: posture stops command response: system opens joint coordinates monitoring, the coordinate of each artis is obtained in real time, wherein ordering
Module determines whether each coordinate meets the position of trigger command, eligible -10 <<10 and -10< <10；and -
10<<10 and -10< <10；When the coordinate of HandLeft, HandRight passed into posture order detection module；
Posture order detection module obtains in joint coordinates, it is preferable that Abs()-Abs(When) < 18；According toValue section set
It sets corresponding residence parameter and carries out add-one operation: excellent -10 <Parameter A is corresponded to when < 10;Abs(-Parameter is corresponded to when) < 10
B；Abs(-Parameter C is corresponded to when) < 10；Abs(-Parameter D is corresponded to when) < 10；And so on；Other residence parameters
It is assigned a value of 0；The corresponding residence parameter of the data being connected on posture order detection module is verified simultaneously, and if this correspondence
Residence parameter it is identical, it is different then this corresponding residence parameter amendment is assigned a value of 1；Residence time threshold values is set, setting is every
Program event process corresponding to one residence parameter triggers corresponding order when residence parameter is greater than or equal to the threshold values
Program event realizes the operation of system response user command.
4. a kind of the step of method for beating of intelligent behaviour that alignment is forced based on master pattern, feature includes with to be known as:
Based on target standard in claim 1 and claim 2 attention stance identification method, user attention stance by system it is true
After recognizing, respond module method is stopped by the posture in claim 1 and claim 3, when monitoring initiation command, starts phase
The motion test answered, system obtains the user's joint coordinates for crying out timestamp in real time, and is recorded in system, at the end of record the survey
When the duration t1 of examination, t1 are greater than t0, at system interface display reminding " your rhythm of action t1-t0 slower than standard ", t1 is less than
When equal to t0, at system interface display reminding " your rhythm of action t0-t1 faster than standard "；Simultaneously by user coordinates data group
The timestamp of middle data carries out pressure alignment:
By the amendment of timestamp corresponding to coordinate value, force compression equal with t0 the duration t1 of user's test action；With
Variable t come represent 0 arrive t0 timestamp, then t time point it is corresponding force alignment after master pattern and time scale modification
In the frame frequency image in user's test video afterwards, the coordinate data of personage's node, in order to distinguish master pattern and user's test
The difference of video coordinates, the x, y, z of the coordinate under user's test is respectively with 1 combination, i.e., x1, y1, z1 represent coordinate parameters, will
Master pattern coordinate still represents corresponding coordinate parameters with x, y, z, then the score value for being in the mean difference of both t time points coordinate
Are as follows:
One system-frame score value qualification determination threshold values H is set, it willIn addition record lower than H is stored in an array R, institute
There is the corresponding inequality of t to carry out accumulative superposition and is averaged, so that it may the equal difference value between the user of execution and standard is obtained,
Calculation formula is as follows:
It, will by above-mentioned algorithmResult be shown in user interface as one of appraisal result, and set grade: poor, close
Lattice, good, outstanding threshold values, and result score value is determined which threshold values section to show corresponding grade in；
Obtain the t time point corresponding frame forced in master pattern and the revised user's test video of duration after being aligned
In frequency image, the coordinate data of personage's node is obtained t time point mapped according to these coordinate datas and is calculated based on variance principle
Score value, wherein with force alignment after master pattern in corresponding coordinate value be set as variance calculate in average value, the t time
The score value formula of point is as follows:
One system-frame score value qualification determination threshold values G is set, it willIn addition record lower than G is stored in an array S；
WhereinValue is bigger, and the offset fluctuation for indicating user action and standard is very big, and an absolute number is difficult to give user's shape
As the cognition to error, in order to which the cognition further with user integrates with, the result based on variance operation is converted to hundred-mark system
Marking is easier to be easily accepted by a user and understand, so one threshold values K of setting,Value is greater than K value, direct 0 point, on the contrary to be less than K value
When, the score value of hundred-mark system is converted to by following algorithm:
Wherein F is the error deviation score value of hundred-mark system, the result of F value is pushed to user interface, as average difference algorithm score value
A secondary outcome, give the reference of measurement fluctuation size score value one of deviation of movement one.
5. a kind of method for forcing the intelligent behaviour of alignment to be beaten based on master pattern as claimed in claim 4, what feature included
Step and to be known as: a kind of method specific implementation that action error picture compares is as follows:
In step 1, array R, S recorded in claim 4 marking technical method, by loop computation, R, S intersection are obtained
Parameter member, and these members are stored in new array Q;
Step 2, action error picture extract, and sequential read out member's timestamp numerical value in Q array, and the timestamp is corresponding
Master pattern and user's test video in correspond to frame and go static background process, static decision threshold is arranged in video pre-filtering
Software is obtained the unit conversion pixel of video frame picture by U first；
Further by the frame data of acquisition map pixel RGB numerical value extract, and with before member's timestamp in current Q
The RGB of the one frame position pixel is compared, and the R/G/B in RGB is subtracted each other respectively, and by the sum of results added divided by 3 as a result,
Be compared with static decision threshold K, less than threshold values then by the rgb value of the pixel be assigned a value of three fixed value u1, u2,
U3, obtaining fixed value described in the best visual effect through overtesting is preferably 166,166,166, and filtering out to compare with former frame does not have
The part that movement and image change occurs, including the typical skeleton point not changed；
Preferably, the value of static decision threshold U is 11；
Further, optimize decision threshold algorithm, using specified span siding-to-siding block length, determine, and life is prolonged into the operation for determining result
The range of front and back span value half, frame object to before changing,
From x be 0 to.Width value, gap periods length are k, carry out following operation；
From y be 0 to.Height value, gap periods length are k, carry out following operation；
Set point of the coordinate as current x, y value in present frame picture, corresponding RGB numerical value is respectively.R、.G、
.B, the previous timestamp of the corresponding present frame of x, y value corresponds to the point in frame picture, corresponding RGB numerical value is respectively.R、.G、.B, V value is calculated according to following formula；
V is less than 0The RGB of the position (X, Y) is set as 166,166, and will be with x difference or with y difference in positive and negative k/2 value area
Between location point RGB be set as 166,166,166；
Further progress y is incremented by 1 circulation, then carries out the circulation that x is incremented by 1, the processing of the frame picture；
By the way that with the processing of the technical method, the member's timestamp numerical value obtained in Q array respectively corresponds master pattern and use
Frame picture is corresponded in the video of family treated picture, is recorded in system, a movement score value test image assay surface is set,
Using 0 to t0 value as section, one progress bar is set；And member's timestamp numerical value in Q array is carried out pair on the progress bar
The mark answered, the mark is corresponding with the picture that action error compares, and the photo that corresponding master pattern is handled and user
The photo handled in video separately compares placement, and user clicks mark, and interface different location shows described two, and treated
Playing function button is arranged in picture, then shows corresponding two picture of mark at set time intervals.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910798879.1A CN110490173B (en) | 2019-08-28 | 2019-08-28 | Intelligent action scoring system based on 3D somatosensory model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910798879.1A CN110490173B (en) | 2019-08-28 | 2019-08-28 | Intelligent action scoring system based on 3D somatosensory model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110490173A true CN110490173A (en) | 2019-11-22 |
| CN110490173B CN110490173B (en) | 2022-11-18 |
Family
ID=68554740
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910798879.1A Active CN110490173B (en) | 2019-08-28 | 2019-08-28 | Intelligent action scoring system based on 3D somatosensory model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110490173B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111275031A (en) * | 2020-05-07 | 2020-06-12 | 西南交通大学 | Flat plate support detection method, device, equipment and medium based on human body key points |
| CN112309181A (en) * | 2020-10-23 | 2021-02-02 | 百色学院 | Dance teaching auxiliary method and device |
| AT525333A1 (en) * | 2021-07-26 | 2023-02-15 | Istec Innovative Sport Tech Gmbh | System for automated scoring in martial arts |
| CN117649061A (en) * | 2024-01-30 | 2024-03-05 | 山东达斯特信息技术有限公司 | Multi-node networking electricity analysis method and system for environmental protection monitoring |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011009302A1 (en) * | 2009-07-22 | 2011-01-27 | 深圳泰山在线科技有限公司 | Method for identifying actions of human body based on multiple trace points |
| CN104598867A (en) * | 2013-10-30 | 2015-05-06 | 中国艺术科技研究所 | Automatic evaluation method of human body action and dance scoring system |
| US20170266491A1 (en) * | 2016-03-21 | 2017-09-21 | Ying Chieh Mitchell | Method and system for authoring animated human movement examples with scored movements |
| CN107293175A (en) * | 2017-08-04 | 2017-10-24 | 华中科技大学 | A kind of locomotive hand signal operation training method based on body-sensing technology |
| CN107349594A (en) * | 2017-08-31 | 2017-11-17 | 华中师范大学 | A kind of action evaluation method of virtual Dance System |
-
2019
- 2019-08-28 CN CN201910798879.1A patent/CN110490173B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011009302A1 (en) * | 2009-07-22 | 2011-01-27 | 深圳泰山在线科技有限公司 | Method for identifying actions of human body based on multiple trace points |
| CN104598867A (en) * | 2013-10-30 | 2015-05-06 | 中国艺术科技研究所 | Automatic evaluation method of human body action and dance scoring system |
| US20170266491A1 (en) * | 2016-03-21 | 2017-09-21 | Ying Chieh Mitchell | Method and system for authoring animated human movement examples with scored movements |
| CN107293175A (en) * | 2017-08-04 | 2017-10-24 | 华中科技大学 | A kind of locomotive hand signal operation training method based on body-sensing technology |
| CN107349594A (en) * | 2017-08-31 | 2017-11-17 | 华中师范大学 | A kind of action evaluation method of virtual Dance System |
Non-Patent Citations (2)
| Title |
|---|
| 梅阳等: "一种基于关键帧的人体行为识别方法", 《光学技术》 * |
| 詹彬等: "基于视觉传感的智能运动训练系统的研发", 《物联网技术》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111275031A (en) * | 2020-05-07 | 2020-06-12 | 西南交通大学 | Flat plate support detection method, device, equipment and medium based on human body key points |
| CN112309181A (en) * | 2020-10-23 | 2021-02-02 | 百色学院 | Dance teaching auxiliary method and device |
| AT525333A1 (en) * | 2021-07-26 | 2023-02-15 | Istec Innovative Sport Tech Gmbh | System for automated scoring in martial arts |
| AT525333B1 (en) * | 2021-07-26 | 2023-05-15 | Istec Innovative Sport Tech Gmbh | System for automated scoring in martial arts |
| CN117649061A (en) * | 2024-01-30 | 2024-03-05 | 山东达斯特信息技术有限公司 | Multi-node networking electricity analysis method and system for environmental protection monitoring |
| CN117649061B (en) * | 2024-01-30 | 2024-04-26 | 山东达斯特信息技术有限公司 | Multi-node networking electricity analysis method and system for environmental protection monitoring |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110490173B (en) | 2022-11-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110490173A (en) | A kind of intelligent behaviour scoring system based on 3D body-sensing model | |
| CN113706960B (en) | Nursing operation exercise platform based on VR technology and use method | |
| Anzalone et al. | Evaluating the engagement with social robots | |
| CN109298779B (en) | Virtual training system and method based on virtual agent interaction | |
| US9520072B2 (en) | Systems and methods for projecting images onto an object | |
| US20150079565A1 (en) | Automated intelligent mentoring system (aims) | |
| CN110349667A (en) | The autism assessment system analyzed in conjunction with questionnaire and multi-modal normal form behavioral data | |
| Lakshmiprabha et al. | [Poster] An augmented and virtual reality system for training autistic children | |
| CN107402633B (en) | A kind of safety education method based on image simulation technology | |
| Emerson et al. | Early prediction of visitor engagement in science museums with multimodal learning analytics | |
| CN111539245A (en) | CPR (CPR) technology training evaluation method based on virtual environment | |
| CN116229793A (en) | Training and checking system based on virtual reality technology | |
| CN115933930A (en) | Method, terminal and device for analyzing attention of learning object in education meta universe | |
| Khosravi et al. | Self-directed learning using eye-tracking: a comparison between wearable head-worn and webcam-based technologies | |
| Pande et al. | Eye-tracking in STEM education research: limitations, experiences and possible extensions | |
| Xipeng et al. | Research on badminton teaching technology based on human pose estimation algorithm | |
| Gupta et al. | An evolving multi-agent scenario generation framework for simulations in preventive medicine education | |
| CN110322098A (en) | S.O.P. feedback during interactive computer simulation | |
| CN115271621A (en) | Campus physical education comprehensive management platform | |
| CN114282837A (en) | Physics chemical experiment teaching system | |
| CN113673856A (en) | 3D-based hospital sensory control simulation training and examination system | |
| CN113611416A (en) | Psychological scene assessment method and system based on virtual reality technology | |
| Bertrand et al. | Visual Attention to Wayfinding Aids in Virtual Environments. | |
| Dziak et al. | Wireless Monitoring System for Fireman’s Competence Objective Assessment | |
| Jeelani et al. | Automating Analysis of Construction Workers Viewing Patterns for Personalized Safety Training and Management |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |