CN104524742A - Cerebral palsy child rehabilitation training method based on Kinect sensor - Google Patents
Cerebral palsy child rehabilitation training method based on Kinect sensor Download PDFInfo
- Publication number
- CN104524742A CN104524742A CN201510001948.3A CN201510001948A CN104524742A CN 104524742 A CN104524742 A CN 104524742A CN 201510001948 A CN201510001948 A CN 201510001948A CN 104524742 A CN104524742 A CN 104524742A
- Authority
- CN
- China
- Prior art keywords
- angle
- line
- straight line
- face
- children
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Rehabilitation Tools (AREA)
Abstract
The invention discloses a cerebral palsy child rehabilitation training method based on a Kinect sensor. The method comprises the following steps: S1, acquiring skeleton point data of a child; S2, carrying out limb movement training and judging the tilting angle and uplifting angle of skeleton points of the child, wherein after the child conducts a movement, the movement of the child is captured, and the tilting angle and uplifting angle of the joint points of the head, the upper limbs and the lower limbs are judged; S3, carrying out robust interactive processing on the movement of the child; S4, connecting a game engine and sending child skeleton data obtained after robust interactive processing in the step S3 to the game engine; S5, feeding back the movement of the child in a voice mode to remind the child of non-standard movements and encourage the child to conduct standard movements; S6, estimating the progress of rehabilitation training of the child. According to the method, the movement behavior characteristics of the child in training are acquired based on microsoft Kinect, so that the cardiovascular endurance, muscular endurance, muscular force, balance and flexibility of the child are comprehensively developed.
Description
Technical field
The present invention relates to a kind of children with cerebral palsy rehabilitation training system and technology, particularly relate to a kind of children with cerebral palsy recovery training method based on Kinect sensor.
Background technology
Cerebral palsy is a kind of Childhood modal nervous system disability illness, existing 6,000,000 patients with cerebral palsy of China, wherein 0-6 year children with cerebral palsy about 1,200,000, and the children with cerebral palsy of 70% lives in poverty-stricken area, and it is painful that this makes their family be subject to great body and mind.Carrying out motion function rehabilitation training in the child development phase is that children with cerebral palsy recovers proper motion function, improves able-bodied important rehabilitation measure.But because the rehabilitation of cerebral palsy is started late in China, and brain paralysis countermeasure system is still not well established, the rehabilitation institution of brain paralysis is also relatively less, and community, family care, on the spot conveniently, economical and effective, be beneficial to and adhere to, so community rehabilitation is the rehabilitation approach geared to actual circumstances most.
Active movement is rehabilitation maneuver important in athletic rehabilitation, but children with cerebral palsy lacks the enthusiasm of adhering to training for active movement dry as dust, simultaneously children be in training process also cannot for expert opinion to carry out scientific guidance timely.
Prior art is only by human-computer interaction technology, rehabilitation training is carried out by Kinect sensor, as patent CN103230664A provides a kind of upper extremity exercise rehabilitation training system based on Kinect sensor and training method, simple joint training and the shoulder of upper limbs can be carried out, elbow, the combined training of wrist, but the rehabilitation training of upper limbs is only provided, and training effect is imperfect, such as make a stretch of the arm, utilize existing line face angle technology cannot judge whether arm stretches, simultaneously, prior art can not train lower limb, and interest is not strong, the rehabilitation training interest of children can not be improved, can not by training progress feedback.
Summary of the invention
For problems of the prior art, the invention provides a kind of children with cerebral palsy recovery training method based on Kinect sensor, lay particular emphasis on the training for children's torso harmony and leg training, the present invention increases voice feedback technology, by voice feedback, children are carried out to the encouragement of action prompting lack of standardization and execution, be connected with game engine, increase training interesting.
Its technical scheme of the present invention is:
Based on a children with cerebral palsy recovery training method for Kinect sensor, comprise the following steps:
The skeleton point data of S1, acquisition children: Kinect sensor is connected with computer, computer accesses network, Kinect sensor obtains child image, the skeleton point data that function obtains children are caught by the bone of Kinect, the coordinate value that skeleton point data are is initial point with Kinect depth camera, skeleton point comprises head node H, shoulder joint Centroid S
0, backbone Centroid S, left shoulder joint node S
1, right shoulder joint node S
2, left elbow joint node E
1, right elbow joint node E
2, left wrist joint node W
1, right wrist joint node W
2, left hip joint node H
1, right hip joint node H
2, left knee joint node K
1, right knee joint node K
2, left ankle-joint node A
1, right ankle-joint node A
2;
S2, limb motion is trained, the skeleton point of children is tilted, lifts angle judgement: after children show action, catch child motor, in three-dimensional planar rectangular coordinate system, getting YOZ plane is α face, and getting XOY plane is β face, tilts, lifts angle judgement to each artis of head, upper limbs and lower limb;
S3, the sane interaction process of child motor: processed by wavelet de-noising, the bone shake of moving average filter noise reduction to motion, by medium filtering, image cavity is processed, rejecting abnormalities depth value, degree of depth missing values is estimated; Adopt magnetite method that autoregistration is carried out in the action in threshold range and virtual image; Virtual image is be mapped to by skeleton point data syn-chronization on the virtual portrait on computer screen, the posture behavior of real-time tracking children.Step S3 is for the particularity of CP children, and likely shake appears in action instability, uses the bone shake to motion of a series of prediction and noise reduction smoothing technique to process, and realizes the alternately sane of children with cerebral palsy and computer.
S4, connects game engine, the children's skeleton data after sane for step S3 interaction process is sent to game engine, and the action that skeleton data is formed is converted to the event of game engine definition, game is included in children's rehabilitation training;
Step S4 converts the action of children the event that game engine defines to, in gaming for these events add the access that the method processed just can realize human body tracing equipment, this is relevant with concrete game design, the game such as trundled, if children imitate the action hand that trundles and brandish from back to front, virtual portrait in computer scene also will complete the action that trundles, the motion of ball is that the event that predefined is good (comprises the original position of ball, the movement velocity of ball, the direction of motion of ball, the whole motion process of stop position of ball), after the gesture that system identification is brandished from back to front to children, understand the event that this predefined of motion of trigger ball immediately is good.As, if children draw a circle aloft, game just suspends.Draw the action that circle is children, after system identification to picture circle gesture, trigger this event of suspension of game play, for suspension of game play adds processing method in games, the core of this part is in game engine, write Kinect plug-in unit, complete the access of Kinect, the gesture of such children could by system identification.
S5, voice feedback child motor, by the result of autoregistration described in step S3 by voice feedback to children, carry out the encouragement of action prompting lack of standardization and execution;
S6, the rehabilitation training progress of assessment children: the skeleton point of real time record children and the registration of virtual image, and described registration is sent to remote evaluation terminal, assessment rehabilitation training performance.
More preferably, step S2 also comprises ambulation training, calculates left right foot striding step, speed index, differentiates whether the gait of children is normal gait.
More preferably, the skeleton point of step S2 to children tilts, lift angle judges, specifically comprises the following steps:
(1) judge whether body tilts: with α face for the plane of reference, shoulder joint Centroid S
0, backbone Centroid S 2 connect be in line be a, be the lateral tilting oblique angle of body based on line face angle calcu-lation method calculated line a and α face angle=<a, α >, described straight line a and α face angle; Under normal circumstances, (namely straight line is a) 0 with the angle in α face to trunk.When body tilts, straight line a and α face angle=<a, α > is not 0;
(2) judge whether body tilts forward and back: with β face for the plane of reference, shoulder joint Centroid S
0, backbone Centroid S, 2 connect be in lines be a, be the tilt fore and aft of body based on line face angle calcu-lation method calculated line a and β face angle=<a, β >, described straight line a and β face angle;
(3) angle is lifted about judging arm: with α face as the plane of reference, left/right shoulder joint node S
1/ S
2, left/right elbow joint node E
1/ E
2, S
1, E
12 connect be in line is b
1, S
2, E
22 connect be in line is b
2, based on line face angle calcu-lation method calculated line b
1with α face angle=<b
1, α >, straight line b
2with α face angle=<b
2, α >, described straight line b
1be that left arm lifts angle with α face angle, described straight line b
2be that right arm lifts angle with α face angle;
(4) judge to lift angle before and after arm: with β face as the plane of reference, left/right shoulder joint node S
1/ S
2, left/right elbow joint node E
1/ E
2, S
1, E
12 connect be in line is b
1, S
2, E
22 connect be in line is b
2, based on line face angle calcu-lation method calculated line b
1with β face angle=<b
1, β >, straight line b
2with β face angle=<b
2, β >, described straight line b
1be lift angle before and after left arm with β face angle, described straight line b
2be lift angle before and after right arm with β face angle;
(5) angle is lifted about judging leg: with α face as the plane of reference, left/right hip joint node H
1/ H
2, left/right knee joint node K
1/ K
2, H
1, K
12 connect be in line is c
1, H
2, K
22 connect be in line is c
2, based on line face angle calcu-lation method calculated line c
1with α face angle namely=<c
1, α >, straight line c
2with α face angle namely=<c
2, α >, described straight line c
1be that left leg lifts angle with α face angle, straight line c
2be that right leg lifts angle with α face angle;
(6) judge to lift angle before and after leg: with β face as the plane of reference, left/right hip joint node H
1/ H
2, left/right knee joint node K
1/ K
2, H
1, K
12 connect be in line is c
1, H
2, K
22 connect be in line is c
2, based on line face angle calcu-lation method calculated line c
1with β face angle namely=<c
1, β >, straight line c
2with β face angle namely=<c
2, β >, straight line c
1be lift angle before and after left leg, straight line c with β face angle
2be lift angle before and after right leg with β face angle;
(7) judge whether head tilts: with α face for the plane of reference, head node H, shoulder joint Centroid S
02 connect be in lines is d, is head lateral tilting oblique angle based on line face angle calcu-lation method calculated line d and α face angle=<d, α >, straight line d and α face angle;
(8) judge whether head tilts forward and back: with β face for the plane of reference, head node H, shoulder joint Centroid S
02 connect be in lines is d, is head tilt fore and aft based on line face angle calcu-lation method calculated line d and β face angle=<d, β >, straight line d and β face angle.
(9) large arm and forearm angle is judged: left/right shoulder joint node S
1/ S
2, left/right elbow joint node E
1/ E
2, left/right wrist joint node W
1/ W
2, S
1, E
12 connect be in line is b
1, S
2, E
22 connect be in line is b
2, E
1, W
12 connect be in line is e
1, E
2, W
22 connect be in line is e
2, calculate b based on line wire clamp angle computational methods
1, e
1two straight line angle θ
1=<b
1, e
1>, b
2, e
2two straight line angle θ
2=<b
2, e
2>, described b
1, e
1two straight line angle θ
1for left large arm and left forearm angle, described b
2, e
2two straight line angle θ
2for right large arm and right forearm angle;
(10) thigh and shank angle is judged: left/right hip joint node H
1/ H
2, left/right knee joint node K
1/ K
2, left/right ankle-joint node A
1/ A
2, H
1, K
12 connect be in line is c
1, H
2, K
22 connect be in line is c
2, K
1, A
12 connect be in line is f
1, K
2, A
22 connect be in line is f
2, calculate c based on line wire clamp angle computational methods
1, f
1two straight line angles
c
2, f
2two straight line angles
described c
1, f
1two straight line angles
for left thigh and left leg angle, described c
2, f
2two straight line angles
for right thigh and right leg angle.
Line face angle calcu-lation method specifically comprises the following steps:
Suppose straight line AB, plane δ, calculated line AB and plane δ angle and angle between calculated line AB and plane δ normal vector, amount of orientation
for the normal vector of plane δ, suppose
vector representation is
straight line AB vector representation is
Then the angle of straight line AB and plane δ normal vector is:
Then the Circular measure of straight line AB and plane δ angle is expressed as γ=PI-ε.
Owing to using reference axis place plane, so the normal vector of YOZ plane optional (1,0,0); The normal vector of XOY plane is optional (0,0,1).
Line wire clamp angle computational methods specifically comprise the following steps:
Suppose straight line BC, straight line CD, calculated line BC and straight line CD angle and angle between calculated line BC and straight line CD two vector;
Suppose that straight line BC vector representation is
straight line CD vector representation is
Then the angle of straight line BC and straight line CD is:
Compare and prior art, the present invention has the following advantages:
(1) the present invention lays particular emphasis on the training for children's torso harmony, and leg training, can Obtaining Accurate children head, upper limbs, lower limb movement, accurately detect the health total tune action of children in real time, it is high that integrity degree is coordinated in training, be conducive to the raising of child motor exercise for coordination and intelligence, can accurately obtain the limb motion parameters such as children head, upper limbs, lower limb movement, assist rehabilitation;
(2) the present invention increases voice feedback technology, by voice feedback, children are carried out to the encouragement of action prompting lack of standardization and execution, the cordial feeling of enhancing system, make children prefer study and take exercise, by voice feedback, children are carried out to the encouragement of action prompting lack of standardization and execution;
(3) be connected with game engine, increase training interesting; By an interesting game process, complete rehabilitation training, not only contribute to limb function rehabilitation, and contribute to intelligence development; Game is included in children's rehabilitation training, can strengthen human brain bioelectrical signals and connect the stable loop of brain.Transfer the direct interest of children with cerebral palsy with ludic activity form, make children with cerebral palsy participate in therapeutic activity more on one's own initiative, the more important thing is that children with cerebral palsy and should only have the growth that could be realized its physical development and psychology by game;
(4) the present invention utilizes line wire clamp angle to assess upper limbs or whether lower limb stretch and whether can make required movement, and calculate simple, response is fast;
(5) exercise data of in real time and dynamically record patient, for doctor and therapist's remote evaluation, instruct formulate, adjustment and optimize rehabilitation scheme; Liberated the labour of therapist, made again training avoid blindness, doctor and therapist, for the rehabilitation situation of children, optimize training program.
(6) leg training, can train the attitude of walking and the attitude of standing;
(7) low cost and personalized service: Kinect are compared with other train medicine equipment, and advantage is cost.Utilize general family existence conditions, the training system with feedback function can be realized.Compared to the transport cost of hurrying back and forth all the year round and human cost, total system cost is very low, and general families with low and middle income also can accept substantially; And by internet, utilizing this system, the patient of remote districts also can obtain personalized medical services.
Accompanying drawing explanation
Fig. 1 is the children with cerebral palsy rehabilitation training system operating diagram based on Kinect sensor of the present invention;
Fig. 2 is the children with cerebral palsy rehabilitation training system structural representation figure based on Kinect sensor of the present invention;
Fig. 3 is seal of the present invention game schematic diagram;
Fig. 4 is the body artis location drawing;
Fig. 5 is body normal stand schematic diagram;
Fig. 6 is that body tilts schematic diagram;
Fig. 7 is that body tilts judgement figure;
Fig. 8 is that body tilts forward and back judgement figure;
Fig. 9 is a kind of children with cerebral palsy recovery training method schematic flow sheet based on Kinect sensor of the present invention.
Detailed description of the invention
Below in conjunction with accompanying drawing, the present invention is further described.
In order to deepen the understanding of the present invention, below in conjunction with embodiment and accompanying drawing, the invention will be further described, and this embodiment only for explaining the present invention, does not form limiting the scope of the present invention.
As depicted in figs. 1 and 2, the children with cerebral palsy rehabilitation training system based on Kinect sensor comprises Kinect depth camera, computer, in addition, because needs carry out remote evaluation, so computer needs interconnection network.When children train, open the training system on computer, Kinect depth camera is connected with computer; Before children stand in Kinect depth camera, according to the guiding on the human-computer interaction interface that computer shows, make corresponding action; Kinect depth camera captures the limb action of children, and data are passed to computer; Computer processing data, obtains a result, and result is presented on human-computer interaction interface, if action of doing meets the requirements, then carries out the next item down, otherwise proceeds this item.The data obtained can be real-time transmitted to doctor by native system, and doctor according to data, can instruct training effect, and then improves drill program.
For body-correcting training, designer's body seal game of the present invention.People's body seal game point is two kinds of patterns: plane chop mode and three-dimensional chop mode.Plane chop mode uses when namely doing planar gesture, as shown in Figure 3, uses front view seal; Three-dimensional chop mode uses when namely doing stereoscopic motion, uses front view seal and left view seal.Children only show the action identical with seal, just can enter next link and obtain corresponding award; If it is different from seal to show action, then according to estimate of situation, use image or voice to guide children, make it show identical action.Game the data obtained can be fed back to doctor by system, and doctor can assess according to the rehabilitation situation of data pair child, optimizes training program.
As shown in Figure 9, a kind of children with cerebral palsy recovery training method based on Kinect sensor, comprises the following steps:
The skeleton point data of S1, acquisition children: Kinect sensor (Kinect depth camera) is connected with computer, computer accesses network (connection Internet network), the present embodiment obtains skeleton point data by the api function in Kinect for Windows, (the x that skeleton point data are is initial point with Kinect depth camera, y, z) coordinate value; As shown in Figure 4, be the skeleton point data that Kinect depth camera can capture, the skeleton point needed for native system is head node H, shoulder joint Centroid S
0, backbone Centroid S, left shoulder joint node S
1, right shoulder joint node S
2, left elbow joint node E
1, right elbow joint node E
2, left wrist joint node W
1, right wrist joint node W
2, left hip joint node H
1, right hip joint node H
2, left knee joint node K
1, right knee joint node K
2, left ankle-joint node A
1, right ankle-joint node A
2.
S2, limb motion is trained, the skeleton point of children is tilted, lifts angle judgement: after children show action, catch child motor, in three-dimensional planar rectangular coordinate system, getting YOZ plane is α face, and getting XOY plane is β face, tilts, lifts angle judgement to each artis of head, upper limbs and lower limb.
S3, the sane interaction process of child motor: processed by wavelet de-noising, the bone shake of moving average filter noise reduction to motion, by medium filtering, image cavity is processed, rejecting abnormalities depth value, degree of depth missing values is estimated; (magnetite method is a kind of automatic absorbing method to adopt magnetite method the action in threshold range and virtual image to be carried out autoregistration, children action may have small size shake when training, as long as with standard operation in certain threshold range, automatic absorbing is standard operation); Virtual image is the Avatar cartoon character with Practical computer teaching that children like, children's bone point data is obtained according to the api function in Kinect forWindows, skeleton point data syn-chronization is mapped on the virtual portrait on computer screen, the posture behavior of real-time tracking children.
S4, connects game engine, the children's skeleton data after sane for step S3 interaction process is sent to game engine, and the action that skeleton data is formed is converted to the event of game engine definition, game is included in children's rehabilitation training;
Step S4 converts the action of children the event that game engine defines to, in gaming for these events add the access that the method processed just can realize human body tracing equipment, this is relevant with concrete game design, the game such as trundled, if children imitate the action hand that trundles and brandish from back to front, virtual portrait in computer scene also will complete the action that trundles, the motion of ball is that the event that predefined is good (comprises the original position of ball, the movement velocity of ball, the direction of motion of ball, the whole motion process of stop position of ball), after the gesture that system identification is brandished from back to front to children, understand the event that this predefined of motion of trigger ball immediately is good.As, if children draw a circle aloft, game just suspends.Draw the action that circle is children, after system identification to picture circle gesture, trigger this event of suspension of game play, for suspension of game play adds processing method in games, the core of this part is in game engine, write Kinect plug-in unit, complete the access of Kinect, the gesture of such children could by system identification.
S5, voice feedback child motor, by the result of autoregistration described in step S3 by voice feedback to children, carry out the encouragement of action prompting lack of standardization and execution.
S6, the rehabilitation training progress of assessment children: the skeleton point of real time record children and the registration of virtual image, and described registration is sent to remote evaluation terminal, assessment rehabilitation training performance.
Step S2 also comprises ambulation training, calculates left right foot striding step, speed index, differentiates whether the gait of children is normal gait.
The skeleton point of step S2 to children tilts, lift angle judges, the inclination of children's body, poised state can be followed the trail of, the position of the artis such as follower head, trunk, shoulder, pelvis, knee, ankle and angle of inclination, judge that whether the posture of children is abnormal, the coordination realizing children obtains, and specifically comprises the following steps:
(1) judge whether body tilts: with α face for the plane of reference, shoulder joint Centroid S
0, backbone Centroid S 2 connect be in line be a, be the lateral tilting oblique angle of body based on line face angle calcu-lation method calculated line a and α face angle=<a, α >, described straight line a and α face angle; As shown in Figure 5, under normal circumstances, (namely straight line is a) 0 with the angle in α face to trunk.When body tilts, as shown in Figure 6 and Figure 7, straight line a and α face angle=<a, α >;
(2) judge whether body tilts forward and back: with β face for the plane of reference, shoulder joint Centroid S
0, backbone Centroid S, 2 connect be in lines be a, be the tilt fore and aft of body based on line face angle calcu-lation method calculated line a and β face angle=<a, β >, described straight line a and β face angle; As shown in Figure 8, then straight line a and β face angle=<a, β >;
(3) angle is lifted about judging arm: with α face as the plane of reference, left/right shoulder joint node S
1/ S
2, left/right elbow joint node E
1/ E
2, S
1, E
12 connect be in line is b
1, S
2, E
22 connect be in line is b
2, based on line face angle calcu-lation method calculated line b
1with α face angle=<b
1, α >, straight line b
2with α face angle=<b
2, α >, described straight line b
1be that left arm lifts angle with α face angle, described straight line b
2be that right arm lifts angle with α face angle;
(4) judge to lift angle before and after arm: with β face as the plane of reference, left/right shoulder joint node S
1/ S
2, left/right elbow joint node E
1/ E
2, S
1, E
12 connect be in line is b
1, S
2, E
22 connect be in line is b
2, based on line face angle calcu-lation method calculated line b
1with β face angle=<b
1, β >, straight line b
2with β face angle=<b
2, β >, described straight line b
1be lift angle before and after left arm with β face angle, described straight line b
2be lift angle before and after right arm with β face angle;
(5) angle is lifted about judging leg: with α face as the plane of reference, left/right hip joint node H
1/ H
2, left/right knee joint node K
1/ K
2, H
1, K
12 connect be in line is c
1, H
2, K
22 connect be in line is c
2, based on line face angle calcu-lation method calculated line c
1with α face angle namely=<c
1, α >, straight line c
2with α face angle namely=<c
2, α >, described straight line c
1be that left leg lifts angle with α face angle, straight line c
2be that right leg lifts angle with α face angle;
(6) judge to lift angle before and after leg: with β face as the plane of reference, left/right hip joint node H
1/ H
2, left/right knee joint node K
1/ K
2, H
1, K
12 connect be in line is c
1, H
2, K
22 connect be in line is c
2, based on line face angle calcu-lation method calculated line c
1with β face angle namely=<c
1, β >, straight line c
2with β face angle namely=<c
2, β >, straight line c
1be lift angle before and after left leg, straight line c with β face angle
2be lift angle before and after right leg with β face angle;
(7) judge whether head tilts: with α face for the plane of reference, head node H, shoulder joint Centroid S
02 connect be in lines is d, is head lateral tilting oblique angle based on line face angle calcu-lation method calculated line d and α face angle=<d, α >, straight line d and α face angle;
(8) judge whether head tilts forward and back: with β face for the plane of reference, head node H, shoulder joint Centroid S
02 connect be in lines is d, is head tilt fore and aft based on line face angle calcu-lation method calculated line d and β face angle=<d, β >, straight line d and β face angle.
(9) large arm and forearm angle is judged: left/right shoulder joint node S
1/ S
2, left/right elbow joint node E
1/ E
2, left/right wrist joint node W
1/ W
2, S
1, E
12 connect be in line is b
1, S
2, E
22 connect be in line is b
2, E
1, W
12 connect be in line is e
1, E
2, W
22 connect be in line is e
2, calculate b based on line wire clamp angle computational methods
1, e
1two straight line angle θ
1=<b
1, e
1>, b
2, e
2two straight line angle θ
2=<b
2, e
2>, described b
1, e
1two straight line angle θ
1for left large arm and left forearm angle, described b
2, e
2two straight line angle θ
2for right large arm and right forearm angle;
(10) thigh and shank angle is judged: left/right hip joint node H
1/ H
2, left/right knee joint node K
1/ K
2, left/right ankle-joint node A
1/ A
2, H
1, K
12 connect be in line is c
1, H
2, K
22 connect be in line is c
2, K
1, A
12 connect be in line is f
1, K
2, A
22 connect be in line is f
2, calculate c based on line wire clamp angle computational methods
1, f
1two straight line angles
c
2, f
2two straight line angles
described c
1, f
1two straight line angles
for left thigh and left leg angle, described c
2, f
2two straight line angles
for right thigh and right leg angle.
Line face angle calcu-lation method specifically comprises the following steps:
Suppose straight line AB, plane δ, calculated line AB and plane δ angle and angle between calculated line AB and plane δ normal vector, amount of orientation
for the normal vector of plane δ, suppose
vector representation is
straight line AB vector representation is
Then the angle of straight line AB and plane δ normal vector is:
Then the Circular measure of straight line AB and plane δ angle is expressed as γ=PI-ε.
Owing to using reference axis place plane, so the normal vector of YOZ plane optional (1,0,0); The normal vector of XOY plane is optional (0,0,1).
Line wire clamp angle computational methods specifically comprise the following steps:
Suppose straight line BC, straight line CD, calculated line BC and straight line CD angle and angle between calculated line BC and straight line CD two vector;
Suppose that straight line BC vector representation is
Straight line CD vector representation is
(AX, AY, AZ), (BX, BY, BZ), (CX, CY, CZ), (DX, DY, DZ) are respectively the upper line segment extreme coordinates of straight line AB, straight line BC, straight line CD;
Then the angle of straight line BC and straight line CD is:
According to above computational methods, the seal action (seal of the present embodiment is played) child motor and interactive interface shown is compared.In order to native system can be enable to be suitable for the different children of extent, the scope of seal action can be adjusted, namely can produce the different game of complexity, the interest of children for learning can be excited better.
Below be only the preferred embodiment of the present invention; be noted that for those skilled in the art; under the premise without departing from the principles of the invention, can also make some improvements and modifications, these improvements and modifications also should be considered as protection scope of the present invention.
Claims (6)
1., based on a children with cerebral palsy recovery training method for Kinect sensor, it is characterized in that, comprise the following steps:
The skeleton point data of S1, acquisition children: Kinect sensor is connected with computer, computer accesses network, Kinect sensor obtains child image, the skeleton point data that function obtains children are caught by the bone of Kinect, the coordinate value that skeleton point data are is initial point with Kinect depth camera, skeleton point comprises head node H, shoulder joint Centroid S
0, backbone Centroid S, left shoulder joint node S
1, right shoulder joint node S
2, left elbow joint node E
1, right elbow joint node E
2, left wrist joint node W
1, right wrist joint node W
2, left hip joint node H
1, right hip joint node H
2, left knee joint node K
1, right knee joint node K
2, left ankle-joint node A
1, right ankle-joint node A
2;
S2, limb motion is trained, the skeleton point of children is tilted, lifts angle judgement: after children show action, catch child motor, in three-dimensional planar rectangular coordinate system, getting YOZ plane is α face, and getting XOY plane is β face, tilts, lifts angle judgement to each artis of head, upper limbs and lower limb;
S3, the sane interaction process of child motor;
S4, connects game engine, the children's skeleton data after sane for step S3 interaction process is sent to game engine, and the action that skeleton data is formed is converted to the event of game engine definition, game is included in children's rehabilitation training;
S5, voice feedback child motor: by the result of autoregistration described in step S3 by voice feedback to children, carry out the encouragement of action prompting lack of standardization and execution;
S6, the rehabilitation training progress of assessment children: the skeleton point of real time record children and the registration of virtual image, and described registration is sent to remote evaluation terminal, assessment rehabilitation training performance.
2. a kind of children with cerebral palsy recovery training method based on Kinect sensor according to claim 1, it is characterized in that, step S2 also comprises ambulation training, calculates left right foot striding step, speed index, differentiates whether the gait of children is normal gait.
3. a kind of children with cerebral palsy recovery training method based on Kinect sensor according to claim 1, is characterized in that, the skeleton point of step S2 to children tilts, lifts angle judgement, specifically comprise the following steps:
(1) judge whether body tilts: with α face for the plane of reference, shoulder joint Centroid S
0, backbone Centroid S 2 connect be in line be a, be the lateral tilting oblique angle of body based on line face angle calcu-lation method calculated line a and α face angle=<a, α >, described straight line a and α face angle;
(2) judge whether body tilts forward and back: with β face for the plane of reference, shoulder joint Centroid S
0, backbone Centroid S, 2 connect be in lines be a, be the tilt fore and aft of body based on line face angle calcu-lation method calculated line a and β face angle=<a, β >, described straight line a and β face angle;
(3) angle is lifted about judging arm: with α face as the plane of reference, left/right shoulder joint node S
1/ S
2, left/right elbow joint node E
1/ E
2, S
1, E
12 connect be in line is b
1, S
2, E
22 connect be in line is b
2, based on line face angle calcu-lation method calculated line b
1with α face angle=<b
1, α >, straight line b
2with α face angle=<b
2, α >, described straight line b
1be that left arm lifts angle with α face angle, described straight line b
2be that right arm lifts angle with α face angle;
(4) judge to lift angle before and after arm: with β face as the plane of reference, left/right shoulder joint node S
1/ S
2, left/right elbow joint node E
1/ E
2, S
1, E
12 connect be in line is b
1, S
2, E
22 connect be in line is b
2, based on line face angle calcu-lation method calculated line b
1with β face angle=<b
1, β >, straight line b
2with β face angle=<b
2, β >, described straight line b
1be lift angle before and after left arm with β face angle, described straight line b
2be lift angle before and after right arm with β face angle;
(5) angle is lifted about judging leg: with α face as the plane of reference, left/right hip joint node H
1/ H
2, left/right knee joint node K
1/ K
2, H
1, K
12 connect be in line is c
1, H
2, K
22 connect be in line is c
2, based on line face angle calcu-lation method calculated line c
1with α face angle namely=<c
1, α >, straight line c
2with α face angle namely=<c
2, α >, described straight line c
1be that left leg lifts angle with α face angle, straight line c
2be that right leg lifts angle with α face angle;
(6) judge to lift angle before and after leg: with β face as the plane of reference, left/right hip joint node H
1/ H
2, left/right knee joint node K
1/ K
2, H
1, K
12 connect be in line is c
1, H
2, K
22 connect be in line is c
2, based on line face angle calcu-lation method calculated line c
1with β face angle namely=<c
1, β >, straight line c
2with β face angle namely=<c
2, β >, straight line c
1be lift angle before and after left leg, straight line c with β face angle
2be lift angle before and after right leg with β face angle;
(7) judge whether head tilts: with α face for the plane of reference, head node H, shoulder joint Centroid S
02 connect be in lines is d, is head lateral tilting oblique angle based on line face angle calcu-lation method calculated line d and α face angle=<d, α >, straight line d and α face angle;
(8) judge whether head tilts forward and back: with β face for the plane of reference, head node H, shoulder joint Centroid S
02 connect be in lines is d, is head tilt fore and aft based on line face angle calcu-lation method calculated line d and β face angle=<d, β >, straight line d and β face angle;
(9) large arm and forearm angle is judged: left/right shoulder joint node S
1/ S
2, left/right elbow joint node E
1/ E
2, left/right wrist joint node W
1/ W
2, S
1, E
12 connect be in line is b
1, S
2, E
22 connect be in line is b
2, E
1, W
12 connect be in line is e
1, E
2, W
22 connect be in line is e
2, calculate b based on line wire clamp angle computational methods
1, e
1two straight line angle θ
i=<b
1, e
1>, b
2, e
2two straight line angle θ
2=<b
2, e
2>, described b
1, e
1two straight line angle θ
ifor left large arm and left forearm angle, described b
2, e
2two straight line angle θ
2for right large arm and right forearm angle;
(10) thigh and shank angle is judged: left/right hip joint node H
1/ H
2, left/right knee joint node K
1/ K
2, left/right ankle-joint node A
1/ A
2, H
1, K
12 connect be in line is c
1, H
2, K
22 connect be in line is c
2, K
1, A
12 connect be in line is f
1, K
2, A
22 connect be in line is f
2, calculate c based on line wire clamp angle computational methods
1, f
1two straight line angles
c
2, f
2two straight line angles
described c
1, f
1two straight line angles
for left thigh and left leg angle, described c
2, f
2two straight line angles
for right thigh and right leg angle.
4. a kind of children with cerebral palsy recovery training method based on Kinect sensor according to claim 1, is characterized in that,
The sane interaction process of child motor described in step S3 comprises:
Processed by wavelet de-noising, the bone shake of moving average filter noise reduction to motion; By medium filtering, image cavity is processed, rejecting abnormalities depth value, degree of depth missing values is estimated; Adopt magnetite method that registration is carried out in the action in threshold range and virtual image.
5. a kind of children with cerebral palsy recovery training method based on Kinect sensor according to claim 3, is characterized in that, described line face angle calcu-lation method specifically comprises the following steps:
Suppose straight line AB, plane δ, calculated line AB and plane δ angle and angle between calculated line AB and plane δ normal vector, amount of orientation
for the normal vector of plane δ, suppose
vector representation is
straight line AB vector representation is
Then the angle of straight line AB and plane δ normal vector is:
Then the Circular measure of straight line AB and plane δ angle is expressed as γ=PI-ε.
6. a kind of children with cerebral palsy recovery training method based on Kinect sensor according to claim 3, is characterized in that, described line wire clamp angle computational methods specifically comprise the following steps:
Suppose straight line BC, straight line CD, calculated line BC and straight line CD angle and angle between calculated line BC and straight line CD two vector;
Suppose that straight line BC vector representation is
straight line CD vector representation is
y
3);
Then the angle of straight line BC and straight line CD is:
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510001948.3A CN104524742A (en) | 2015-01-05 | 2015-01-05 | Cerebral palsy child rehabilitation training method based on Kinect sensor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510001948.3A CN104524742A (en) | 2015-01-05 | 2015-01-05 | Cerebral palsy child rehabilitation training method based on Kinect sensor |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104524742A true CN104524742A (en) | 2015-04-22 |
Family
ID=52840478
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510001948.3A Pending CN104524742A (en) | 2015-01-05 | 2015-01-05 | Cerebral palsy child rehabilitation training method based on Kinect sensor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104524742A (en) |
Cited By (30)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105469679A (en) * | 2015-11-14 | 2016-04-06 | 辽宁大学 | Cardio-pulmonary resuscitation assisted training system and cardio-pulmonary resuscitation assisted training method based on Kinect |
| CN106485055A (en) * | 2016-09-22 | 2017-03-08 | 吉林大学 | A kind of old type ii diabetes patient moving training system based on Kinect sensor |
| CN106650687A (en) * | 2016-12-30 | 2017-05-10 | 山东大学 | Posture correction method based on depth information and skeleton information |
| CN106667493A (en) * | 2017-01-22 | 2017-05-17 | 河北大学 | Human body balance assessment system and assessment method |
| CN107158659A (en) * | 2017-07-12 | 2017-09-15 | 南京邮电大学 | A kind of long-range rehabilitation training of upper limbs system and method for game type based on Kinect |
| CN107174255A (en) * | 2017-06-15 | 2017-09-19 | 西安交通大学 | Three-dimensional gait information gathering and analysis method based on Kinect somatosensory technology |
| CN107212975A (en) * | 2017-07-17 | 2017-09-29 | 徐彬 | Wheelchair and method with intelligent rescue function |
| CN107273677A (en) * | 2017-06-08 | 2017-10-20 | 中国科学院软件研究所 | A kind of multi-channel nerve function quantitative evaluation system |
| CN107485542A (en) * | 2017-08-24 | 2017-12-19 | 深圳市罗伯医疗科技有限公司 | A kind of healing robot leg speed method of adjustment and system |
| CN107831897A (en) * | 2017-11-17 | 2018-03-23 | 吉林大学 | RULA evaluating methods in a kind of Virtual assemble operation |
| CN108830206A (en) * | 2018-06-06 | 2018-11-16 | 成都邑教云信息技术有限公司 | A kind of course axis Internet Educational System |
| CN108919950A (en) * | 2018-06-26 | 2018-11-30 | 上海理工大学 | Autism children based on Kinect interact device for image and method |
| CN108926813A (en) * | 2017-05-22 | 2018-12-04 | 北京大学 | Training method based on total balance of the body data |
| CN108968973A (en) * | 2018-08-07 | 2018-12-11 | 南通大学 | A kind of acquisition of body gait and analysis system and method |
| CN109035879A (en) * | 2018-07-26 | 2018-12-18 | 张家港市青少年社会实践基地 | A kind of teenager's intelligent robot teaching method and device |
| CN109102857A (en) * | 2018-05-31 | 2018-12-28 | 杭州同绘科技有限公司 | A kind of intelligence limb rehabilitation training system and method |
| CN109550233A (en) * | 2018-11-15 | 2019-04-02 | 东南大学 | Autism child attention training system based on augmented reality |
| CN109788909A (en) * | 2016-08-09 | 2019-05-21 | 阿达森瑟健康有限公司 | For body gesture and the system and method for motor adjustment |
| CN110855861A (en) * | 2019-10-16 | 2020-02-28 | 建荣半导体（深圳）有限公司 | Online game method and device for child camera and child camera |
| CN110941990A (en) * | 2019-10-22 | 2020-03-31 | 泰康保险集团股份有限公司 | Method and device for evaluating human body actions based on skeleton key points |
| CN112044023A (en) * | 2019-06-07 | 2020-12-08 | 丰田自动车株式会社 | Walking training system, storage medium storing control program for walking training system, and method for controlling walking training system |
| CN112966370A (en) * | 2021-02-09 | 2021-06-15 | 武汉纺织大学 | Design method of human body lower limb muscle training system based on Kinect |
| CN113398537A (en) * | 2021-06-15 | 2021-09-17 | 黄富表 | Microsoft Kinect-based upper limb motor function training method for stroke patient |
| WO2021219643A1 (en) | 2020-04-29 | 2021-11-04 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Method for ascertaining the development or developmental state of a small child or infant |
| CN114145710A (en) * | 2020-09-08 | 2022-03-08 | Oppo（重庆）智能科技有限公司 | Body data detection method and device and electronic equipment |
| WO2022085929A1 (en) * | 2020-10-20 | 2022-04-28 | 삼성전자주식회사 | Electronic apparatus and control method therefor |
| CN115841864A (en) * | 2023-02-22 | 2023-03-24 | 佛山科学技术学院 | Rehabilitation exercise quality assessment method and system |
| CN116843805A (en) * | 2023-06-19 | 2023-10-03 | 上海奥玩士信息技术有限公司 | Method, device, equipment and medium for generating virtual image containing behaviors |
| CN118079329A (en) * | 2024-04-29 | 2024-05-28 | 佛山科学技术学院 | Upper limb rehabilitation training method and device combining blood oxygen signals and virtual reality interaction |
| TWI848235B (en) * | 2020-09-29 | 2024-07-11 | 日商田邊三菱製藥股份有限公司 | Neuromuscular disease assessment system |
-
2015
- 2015-01-05 CN CN201510001948.3A patent/CN104524742A/en active Pending
Non-Patent Citations (1)
| Title |
|---|
| 徐如祥: "《神经肝细胞》", 1 January 2006 * |
Cited By (42)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105469679A (en) * | 2015-11-14 | 2016-04-06 | 辽宁大学 | Cardio-pulmonary resuscitation assisted training system and cardio-pulmonary resuscitation assisted training method based on Kinect |
| CN109788909A (en) * | 2016-08-09 | 2019-05-21 | 阿达森瑟健康有限公司 | For body gesture and the system and method for motor adjustment |
| US11957457B2 (en) | 2016-08-09 | 2024-04-16 | Addasense Health Gmbh | System and method for posture and movement regulation |
| CN106485055A (en) * | 2016-09-22 | 2017-03-08 | 吉林大学 | A kind of old type ii diabetes patient moving training system based on Kinect sensor |
| CN106650687A (en) * | 2016-12-30 | 2017-05-10 | 山东大学 | Posture correction method based on depth information and skeleton information |
| CN106667493A (en) * | 2017-01-22 | 2017-05-17 | 河北大学 | Human body balance assessment system and assessment method |
| CN108926813A (en) * | 2017-05-22 | 2018-12-04 | 北京大学 | Training method based on total balance of the body data |
| CN107273677A (en) * | 2017-06-08 | 2017-10-20 | 中国科学院软件研究所 | A kind of multi-channel nerve function quantitative evaluation system |
| CN107174255B (en) * | 2017-06-15 | 2020-04-10 | 西安交通大学 | Three-dimensional gait information acquisition and analysis method based on Kinect somatosensory technology |
| CN107174255A (en) * | 2017-06-15 | 2017-09-19 | 西安交通大学 | Three-dimensional gait information gathering and analysis method based on Kinect somatosensory technology |
| CN107158659A (en) * | 2017-07-12 | 2017-09-15 | 南京邮电大学 | A kind of long-range rehabilitation training of upper limbs system and method for game type based on Kinect |
| CN107212975A (en) * | 2017-07-17 | 2017-09-29 | 徐彬 | Wheelchair and method with intelligent rescue function |
| CN107485542A (en) * | 2017-08-24 | 2017-12-19 | 深圳市罗伯医疗科技有限公司 | A kind of healing robot leg speed method of adjustment and system |
| CN107831897B (en) * | 2017-11-17 | 2018-09-07 | 吉林大学 | RULA evaluating methods in a kind of operation of Virtual assemble |
| CN107831897A (en) * | 2017-11-17 | 2018-03-23 | 吉林大学 | RULA evaluating methods in a kind of Virtual assemble operation |
| CN109102857A (en) * | 2018-05-31 | 2018-12-28 | 杭州同绘科技有限公司 | A kind of intelligence limb rehabilitation training system and method |
| CN109102857B (en) * | 2018-05-31 | 2021-08-27 | 杭州同绘科技有限公司 | Intelligent limb rehabilitation training system and method |
| CN108830206A (en) * | 2018-06-06 | 2018-11-16 | 成都邑教云信息技术有限公司 | A kind of course axis Internet Educational System |
| CN108919950A (en) * | 2018-06-26 | 2018-11-30 | 上海理工大学 | Autism children based on Kinect interact device for image and method |
| CN109035879A (en) * | 2018-07-26 | 2018-12-18 | 张家港市青少年社会实践基地 | A kind of teenager's intelligent robot teaching method and device |
| CN108968973A (en) * | 2018-08-07 | 2018-12-11 | 南通大学 | A kind of acquisition of body gait and analysis system and method |
| CN109550233A (en) * | 2018-11-15 | 2019-04-02 | 东南大学 | Autism child attention training system based on augmented reality |
| CN112044023B (en) * | 2019-06-07 | 2022-04-08 | 丰田自动车株式会社 | Walking training system, storage medium storing control program for walking training system, and method for controlling walking training system |
| CN112044023A (en) * | 2019-06-07 | 2020-12-08 | 丰田自动车株式会社 | Walking training system, storage medium storing control program for walking training system, and method for controlling walking training system |
| CN110855861A (en) * | 2019-10-16 | 2020-02-28 | 建荣半导体（深圳）有限公司 | Online game method and device for child camera and child camera |
| CN110855861B (en) * | 2019-10-16 | 2021-11-30 | 建荣半导体（深圳）有限公司 | Online game method and device for child camera and child camera |
| CN110941990A (en) * | 2019-10-22 | 2020-03-31 | 泰康保险集团股份有限公司 | Method and device for evaluating human body actions based on skeleton key points |
| CN110941990B (en) * | 2019-10-22 | 2023-06-16 | 泰康保险集团股份有限公司 | Method and device for evaluating human body actions based on skeleton key points |
| WO2021219643A1 (en) | 2020-04-29 | 2021-11-04 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Method for ascertaining the development or developmental state of a small child or infant |
| DE102020205434A1 (en) | 2020-04-29 | 2021-11-04 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung eingetragener Verein | Method for determining a development or a developmental stage of a toddler or infant |
| CN114145710B (en) * | 2020-09-08 | 2023-08-29 | Oppo（重庆）智能科技有限公司 | Body data detection method and device and electronic equipment |
| CN114145710A (en) * | 2020-09-08 | 2022-03-08 | Oppo（重庆）智能科技有限公司 | Body data detection method and device and electronic equipment |
| TWI848235B (en) * | 2020-09-29 | 2024-07-11 | 日商田邊三菱製藥股份有限公司 | Neuromuscular disease assessment system |
| WO2022085929A1 (en) * | 2020-10-20 | 2022-04-28 | 삼성전자주식회사 | Electronic apparatus and control method therefor |
| CN112966370B (en) * | 2021-02-09 | 2022-04-19 | 武汉纺织大学 | Design method of human body lower limb muscle training system based on Kinect |
| CN112966370A (en) * | 2021-02-09 | 2021-06-15 | 武汉纺织大学 | Design method of human body lower limb muscle training system based on Kinect |
| CN113398537A (en) * | 2021-06-15 | 2021-09-17 | 黄富表 | Microsoft Kinect-based upper limb motor function training method for stroke patient |
| CN115841864A (en) * | 2023-02-22 | 2023-03-24 | 佛山科学技术学院 | Rehabilitation exercise quality assessment method and system |
| CN116843805A (en) * | 2023-06-19 | 2023-10-03 | 上海奥玩士信息技术有限公司 | Method, device, equipment and medium for generating virtual image containing behaviors |
| CN116843805B (en) * | 2023-06-19 | 2024-03-19 | 上海奥玩士信息技术有限公司 | Method, device, equipment and medium for generating virtual image containing behaviors |
| CN118079329A (en) * | 2024-04-29 | 2024-05-28 | 佛山科学技术学院 | Upper limb rehabilitation training method and device combining blood oxygen signals and virtual reality interaction |
| CN118079329B (en) * | 2024-04-29 | 2024-06-25 | 佛山科学技术学院 | Upper limb rehabilitation training method and device combining blood oxygen signals and virtual reality interaction |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104524742A (en) | Cerebral palsy child rehabilitation training method based on Kinect sensor | |
| CN108854034B (en) | Cerebral apoplexy rehabilitation training system based on virtual reality and inertial motion capture | |
| Zhou et al. | Human motion tracking for rehabilitation—A survey | |
| EP3986266A1 (en) | Wearable joint tracking device with muscle activity and methods thereof | |
| CN104274183B (en) | Action message processing unit | |
| Zannatha et al. | Development of a system based on 3D vision, interactive virtual environments, ergonometric signals and a humanoid for stroke rehabilitation | |
| US8900165B2 (en) | Balance training system | |
| CN106420254A (en) | Multi-person interactive virtual reality rehabilitation training and evaluation system | |
| CN104274182A (en) | Motion information processing apparatus and method | |
| WO2012106978A1 (en) | Method for controlling man-machine interaction and application thereof | |
| CN106693280A (en) | Virtual-reality-based Parkinsonism training method, system and device | |
| CN107281728B (en) | Sensor-matched augmented reality skiing auxiliary training system and method | |
| CN109288651A (en) | Personalized upper-limbs rehabilitation training robot system and its recovery training method | |
| CN104856684A (en) | Moving object acquisition method and system | |
| CN113571153A (en) | Passive training sensing system for limb rehabilitation of cerebral palsy patient and client side thereof | |
| CN113066260B (en) | Parkinson's disease early warning system based on daily behavior analysis | |
| CN110404243A (en) | A kind of method of rehabilitation and rehabilitation system based on posture measurement | |
| CN110812798B (en) | VR-based forearm internal and external rotation training evaluation system and method and storage medium | |
| Brooks et al. | Quantifying upper‐arm rehabilitation metrics for children through interaction with a humanoid robot | |
| CN110415783A (en) | A kind of Functional Activities of OT method of rehabilitation based on body-sensing | |
| CN114366557A (en) | Man-machine interaction system and method for lower limb rehabilitation robot | |
| CN111312363B (en) | Double-hand coordination enhancement system based on virtual reality | |
| CN117503115A (en) | Rehabilitation training system and training method for nerve injury | |
| CN111243705B (en) | Self-adaptive VR mirror training system | |
| CN110473602B (en) | Body state data collection processing method for wearable body sensing game device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20150422 |