CN114100102B

CN114100102B - Open-close jump test counting method, system and medium based on vision technology

Info

Publication number: CN114100102B
Application number: CN202111191675.5A
Authority: CN
Inventors: 章珠明; 曹礼玉
Original assignee: Hengonda Technology Co ltd
Current assignee: Henghongda Fujian Sports Technology Co ltd
Priority date: 2021-10-13
Filing date: 2021-10-13
Publication date: 2023-03-28
Anticipated expiration: 2041-10-13
Also published as: CN114100102A

Abstract

The invention provides a method, a system and a medium for counting open-close jump tests based on a vision technology, wherein the method comprises the following steps: judging whether the testing personnel meet the preparation action standard or not according to the acquired preparation picture, if so, starting testing and counting, and entering the next step; if not, prompting; acquiring all pictures according to the time sequence, judging the action state type in each picture and the keeping time length of the action state, deleting the picture if the keeping time length of the action state is less than a set threshold value, and entering the next step if the keeping time length of the action state is not less than the set threshold value; judging whether the front and back pictures accord with the condition of completing one opening and closing jump or not, and if so, counting and adding 1; if not, not counting; the counting is more accurate, and the action is more standardized.

Description

Open-close jump test counting method, system and medium based on vision technology

Technical Field

The invention relates to the technical field of computers, in particular to a method, a system and a medium for counting open-close jump tests based on a vision technology.

Background

Along with the development of society and economy, people attach more and more importance to the physical training, open and shut and jump is a systemic exercise action, belongs to the aerobic exercise, can temper the muscle of arm, shank, and whole body's muscle can both be driven moreover, and your rhythm of the heart of improvement that it can be quick lets your health get into the state of burning fat fast.

In the prior art, a special test counting method for the open-close jump is not provided, and manual counting is performed, so that the open-close jump movement specification cannot be unified, the open-close jump movement specification is judged by a counting person, and the randomness is high.

Disclosure of Invention

The invention aims to solve the technical problem of providing a method, a system and a medium for counting the open-close jump tests based on a visual technology, so that the actions are more standard, the counting is more accurate, and the efficiency is higher.

In a first aspect, the invention provides a method for counting open-close jump tests based on a visual technology, which comprises the following steps:

step 1, judging whether a tester accords with a preparation action standard or not according to the acquired preparation picture, if so, starting test counting, and entering step 2; if not, prompting is carried out;

step 2, acquiring all pictures according to the time sequence, judging the action state type in each picture and the holding time of the action state, if the holding time of the action state is less than a set threshold value, deleting the picture, otherwise, entering step 3;

step 3, judging whether the front and back pictures accord with each other to finish one opening and closing jump, and if so, counting and adding 1; if not, not counting;

step 4, repeating the step 2 and the step 3 until the test is finished, and then displaying a result;

the preparation action criteria in step 1 include:

the judgment basis of the state of the human body facing the camera is as follows: when the X-axis coordinate of the right ear node is smaller than that of the right eye node, the X-axis coordinate of the right eye node is smaller than that of the left eye node, and the X-axis coordinate of the left eye node is smaller than that of the left ear node, the human body is in a state of being over against the camera; if not, the human body is not in a state of facing the camera;

the judgment basis of the two-leg opening and straightening state is as follows: when the included angle between the connecting line of the left hip node and the left knee joint node and the connecting line of the left knee joint node and the left ankle joint node is larger than a first set threshold value, and the included angle between the connecting line of the right hip node and the right knee joint node and the connecting line of the right knee joint node and the right ankle joint node is larger than the first set threshold value, the state is that the two legs are in a straight state, otherwise, the state is that the two legs are not in a straight state; when the angle of a connecting line between the left knee joint node and the left hip node and the angle of a connecting line between the right knee joint node and the right hip node are larger than a second set threshold value, the two legs are in an open state; when the angle of a connecting line between the left knee joint node and the left hip node and the angle of a connecting line between the right knee joint node and the right hip node are smaller than a third set threshold value, the two legs are in a close state; only when the two legs are in an opened state and in a stretched state, the two legs are in an opened and stretched state;

the judgment basis of the lifting and closing state of the two hands is as follows: defining: the numerical value on the X-axis coordinate is small at the left and large at the right, and the numerical value on the Y-axis coordinate is small at the top and large at the bottom; if the angle of a connecting line between the left wrist node and the left elbow joint node and the angle of a connecting line between the left elbow joint node and the left shoulder node are larger than a fourth set threshold value; and the angle of a connecting line between the right wrist node and the right elbow joint node and the angle of a connecting line between the right elbow joint node and the right shoulder node are larger than a fourth set threshold value; the X-axis coordinate of the right shoulder node is smaller than that of the right wrist node, the X-axis coordinate of the right wrist node is smaller than or equal to that of the left wrist, and the X-axis coordinate of the left wrist is smaller than that of the left shoulder node; the Y-axis coordinate of the left wrist joint is smaller than the Y-axis coordinate of the left elbow joint node, and the Y-axis coordinate of the left elbow joint node is smaller than the Y-axis coordinate of the left shoulder joint node; the Y-axis coordinate of the right wrist joint is smaller than the Y-axis coordinate of the right elbow joint, and the Y-axis coordinate of the right elbow joint is smaller than the Y-axis coordinate of the right shoulder joint; if the two hands are lifted up and gathered, otherwise, the two hands are not lifted up and gathered;

the preparation action standard needs to reach a state that the human body is over against the camera, a state that the two legs are opened and extended and a state that the two hands are lifted and closed.

Further, the action state types in step 2 include: s0, S1, S2 and unknown operating states, the operations represented by the respective operating states are as follows:

s0: the two legs are in an open state, and the two hands are in a lifted state;

s1: a take-off state and a two-hand open state;

s2: the legs are closed and the hands are vertically downward;

states not belonging to S0, S1 and S2 are all unknown action states;

if the X-axis coordinate of the right shoulder node is smaller than the X-axis coordinate of the right wrist node, the X-axis coordinate of the right wrist node is smaller than or equal to the X-axis coordinate of the left wrist, and the X-axis coordinate of the left wrist is smaller than the X-axis coordinate of the left shoulder node, the wrist is in a closed state; if the X-axis coordinate of the right wrist node is smaller than the X-axis coordinate of the right shoulder node, the X-axis coordinate of the right shoulder node is smaller than the X-axis coordinate of the left shoulder node, and the X-axis coordinate of the left shoulder node is smaller than the X-axis coordinate of the left wrist, the two hands are in an open state;

if the angle between the connecting line of the left wrist node and the left elbow joint node and the angle between the connecting line of the left elbow joint node and the left shoulder node are larger than a third set threshold value; and the angle of a connecting line between the right wrist node and the right elbow joint node and the angle of a connecting line between the right elbow joint node and the right shoulder node are larger than a third set threshold value; and an included angle between a connecting line of the left wrist joint and the left elbow joint and the vertical direction is more than 160 degrees; and an included angle between a connecting line of the right wrist joint and the right elbow joint and the vertical direction is more than 160 degrees; the Y-axis coordinate of the right shoulder node is smaller than that of the right elbow joint node, and the Y-axis coordinate of the left shoulder node is smaller than that of the left elbow joint node, so that the two hands are in a vertically downward state; if not, the two hands are not in a vertically downward state;

a take-off state judgment mode: taking the Y-axis coordinate value of the left shoulder node as Y1, the Y-axis coordinate value of the left hip node as Y2 and delta as the value obtained by subtracting Y1 from Y2 when the action preparation standard is used; if the delta/4 added to the Y-axis coordinate of the current left hip node is smaller than Y2; and the delta/4 added to the Y-axis coordinate of the current right hip node is less than Y2; and the Y-axis coordinate plus delta/4 of the left ankle joint node is smaller than the Y-axis coordinate of the left ankle joint node in the S0 state; and the delta/4 added to the Y-axis coordinate of the right ankle joint node is smaller than the Y-axis coordinate of the right ankle joint node in the S0 state; the state is a take-off state; if not, the state is the non-take-off state.

Further, the duration of the action state in step 2 is specifically:

action specification exception-counter: in the state that the two hands are not straightened in S0, S1 and S2, the action specification is abnormal, namely a counter is added with 1; the two hands are not straightened, and the angle between a connecting line of the left wrist node and the left elbow joint node and the angle between the connecting line of the left elbow joint node and the left shoulder node are smaller than or equal to a third set threshold value; or the angle between the connecting line of the right wrist joint and the right elbow joint node and the connecting line of the right elbow joint node and the right shoulder node is less than or equal to a third set threshold value;

action specification exception two counter: when the motion specification abnormal two counters are in S0, the angle of a connecting line of the left knee joint node and the left hip node and the angle of a connecting line of the right knee joint node and the right hip node are smaller than or equal to a second set threshold, and the number of the motion specification abnormal two counters is increased by 1;

action specification exception three counters: when the robot is in the S0 state, the two hands are not lifted and closed, and 1 is added to the three counters with abnormal action specifications;

action specification exception four counter: when the abnormal four-counter is in the S1 state, the abnormal four-counter of the action specification is increased by 1;

action specification exception five counter: when the angle between the connecting line of the left knee joint node and the left hip node and the angle between the connecting line of the right knee joint node and the right hip node are larger than or equal to a third set threshold value, adding 1 to the action specification abnormity five counter;

the state counter Count: representing the current action state, wherein the times collected when the current state occurs are used for calculating the duration of the state;

unknown state counter Cnone: and indicating that the current action state is an unknown state, and calculating the duration of the state by using the acquired unknown action times when the current state occurs.

Further, the step 3 is further specifically:

acquiring continuous pictures which finish one-time opening and closing jump,

a. if the picture sequence is: s0, S1, S2 and S1 show that the leg lifting data is valid, and the step d is entered;

b. if the picture order is: s0, S1 and S0 indicate that S2 is not available, and the voice broadcast indicates that the hand is down when the two legs are not closed;

c. if the picture sequence is: s2, S1 and S2 indicate that S0 is not available, and voice broadcast is performed to open legs when hands are not lifted;

d. if the value of a counter is larger than or equal to a set threshold value one when the action specification abnormity occurs in S0, S1 and S2, voice broadcasting 'the hands are not straightened';

e. if the value of the second counter with abnormal motion specification is larger than or equal to the second set threshold value in S0, voice broadcasting 'two legs are not opened';

f. if the value of the three counters with abnormal action specification is larger than or equal to the set threshold value three in S0, voice broadcasting that the two hands are not lifted and gathered;

g. if the value of the four counters with abnormal action specifications is larger than or equal to the set threshold value four in S1, voice broadcasting 'no jump';

h. if the value of the five counter with abnormal action specification is greater than or equal to the set threshold value five in S2, voice broadcasting that the two legs are not closed;

and when the b, c, d, e, f, g and h are not abnormal, the data is valid, the count is increased by 1, and the added number is broadcasted.

In a second aspect, the present invention provides a system for counting open-close jump tests based on a vision technology, including:

the preparation testing module is used for judging whether the testing personnel meet the preparation action standard or not according to the obtained preparation picture, if so, starting testing counting, and entering the obtaining and rejecting module; if not, prompting;

the obtaining and removing module is used for obtaining all the pictures according to the time sequence, judging the action state type in each picture and the keeping time length in the action state, deleting the picture if the keeping time length of the action state is less than a set threshold value, and entering the judging and counting module if the keeping time length of the action state is not less than the set threshold value;

the judging and counting module is used for judging whether the front picture and the rear picture are in accordance with one opening and closing jump, and if the front picture and the rear picture are in accordance with the opening and closing jump, counting is carried out and 1 is added; if not, not counting;

the counting completion module is used for repeatedly acquiring the rejection module and the judging and counting module until the test is completed, and then displaying the result;

the preparatory action criteria in the preparatory test module include:

the judgment basis of the two-leg opening and straightening state is as follows: when the included angle between the connecting line of the left hip node and the left knee joint node and the connecting line of the left knee joint node and the left ankle joint node is larger than a first set threshold value, and the included angle between the connecting line of the right hip node and the right knee joint node and the connecting line of the right knee joint node and the right ankle joint node is larger than the first set threshold value, the state is that the two legs are in a straight state, otherwise, the state is that the two legs are not in a straight state; when the angle of a connecting line between the left knee joint node and the left hip node and the angle of a connecting line between the right knee joint node and the right hip node are larger than a second set threshold value, the two legs are in an open state; when the angle of a connecting line between the left knee joint node and the left hip node and the angle of a connecting line between the right knee joint node and the right hip node are smaller than a third set threshold value, the two legs are in a close state; only when the two legs are in the open state and the two legs are in the extended state, the two legs are in the open extended state;

the judgment basis of the lifting and closing state of the two hands is as follows: defining: the numerical value on the X-axis coordinate is small at the left and large at the right, and the numerical value on the Y-axis coordinate is small at the top and large at the bottom; if the angle between the connecting line of the left wrist node and the left elbow joint node and the angle between the connecting line of the left elbow joint node and the left shoulder node are larger than a fourth set threshold value; and the angle of a connecting line between the right wrist node and the right elbow joint node and the angle of a connecting line between the right elbow joint node and the right shoulder node are larger than a fourth set threshold value; the X-axis coordinate of the right shoulder node is smaller than that of the right wrist node, the X-axis coordinate of the right wrist node is smaller than or equal to that of the left wrist, and the X-axis coordinate of the left wrist is smaller than that of the left shoulder node; the Y-axis coordinate of the left wrist joint is smaller than that of the left elbow joint, and the Y-axis coordinate of the left elbow joint is smaller than that of the left shoulder joint; the Y-axis coordinate of the right wrist joint is smaller than the Y-axis coordinate of the right elbow joint, and the Y-axis coordinate of the right elbow joint is smaller than the Y-axis coordinate of the right shoulder joint; if the two hands are lifted up and gathered, otherwise, the two hands are not lifted up and gathered;

the preparation action standard needs to reach a state that the human body is right facing the camera, a state that the two legs are opened and extended and a state that the two hands are lifted and closed.

Further, the obtaining the action state type in the culling module includes: s0, S1, S2 and unknown operating states, the operations represented by the respective operating states are as follows:

s0: the two legs are in an open state, and the two hands are in a lifted state;

s1: a take-off state and a two-hand opening state;

s2: the legs are closed and the hands are vertically downward;

states not belonging to S0, S1 and S2 are all unknown action states;

a take-off state judgment mode: taking a Y-axis coordinate value of a left shoulder node as Y1, a Y-axis coordinate value of a left hip node as Y2 and delta as a value obtained by subtracting Y1 from Y2 when the action standard is prepared; if the delta/4 added to the Y-axis coordinate of the current left hip node is smaller than Y2; and the Y-axis coordinate plus delta/4 of the current right hip node is smaller than Y2; and delta/4 added to the Y-axis coordinate of the left ankle joint node is smaller than the Y-axis coordinate of the left ankle joint node in the S0 state; and the delta/4 added to the Y-axis coordinate of the right ankle joint node is smaller than the Y-axis coordinate of the right ankle joint node in the S0 state; the state is a take-off state; otherwise, the state is the non-jump state.

Further, the obtaining of the duration of the action state in the removing module specifically includes:

action specification exception-counter: in the state that the two hands are not straightened in S0, S1 and S2, the action specification is abnormal, namely a counter is added with 1; the two hands are not straightened, and the angle between a connecting line of the left wrist node and the left elbow joint node and a connecting line of the left elbow joint node and the left shoulder node is smaller than or equal to a third set threshold value; or the angle between the connecting line of the right wrist joint and the right elbow joint node and the connecting line of the right elbow joint node and the right shoulder node is less than or equal to a third set threshold value;

action specification exception four counter: when the abnormal state is in the S1 state, the abnormal four counter of the action specification is increased by 1;

action specification exception five counter: when the angle between the connecting line of the left knee joint node and the left hip node and the angle between the connecting line of the right knee joint node and the right hip node are larger than or equal to a third set threshold value, adding 1 to the five abnormal counters of the action specifications;

unknown state counter Cnone: and the unknown action times collected when the current action state is an unknown state are used for calculating the duration of the state.

Further, the judging and counting module is further specifically:

acquiring continuous pictures which finish one-time opening and closing jumping,

a. if the picture order is: s0, S1, S2 and S1 show that the leg lifting data is valid, and the step d is entered;

b. if the picture sequence is: s0, S1 and S0 indicate that S2 is not available, and the voice broadcast indicates that the hand is dropped when the legs are not closed;

c. if the picture sequence is: s2, S1 and S2 indicate that S0 is not available, and the voice broadcast indicates that the legs are opened when the hands are not lifted;

h. if the value of the five counter with abnormal motion specification is greater than or equal to the set threshold value five in S2, voice broadcasting that the two legs are not closed;

In a third aspect, the invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the method of the first aspect.

One or more technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages:

the embodiment of the application provides a method, a system and a medium for counting the opening and closing jumps of a computer, wherein the method, the system and the medium are used for recognizing posture images through a computer vision technology, digitizing actions and time to realize the counting of the opening and closing jumps of the computer, digitizing the time and the actions, effectively realizing the judgment and counting of the opening and closing jumps and realizing the voice prompt of nonstandard actions;

the phenomenon of sudden position deviation of 'key joint points of a human body' exists when the posture image is recognized by the computer vision technology, and the problem can be effectively solved by filtering the acquired action data and removing invalid data.

The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.

Drawings

The invention will be further described with reference to the following examples and figures.

FIG. 1 is a flow chart of the method of the present invention;

FIG. 2 is a schematic view of a human body node according to the present invention;

FIG. 3 is a schematic diagram of a FIFO data save queue according to the present invention;

FIG. 4 is a diagram of a data save queue according to the present invention;

FIG. 5 is a schematic diagram of a data retention unit according to the present invention;

FIG. 6 is a schematic view of data acquisition according to the present invention;

FIG. 7 is a flow chart of the operation criteria determination of the present invention.

Detailed Description

The technical scheme in the embodiment of the application has the following general idea:

the invention provides a new method for realizing open-close jump test counting by a method of recognizing posture images and digitizing actions and time through a computer vision technology, digitalizes the time and the actions, can effectively judge and count open-close jumps and realize voice prompt on nonstandard actions;

when the posture image is identified by the computer vision technology, the phenomenon of sudden position deviation of a human body key joint point exists, after test data are collected, deviated unreasonable data are removed according to the actual situation of action, drifting abnormal action data (the collected data are short in time and do not accord with actual movement) are removed, and the problem of the position drift of the human body key joint point is solved by filtering the unreasonable abnormal action data;

as shown in fig. 1, the method specifically includes the following steps:

test preparation:

the voice prompts the tester to carry out test preparation according to the opening and closing jumping standard; the judgment of the action preparation state is realized by judging the angle of the key joint point of the human body, and the preparation action of opening and closing jump is taken as: if the camera is over against, the legs are opened and straightened, the hands are lifted and folded up, if the camera is not in line with the preparation action, voice prompts of ' please over the camera ', ' the legs are not opened and straightened ', the hands are not lifted and folded up ', and the like are prompted, and if the camera is in line with the preparation action standard, an ' opening and closing jump starting test ' is reported.

Collecting test data:

the open-close jump action is decomposed, the collected times are recorded according to each action, the action and the times are sequentially added into an FIFO (first in first out) annular buffer area, the action and the time are digitized, so that the duration of the corresponding action can be known according to the picture processing speed (fps) of a camera and the times of a certain action, and the functions of judging the standardization of the action and realizing the open-close jump counting are achieved.

And (3) acquiring and filtering action data:

when the posture image is identified by the computer vision technology, the phenomenon of sudden position deviation of key joint points of a human body exists, after test data are collected, the deviated unreasonable data are removed according to the actual situation of the movement, and abnormal movement data (the collected time is too short and does not accord with the data of the actual movement) which drifts are removed.

Judging an action standard:

judging whether the motion data acquired after filtering the 'floating' data meets the opening and closing jump standard, if not, counting and carrying out corresponding voice prompt and not counting (for example, a voice message that 'two legs are not opened', 'two hands are not lifted and disturbed', 'two hands are not stretched laterally', 'no jump', 'two legs are not closed', and the like) if not, and if so, carrying out voice broadcast on the added numbers according to a standard counter + 1;

the specific implementation mode is as follows:

1. description of key joint points of the human body:

firstly, the posture image is recognized through a computer vision technology, 18 human key joint points of the human body are recognized, and as shown in fig. 2: respectively as follows: nose 0, neck 1, right shoulder 2, right elbow 3, right wrist 4, left shoulder 5, left elbow 6, left wrist 7, right hip 8, right knee 9, right ankle 10, left hip 11, left knee 12, left ankle 13, right eye 14, left eye 15, right ear 16, left ear 17;

2. test preparation:

a. The human body is opposite to the camera: the judgment basis is as follows: taking 4 position points (A left eye, B right eye, C left ear and D right ear), and when the x coordinate of D < the x coordinate of B < the x coordinate of A < the x coordinate of C, indicating that the human body is just opposite to the camera;

b. opening and straightening two legs: the judgment basis is as follows: "straight legs" and "open angle of legs >25 °;

straightening two legs: taking 3 position points (A: buttocks, B: knee joint, C: ankle joint), and when the included angle of the three points ABC at the angle is more than 160 degrees (the ideal extension is 180 degrees, but the actually taken point cannot be 180 degrees), representing the extension state of the two legs;

opening angle of both legs: taking 3 position points (A: left knee joint, B: hip and C: right knee joint), wherein the included angle ABC represents the opening angle of the two legs;

c. lifting and closing the hands, wherein the judgment basis is as follows: the hands are straightened, and wrist nodes of the hands are above shoulders and in the horizontal positions of the shoulders;

straightening two hands: taking 3 position points (A: wrist, B: elbow joint, C: shoulder), this included angle ABC >160 indicates that the hands are straight;

both hands wrist node is above the both shoulders and in the both shoulders horizontal position: the judgment basis is as follows: taking 4 location points (A right shoulder, B right wrist, C left wrist, D left shoulder), when the y coordinate of B is above the y coordinate of A (y of B < y of A) indicates that the hand is above the shoulder; when (x-coordinate of a < x-coordinate of B < = x-coordinate of C < x-coordinate of D) indicates that the double wrist node is within the shoulder horizontal position;

3. collecting test data:

and (3) data storage: when data is stored, the FIFO circular buffer area is used for sequentially entering a queue, and when data is fetched for analysis, the data is sequentially obtained from the tail of the queue;

the open-close jump action is decomposed, the acquired times are recorded according to each action, the action and the times are sequentially added into the FIFO circular buffer area, the action and the time are digitized, so that the time length of the corresponding action can be known according to the image processing speed (fps) of a camera and the times of a certain action, and the functions of judging the standardization of the action and realizing the open-close jump counting are achieved.

As shown in fig. 3, the data FIFO ring buffer includes a data holding unit 1, a data holding unit 2, a data holding unit 3, a data holding unit 4, a data holding unit 5, a data holding unit 6, and a data holding unit n;

3. 1 data save queue description (FIFO ring buffer and enqueue action):

when the application is initialized, a ring queue buffer area with the size of n is distributed; when a new data storage unit is enqueued at each subsequent time, if the queue is not full, the new data storage unit is directly enqueued, and if the queue is full, the data which is enqueued at the earliest time in the existing cache area is replaced;

as shown in fig. 4, a camera inputs an image, a human body key node is counted, and a state type Sn (or a position state Snone) of the image is obtained; comparing the state type Sm in the current data storage unit; if Sn is the same as or is unknown, the current action is directly calculated into Sm, and if Sn is different from Sm, the Sm is indicated to have a new action, and is stored into a ring-shaped queue, and a new data storage unit is created to store Sn and subsequent data;

the data saving unit enqueue action triggers: performing visual technical processing on pictures acquired by a camera every time, and performing human body key joint point technical processing according to the opening and closing jump motion specification; if the acquired 'action state type' does not change or is an unknown action, directly carrying out statistical calculation in the current data storage unit; if the acquired action state type changes (for example, when the action state type changes from S0 to non-S0), the existing data storage unit is put into the fifo ring queue, and a data storage unit is newly built to store the currently acquired action and the subsequent action;

3.2 data storage Unit and data acquisition description (digitizing actions and time):

as shown in figures 5 and 6 of the drawings,

a. counting the abnormal 1 count of the movement specification, namely counting 1 if the two hands are not straightened, wherein the movement is taken as S0/S1/S2, and the two hands are not straightened;

b. action specification exception 2 count: if the opening angle of the two legs does not meet the requirement of more than 25 in S0 (lifting the two hands), adding 1 to the count of 2 in the abnormal action specification;

c. action specification exception 3 count: if the horizontal positions of the two hands are not in the shoulders when the two hands are lifted at S0 (the opening angle of the two legs is more than 25 degrees), adding 1 to the number of the abnormal 3 counts of the action specification;

d. action specification exception 4 count: if the robot does not take off in S1 (half of the two hands lift from bottom to top), adding 1 to the 4 counts of the abnormal motion specification;

e. action specification exception 5 count: if the two legs are not closed (the opening angle of the two legs is less than 10 degrees) in S2 (the two hands drop), adding 1 to the abnormal 5 counts of the action specification;

f. the action state type: S0/S1/S2, the operation represented by each operation state is as follows:

s0, opening the two legs, lifting the two hands to be closed, wherein the opening angle of the two legs is more than 25 degrees, and the two hands are lifted to be closed;

s1, (jumping, opening two sides of two hands), and having a state of jumping and opening two sides of two hands;

s2, closing the two legs, namely, enabling the two hands to be vertically downward, closing the two legs (the opening angle of the two legs is less than 10 degrees), and enabling the two hands to be vertically downward;

the decision rules therein state:

lifting with two hands: taking 4 position points (A right shoulder, B right wrist, C left wrist and D left shoulder), when the y coordinates of B and C are above the y coordinates of A and D (y of B/C < y of A/D), indicating that the hand is above the shoulder;

the hands droop: an included angle between a connecting line of the left wrist joint and the left elbow joint and the vertical direction is more than 160 degrees; and an included angle between a connecting line of the right wrist joint and the right elbow joint and the vertical direction is more than 160 degrees; the Y-axis coordinate of the right shoulder node is smaller than that of the right elbow joint node, and the Y-axis coordinate of the left shoulder node is smaller than that of the left elbow joint node, so that the two hands droop;

the horizontal position of the hands (not) is in the shoulders: taking 4 position points (A right shoulder, B right wrist, C left wrist and D left shoulder), when (X coordinate of A < X coordinate of B < = X coordinate of C < X coordinate of D), indicating that the double-wrist node is in the horizontal position of the double shoulders

Taking delta in the S0 state as the height difference of Y coordinates from the shoulder to the hip, and when the hip joint point and the ankle joint point are increased by delta/4 compared with the Y coordinates in the S0 state, the jump is shown;

the two legs are not closed: the opening angle of the two legs is more than 25 degrees, the closing is indicated, and the non-matching indicates that the two legs are not closed;

lifting and closing the hands: straight hands and with wrist joints above and in horizontal position "

g. The state counter Count represents the current action state and the collected times (duration) when the state occurs;

h. cnone represents the current action state, the number of times of unknown actions (used for calculating the time length) collected when the current state occurs;

if the camera is capturing at a rate of 30fps (in practice, a high-speed mode such as 60fps is generally used), 30 data states should be stored in each second, if the data that we need to acquire is the last 4 seconds, and 120 data states are needed, the data storage units are sequentially retrieved from the tail of the current fifo queue, and when the sum of Count and Cnone values in each unit reaches 120, the action data occurring in the last 4 seconds is represented.

4. Action data acquisition filtering (solving the problem of "drift"):

The concrete expression is as follows: (an example of actual usage), the data acquisition part of step 3, in the opening and closing jump of the S1 state, may actually acquire data at a rate of 30fps even if the motion remains the S1 state for 1 second (due to interference caused by hands and other reasons): s1 action 10 times- > S0 action 2 times- > S2 action 1 time- > S1 action 7 times- > S0 action 2 times- > S1 action 8 times, and the things realized by the step are: filtering out S0/S2 data in the data state according to time, and only keeping effective S1 state data;

the realization method comprises the following steps: if the keeping time of a state action does not reach 100ms, filtering out the corresponding action; according to the sampling rate of 30fps, the number of times of 100ms is more than 3, so that the problem of abnormal motion 'floating' can be solved by filtering data with the number of times of Count < =3 in a data cache region;

5. judging an action standard:

as shown in fig. 7, it is determined whether the motion data obtained after filtering the data of "floating" meets the open-close jump standard, if not, the motion data is not counted and corresponding voice prompt is performed (e.g., "both hands are not extended", "both legs are not opened", "both hands are not lifted and disturbed", "no jump", "both legs are not closed", etc.) and if, the motion data meets the standard counter +1 and the added numbers are broadcasted in voice;

starting from the start S0 if both are normal actions, the forward enqueue order should be as follows:

S0 S1 S2 S1 S0 S1 S2 S1 S0 S1 S2 S1 S0...

data are sequentially obtained from the tail of the queue during data acquisition analysis, analysis and counting are carried out when the current S1 action state is 100ms (Count is 3), and historical data which do not contain the current S1 and are from back to front according to time are as follows:

a. if the history data from back to front is in accordance with S0S 1S 2S 1: the leg lifting data is valid (left/right leg lifting), and the step d is entered;

b. if the history data from back to front is: S0S 1S 0, no S2 is indicated, and the voice broadcast is 'not closing the two legs and hanging down the hands';

c. if the history data from back to front is: S2S 1S 2, no S0 exists, and voice broadcasting is performed to 'do not raise both hands to open legs';

d. if the ' action specification abnormal 1 counting ' > = 1/3 of the corresponding Count ' occurs in S0/S1/S2, the ' hands are not straightened ' is broadcasted through voice;

e. if in S0, "action specification exception 2 Count" > = 1/3 of the corresponding Count, the voice broadcast "both legs are not opened";

f. if in S0, the action specification abnormity Count of 3 occurs = 1/3 of the corresponding Count, then the voice broadcast of 'two hands are not lifted and gathered together';

g. if in S1, the 'action specification abnormal 4 Count' > = 1/3 of the corresponding Count, then the voice broadcast 'no jump';

h. if the 'action specification abnormity 5 Count' > = 1/3 of the corresponding Count in S2, the voice broadcasting 'two legs are not gathered together';

when the b, c, d, e, f, g and h are not abnormal, the data is valid, and the counter +1 broadcasts the added numbers.

Example one

The embodiment provides a method for counting open-close jump tests based on a vision technology, which comprises the following steps:

step 4, repeating the step 2 and the step 3 until the test is finished, and then displaying the result;

the preparation action criteria in step 1 include:

the judgment basis of the lifting and closing state of the two hands is as follows: defining: the numerical value on the X-axis coordinate is small at the left and large at the right, and the numerical value on the Y-axis coordinate is small at the top and large at the bottom; if the angle between the connecting line of the left wrist node and the left elbow joint node and the angle between the connecting line of the left elbow joint node and the left shoulder node are larger than a fourth set threshold value; and the angle of a connecting line between the right wrist node and the right elbow joint node and the angle of a connecting line between the right elbow joint node and the right shoulder node are larger than a fourth set threshold value; the X-axis coordinate of the right shoulder node is smaller than that of the right wrist node, the X-axis coordinate of the right wrist node is smaller than or equal to that of the left wrist, and the X-axis coordinate of the left wrist is smaller than that of the left shoulder node; the Y-axis coordinate of the left wrist joint is smaller than that of the left elbow joint, and the Y-axis coordinate of the left elbow joint is smaller than that of the left shoulder joint; the Y-axis coordinate of the right wrist joint is smaller than the Y-axis coordinate of the right elbow joint, and the Y-axis coordinate of the right elbow joint is smaller than the Y-axis coordinate of the right shoulder joint; the two hands are lifted and gathered, otherwise, the two hands are not lifted and gathered;

In another preferred embodiment, the action state types in step 2 include: s0, S1, S2 and unknown operating states, the operations represented by the respective operating states are as follows:

s0: the two legs are in an open state, and the two hands are in a lifted state;

s1: a take-off state and a two-hand open state;

s2: the two legs are in a close state and the two hands are in a vertically downward state;

states not belonging to S0, S1 and S2 are unknown action states;

if the angle between the connecting line of the left wrist node and the left elbow joint node and the angle between the connecting line of the left elbow joint node and the left shoulder node are larger than a third set threshold value; and the angle of a connecting line between the right wrist node and the right elbow joint node and the angle of a connecting line between the right elbow joint node and the right shoulder node are larger than a third set threshold value; and an included angle between a connecting line of the left wrist joint and the left elbow joint and the vertical direction is more than 160 degrees; and an included angle between a connecting line of the right wrist joint and the right elbow joint and the vertical direction is more than 160 degrees; the Y-axis coordinate of the right shoulder node is smaller than that of the right elbow joint node, and the Y-axis coordinate of the left shoulder node is smaller than that of the left elbow joint node, so that the two hands are in a vertically downward state; if not, the state is not that the two hands are vertically downward;

In another preferred embodiment, the duration of the action state in step 2 is specifically:

In another preferred embodiment, the step 3 is further specifically:

c. if the picture order is: s2, S1 and S2 indicate that S0 is not available, and the voice broadcast indicates that the legs are opened when the hands are not lifted;

f. if the value of the three counters with abnormal action specification is larger than or equal to the set threshold value three in S0, voice broadcasting is performed, and the two hands are not lifted and gathered;

Based on the same inventive concept, the application also provides a system corresponding to the method in the first embodiment, which is detailed in the second embodiment.

Example two

In this embodiment, a system for counting open-close jump tests based on a vision technology is provided, which includes:

the preparation testing module judges whether a tester accords with the preparation action standard according to the acquired preparation picture, if so, the test counting is started, and the test counting enters the acquisition and rejection module; if not, prompting is carried out;

the counting completion module is used for repeatedly acquiring the rejection module and the counting judgment module until the test is completed, and then displaying a result;

the preparatory action criteria in the preparatory test module include:

the judgment basis of the lifting and closing state of the two hands is as follows: defining: the numerical value on the X-axis coordinate is small at the left and large at the right, and the numerical value on the Y-axis coordinate is small at the top and large at the bottom; if the angle between the connecting line of the left wrist node and the left elbow joint node and the angle between the connecting line of the left elbow joint node and the left shoulder node are larger than a fourth set threshold value; and the angle of a connecting line between the right wrist node and the right elbow joint node and the angle of a connecting line between the right elbow joint node and the right shoulder node are larger than a fourth set threshold value; the X-axis coordinate of the right shoulder node is smaller than that of the right wrist node, the X-axis coordinate of the right wrist node is smaller than or equal to that of the left wrist, and the X-axis coordinate of the left wrist is smaller than that of the left shoulder node; the Y-axis coordinate of the left wrist joint is smaller than the Y-axis coordinate of the left elbow joint node, and the Y-axis coordinate of the left elbow joint node is smaller than the Y-axis coordinate of the left shoulder joint node; the Y-axis coordinate of the right wrist joint is smaller than the Y-axis coordinate of the right elbow joint, and the Y-axis coordinate of the right elbow joint is smaller than the Y-axis coordinate of the right shoulder joint; the two hands are lifted and gathered, otherwise, the two hands are not lifted and gathered;

In another preferred embodiment, the obtaining the action state type in the culling module includes: s0, S1, S2 and unknown operating states, the operations represented by the respective operating states are as follows:

s0: the two legs are in an open state, and the two hands are in a lifted and folded state;

s1: a take-off state and a two-hand open state;

s2: the legs are closed and the hands are vertically downward;

states not belonging to S0, S1 and S2 are unknown action states;

if the angle of a connecting line between the left wrist node and the left elbow joint node and the angle of a connecting line between the left elbow joint node and the left shoulder node are larger than a third set threshold value; and the angle of a connecting line between the right wrist node and the right elbow joint node and the angle of a connecting line between the right elbow joint node and the right shoulder node are larger than a third set threshold value; and an included angle between a connecting line of the left wrist joint and the left elbow joint and the vertical direction is more than 160 degrees; and an included angle between a connecting line of the right wrist node and the right elbow joint node and the vertical direction is more than 160 degrees; the Y-axis coordinate of the right shoulder node is smaller than that of the right elbow joint node, and the Y-axis coordinate of the left shoulder node is smaller than that of the left elbow joint node, so that the two hands are in a vertically downward state; if not, the two hands are not in a vertically downward state;

a take-off state judgment mode: taking the Y-axis coordinate value of the left shoulder node as Y1, the Y-axis coordinate value of the left hip node as Y2 and delta as the value obtained by subtracting Y1 from Y2 when the action preparation standard is used; if the delta/4 added to the Y-axis coordinate of the current left hip node is smaller than Y2; and the Y-axis coordinate plus delta/4 of the current right hip node is smaller than Y2; and delta/4 added to the Y-axis coordinate of the left ankle joint node is smaller than the Y-axis coordinate of the left ankle joint node in the S0 state; and the delta/4 added to the Y-axis coordinate of the right ankle joint node is smaller than the Y-axis coordinate of the right ankle joint node in the S0 state; the state is a take-off state; otherwise, the state is the non-jump state.

In another preferred embodiment, the obtaining of the duration of the action state in the culling module specifically includes:

In another preferred embodiment, the judging and counting module further specifically includes:

a. if the picture order is: s0, S1, S2 and S1 indicate that the leg lifting data is valid, and the step d is entered;

Since the system described in the second embodiment of the present invention is a system used for implementing the method of the first embodiment of the present invention, based on the method described in the first embodiment of the present invention, a person skilled in the art can understand the specific structure and the deformation of the system, and thus the detailed description is omitted here. All systems adopted by the method of the first embodiment of the present invention belong to the protection scope of the present invention.

Based on the same inventive concept, the application provides a storage medium corresponding to the fourth embodiment, which is described in detail in the fourth embodiment.

EXAMPLE III

The present embodiment provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, any one of the first embodiment can be implemented.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

Although specific embodiments of the invention have been described above, it will be understood by those skilled in the art that the specific embodiments described are illustrative only and are not limiting upon the scope of the invention, and that equivalent modifications and variations can be made by those skilled in the art without departing from the spirit of the invention, which is to be limited only by the appended claims.

Claims

1. A method for testing and counting open and close jumps based on a visual technology is characterized in that: the method comprises the following steps:

step 2, acquiring all pictures according to the time sequence, judging the action state type in each picture and the holding time of the action state, if the holding time of the action state is less than a set threshold value, deleting the picture, otherwise, entering step 3; the action state types include: s0, S1, S2 and unknown operating states, the operations represented by the respective operating states are as follows:

s0: the two legs are in an open state, and the two hands are in a lifted state;

s1: a take-off state and a two-hand opening state;

s2: the legs are closed and the hands are vertically downward;

states not belonging to S0, S1 and S2 are all unknown action states;

the preparation action criteria in step 1 include:

the judgment basis of the lifting and closing state of the two hands is as follows: defining: the numerical value on the X-axis coordinate is small at the left and large at the right, and the numerical value on the Y-axis coordinate is small at the top and large at the bottom; if the angle of a connecting line between the left wrist node and the left elbow joint node and the angle of a connecting line between the left elbow joint node and the left shoulder node are larger than a fourth set threshold value; and the angle of a connecting line between the right wrist node and the right elbow joint node and the angle of a connecting line between the right elbow joint node and the right shoulder node are larger than a fourth set threshold value; the X-axis coordinate of the right shoulder node is smaller than that of the right wrist node, the X-axis coordinate of the right wrist node is smaller than or equal to that of the left wrist, and the X-axis coordinate of the left wrist is smaller than that of the left shoulder node; the Y-axis coordinate of the left wrist joint is smaller than that of the left elbow joint, and the Y-axis coordinate of the left elbow joint is smaller than that of the left shoulder joint; the Y-axis coordinate of the right wrist joint is smaller than the Y-axis coordinate of the right elbow joint, and the Y-axis coordinate of the right elbow joint is smaller than the Y-axis coordinate of the right shoulder joint; if the two hands are lifted up and gathered, otherwise, the two hands are not lifted up and gathered;

2. The method for counting the open-close jump tests based on the visual technology as claimed in claim 1, characterized in that: the step 2 further comprises:

if the angle between the connecting line of the left wrist node and the left elbow joint node and the angle between the connecting line of the left elbow joint node and the left shoulder node are larger than a third set threshold value; and the angle of a connecting line between the right wrist node and the right elbow joint node and the angle of a connecting line between the right elbow joint node and the right shoulder node are larger than a third set threshold value; and an included angle between a connecting line of the left wrist joint and the left elbow joint and the vertical direction is more than 160 degrees; and an included angle between a connecting line of the right wrist node and the right elbow joint node and the vertical direction is more than 160 degrees; the Y-axis coordinate of the right shoulder node is smaller than that of the right elbow joint node, and the Y-axis coordinate of the left shoulder node is smaller than that of the left elbow joint node, so that the two hands are in a vertically downward state; if not, the two hands are not in a vertically downward state;

a take-off state judgment mode: taking a Y-axis coordinate value of a left shoulder node as Y1, a Y-axis coordinate value of a left hip node as Y2 and delta as a value obtained by subtracting Y1 from Y2 when the action standard is prepared; if the delta/4 added to the Y-axis coordinate of the current left hip node is smaller than Y2; and the delta/4 added to the Y-axis coordinate of the current right hip node is less than Y2; and delta/4 added to the Y-axis coordinate of the left ankle joint node is smaller than the Y-axis coordinate of the left ankle joint node in the S0 state; and the delta/4 added to the Y-axis coordinate of the right ankle joint node is smaller than the Y-axis coordinate of the right ankle joint node in the S0 state; the state is a take-off state; otherwise, the state is the non-jump state.

3. The method for counting the open-close jump tests based on the visual technology as claimed in claim 2, characterized in that: the action state holding duration in the step 2 specifically includes:

action specification exception four counters: when the abnormal state is in the S1 state, the abnormal four counter of the action specification is increased by 1;

4. The open-close jump test counting method based on the visual technology as claimed in claim 3, characterized in that: the step 3 is further specifically as follows:

a. if the picture sequence is: s0, S1, S2 and S1 indicate that the leg lifting data is valid, and the step d is entered;

e. if the value of the second counter with abnormal action specification is larger than or equal to a set threshold value two in S0, voice broadcasting that 'both legs are not opened';

g. if the value of the four counters with abnormal action specifications is larger than or equal to a set threshold value four in S1, voice broadcasting 'no jump';

5. The utility model provides a jump test counting system opens and shuts based on vision technique which characterized in that: the method comprises the following steps:

the obtaining and removing module is used for obtaining all the pictures according to the time sequence, judging the action state type in each picture and the keeping time length in the action state, deleting the picture if the keeping time length of the action state is less than a set threshold value, and entering the judging and counting module if the keeping time length of the action state is not less than the set threshold value; the action state types include: s0, S1, S2 and unknown operating states, the operations represented by the respective operating states are as follows:

s0: the two legs are in an open state, and the two hands are in a lifted state;

s1: a take-off state and a two-hand open state;

states not belonging to S0, S1 and S2 are unknown action states;

the preparatory action criteria in the preparatory test module include:

the judgment basis of the lifting and closing state of the two hands is as follows: defining: the numerical value on the X-axis coordinate is small at the left and large at the right, and the numerical value on the Y-axis coordinate is small at the top and large at the bottom; if the angle of a connecting line between the left wrist node and the left elbow joint node and the angle of a connecting line between the left elbow joint node and the left shoulder node are larger than a fourth set threshold value; and the angle of a connecting line between the right wrist node and the right elbow joint node and the angle of a connecting line between the right elbow joint node and the right shoulder node are larger than a fourth set threshold value; the X-axis coordinate of the right shoulder node is smaller than that of the right wrist node, the X-axis coordinate of the right wrist node is smaller than or equal to that of the left wrist, and the X-axis coordinate of the left wrist is smaller than that of the left shoulder node; the Y-axis coordinate of the left wrist joint is smaller than the Y-axis coordinate of the left elbow joint node, and the Y-axis coordinate of the left elbow joint node is smaller than the Y-axis coordinate of the left shoulder joint node; the Y-axis coordinate of the right wrist joint is smaller than the Y-axis coordinate of the right elbow joint, and the Y-axis coordinate of the right elbow joint is smaller than the Y-axis coordinate of the right shoulder joint; the two hands are lifted and gathered, otherwise, the two hands are not lifted and gathered;

6. The open-close jump test counting system based on the visual technology as claimed in claim 5, characterized in that: the acquisition and rejection module further comprises:

if the angle of a connecting line between the left wrist node and the left elbow joint node and the angle of a connecting line between the left elbow joint node and the left shoulder node are larger than a third set threshold value; and the angle of a connecting line between the right wrist node and the right elbow joint node and the angle of a connecting line between the right elbow joint node and the right shoulder node are larger than a third set threshold value; and an included angle between a connecting line of the left wrist joint and the left elbow joint and the vertical direction is more than 160 degrees; and an included angle between a connecting line of the right wrist joint and the right elbow joint and the vertical direction is more than 160 degrees; the Y-axis coordinate of the right shoulder node is smaller than that of the right elbow joint node, and the Y-axis coordinate of the left shoulder node is smaller than that of the left elbow joint node, so that the two hands are in a vertically downward state; if not, the two hands are not in a vertically downward state;

a take-off state judgment mode: taking the Y-axis coordinate value of the left shoulder node as Y1, the Y-axis coordinate value of the left hip node as Y2 and delta as the value obtained by subtracting Y1 from Y2 when the action preparation standard is used; if the delta/4 added to the Y-axis coordinate of the current left hip node is smaller than Y2; and the delta/4 added to the Y-axis coordinate of the current right hip node is less than Y2; and delta/4 added to the Y-axis coordinate of the left ankle joint node is smaller than the Y-axis coordinate of the left ankle joint node in the S0 state; and the delta/4 added to the Y-axis coordinate of the right ankle joint node is smaller than the Y-axis coordinate of the right ankle joint node in the S0 state; the state is a take-off state; if not, the state is the non-take-off state.

7. The open-close jump test counting system based on the visual technology as claimed in claim 6, characterized in that: the obtaining of the holding time of the action state in the removing module specifically comprises:

action specification exception three counters: when the two hands are not lifted and closed, adding 1 to a three counter with abnormal action specification when the two hands are in an S0 state;

8. The open-close jump test counting system based on the visual technology as claimed in claim 7, wherein: the judging and counting module is further specifically:

acquiring continuous pictures which finish one-time opening and closing jump,

b. if the picture sequence is: s0, S1 and S0 indicate that S2 is not available, and the voice broadcast indicates that the hand is dropped when the two legs are not closed;

9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 4.