CN117746492A - Action scoring and defect analysis method for functional action screening - Google Patents

Abstract

A motion scoring and defect analysis method aiming at functional motion screening belongs to the field of computer vision. Dividing an action instance and sub-actions by a key frame matching algorithm, and then dividing a test action into a simple action and a complex action, and scoring respectively; wherein the simple actions are scored through calculation of parameters such as angle and distance, and the complex actions are scored through a scoring model. Finally, analyzing the action defects, aligning the non-full-score action with the standard action according to the sub-actions, positioning the action defects by replacing the sub-actions and re-scoring, and positioning the defect parts by replacing the body parts and re-scoring. And finally obtaining the score of the FMS and an action defect analysis result. The method is suitable for the field of computer vision, is used for solving the problems of intelligent scoring and defect analysis of the FMS, improves the accuracy of functional action screening, reduces the resource requirement of the functional action screening, and reduces the screening cost.

Description

Action scoring and defect analysis method for functional action screening

Technical Field

The invention relates to a motion scoring and defect analysis method aiming at functional motion screening, belonging to the field of computer vision.

Background

The FMS (FMS, functional Movement Screen) for functional action screening is researched and innovated by gray cook and lee Burton et al, is simple and easy to implement, and consists of seven actions only, and is a set of testing methods used for detecting the overall action control stability, body balance capability, softness, proprioception and other capabilities of the athlete. The FMS is widely applied to professional competitive sports, helps athletes find potential injuries and perform injury prevention training, so that the athletic ability of the athletes is improved, and the athletic life of the athletes is prolonged. During testing, the tester needs to correct the wrong action pattern under the guidance and correction of the commentator or re-execute according to the correct standard. Each test action was completed three times, with the best performance taken as the final result.

However, the popularization and popularity of the FMS are limited by the expertise of the evaluation process, and the testers need to conduct guidance and evaluation by professional panelists to complete the test process and obtain reliable evaluation results, so that the testers cannot independently complete the test. At present, an automatic evaluation method of the FMS often relies on a depth camera or a wearable device to acquire data, the cost of the depth camera is high, and the wearable device can interfere with action execution. In addition, the existing method can only give action scores according to the test actions, can not further analyze the test process, can not obtain necessary guidance and feedback in the test process no matter the tester just touches the FMS or is very familiar with the FMS, and can easily cause the tester to continuously repeat the wrong action mode, so that the test process is worth losing.

Disclosure of Invention

Aiming at the problems of high data acquisition cost, interference with a test process, simple results and the like of the existing action evaluation method, the main purpose of the invention is to provide an action scoring and defect analysis method aiming at functional action screening, which extracts the gesture sequence of a tester from a test video and carries out action decomposition on the test action through a key frame matching algorithm; then, scoring the test action by adopting a complex action scoring model based on a graph rolling network and an LSTM network; and finally obtaining the defect action and the defect part in the test action through a defect action analysis algorithm. The accuracy of functional action screening is improved, the resource requirement of the functional action screening is reduced, the screening cost is reduced, meanwhile, independent and independent test of a tester is realized, and the requirement of the functional action screening is reduced.

The invention aims at realizing the following technical scheme:

the invention discloses an action scoring and defect analysis method aiming at functional action screening, which takes a gesture sequence of a tester as input; performing gesture estimation on the test action by using a gesture estimation model; performing action decomposition on the test action by adopting a key frame matching method to obtain a decomposition action sequence and determining the boundary of an action instance; scoring the complex actions by adopting a complex action scoring model based on a graph rolling network and an LSTM network; and analyzing and positioning the defect actions and defect parts in the action examples by using the FMS defect action analysis method, so that the accuracy of functional action screening is improved, the resource requirement of the functional action screening is reduced, and the screening cost is reduced.

The invention discloses a motion scoring and defect analysis method for functional motion screening, which comprises the following steps:

inputting the video of the testing process into a gesture estimation model in real time to obtain a gesture sequence of the current testing process. And simultaneously, acquiring key frames of standard actions from training data acquired in the earlier stage through a clustering algorithm, namely taking a single gesture frame as a sample, wherein each gesture frame comprises coordinates of all the nodes of the gesture frame, clustering through the clustering algorithm, and then selecting boundary frames of the decomposition actions from a clustering center of the gesture of the standard actions. The selection standard of the boundary frames meets the principle of minimum change for the decomposition action of each action, namely, ensures that the motion trail of each joint point in the same decomposition action is unidirectional. Wherein each complex action includes a number of key frames.

And step two, matching the gesture sequence of the testing process obtained in the step one with the key frame.

And matching the key frames, namely calculating the distances between the current gesture and all standard gestures, wherein the distance calculation method comprises Manhattan distance, euclidean distance and cosine similarity. Thus locating the boundaries of the decomposition actions and dividing the test procedure into sequences of decomposition actions. Wherein each initial action and the next initial action are determined as an action instance, and the boundary of the corresponding decomposition action is determined.

And thirdly, scoring the action by taking the action instance as a unit.

In seven test actions of the FMS, three actions of deep squat, shoulder flexibility and active straight knee leg lifting are divided into simple actions, and four actions of hurdle step, straight bow step, push-up and rotation stability are divided into complex actions. The simple actions directly calculate corresponding data according to the scoring standard and score, and the complex actions are obtained through a complex action scoring model. The complex action model consists of a graph convolution network, an LSTM network and a full-connection network, wherein the graph convolution network is responsible for extracting spatial features of a gesture sequence, the LSTM network is responsible for modeling time features, the LSTM network sends hidden layer output of the last frame into the full-connection network to obtain action scoring when training, and the full-connection network carries out score prediction on hidden layer output of the LSTM at each frame when testing, so that a scoring sequence of testing actions is obtained.

Judging whether the scoring result obtained in the third step reaches full score of the corresponding action, and judging whether the current action belongs to a simple action or a complex action;

and step five, if the judgment result of the step four is yes or the current action example is simple action, outputting the grading result of the current action example, and re-executing the step two to evaluate the next action example.

And step six, if the judgment result of the step four is negative and the current action example is a complex action, carrying out replacement scoring on the action example, and outputting a defect analysis result.

Step 6.1, action alignment: the motion to be scored is subjected to sequence alignment based on the decomposition motion fragments through a dynamic time alignment algorithm and the full-scale motion, wherein the dynamic time alignment uses Euclidean distance to calculate the distance between two gesture frames.

Step 6.2, posture replacement: and in the scoring sequence obtained in the step three, starting from the frame with the score change appearing for the first time, and replacing the frame with the frame of standard action according to the sequence alignment result. Before replacement, the human body gesture in the standard action frame needs to be scaled to the same scale as the replaced gesture through a maximum and minimum distance-based normalization algorithm. The flow of the maximum and minimum value specification algorithm is as follows: firstly, calculating the central point of each human body gesture, namely the average value of all joint point coordinates of the same gesture, then determining the scale of coordinate scaling by calculating a scaling factor, namely the average value of the distances between all joint points and the central point, and finally scaling all coordinates to ensure that the coordinate scale of each gesture is the same.

Step 6.3, action scoring: and (3) re-sending the replaced gesture sequence obtained in the step (6.2) into a complex action scoring model for scoring, and re-executing the step (6.2) and the step (6.3) until a full-score gesture sequence is obtained, wherein the replaced part in the original gesture sequence is the defect action positioned by the algorithm.

Step 6.4, defect part analysis: based on the full-scale replacement gesture sequence obtained in the step 6.3, aiming at the replaced gesture fragments, namely the defect action positioning result, in terms of space, aiming at the defect actions, a defect part analysis algorithm of the defect actions divides the human body joint point into six areas of four limbs, a trunk and a head, and the parts which are more likely to be in error in each area of the body at the moment when the current defect action occurs are ordered for a tester from small to large through the similarity of the joint angles of each area and the standard action. Wherein, the lower the similarity is, the greater the possibility of defects is.

Step 6.5, outputting an analysis report: the action evaluation link realizes intelligent scoring of the test action and accurate positioning of the defect action of the tester in time and space; after the defect analysis result is obtained, formatting the result and prompting a tester in real time, so that the independent and independent test of the tester is realized, and the requirement of functional action screening is reduced.

The beneficial effects are that:

1. the invention discloses a motion scoring and defect analysis method aiming at functional motion screening, which completely provides an overall solution for intelligent evaluation of an FMS (field programmable gate array), and simultaneously provides an intelligent evaluation framework for FMS and other functional tests, wherein the intelligent evaluation framework comprises feature extraction, motion decomposition, motion scoring, defect analysis and the like, and can provide comprehensive, accurate and objective evaluation results for testers.

2. According to the action scoring and defect analysis method for functional action screening, only video data is used as input on the premise of ensuring accuracy and objectivity of results, and no additional sensor equipment or complex hardware setting is needed, so that the cost is low, and meanwhile, unnecessary interference to a test process due to equipment wearing is avoided.

3. The invention discloses an action scoring and defect analysis method aiming at functional action screening, which divides actions into simple actions and complex actions. The simple action score is obtained through angle and distance calculation; the complex action score is obtained through an action scoring network designed based on a graph rolling network and an LSTM network. The method comprehensively considers various factors in the human body movement process, and realizes objective and accurate grading of the FMS action by means of a deep learning algorithm.

4. According to the action scoring and defect analysis method aiming at functional action screening, through intelligent analysis of action videos of testers, not only can defect actions be found in the testing process, but also defect positions of the testers can be positioned and described. The evaluation result is objective and detailed, and the tester can be guided to perform correct action adjustment in real time, so that the FMS independent test is truly realized. By the method, the popularization of FMS can be promoted, and the movement quality and life quality of testers are improved.

Drawings

FIG. 1 is a flow chart of a method of action scoring and defect analysis for functional action screening disclosed herein;

fig. 2 is a schematic diagram of a key frame corresponding to each type of FMS complex actions disclosed in the present embodiment;

FIG. 3 is a schematic diagram of a structure of an FMS complex action scoring model according to the present invention;

fig. 4 is an exemplary diagram of an output result of action evaluation in an actual scenario disclosed in this embodiment;

FIG. 5 is a schematic diagram of an FMS defect action analysis algorithm according to the present invention;

fig. 6 is an exemplary diagram of the result of defect analysis in the actual scenario disclosed in this embodiment.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention. In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.

The embodiment discloses an action scoring and defect analysis method for functional action screening, which is used for solving the problems of intelligent scoring and defect analysis of FMS actions, as shown in FIG. 1, and comprises the following steps:

inputting the video of the testing process into a gesture estimation model in real time to obtain a gesture sequence of the current testing process. And simultaneously, acquiring key frames of standard actions from training data acquired in the earlier stage through a clustering algorithm, namely taking a single gesture frame as a sample, wherein each gesture frame comprises coordinates of all the nodes of the gesture frame, clustering through the clustering algorithm, and then selecting boundary frames of the decomposition actions from a clustering center of the gesture of the standard actions. The selection standard of the boundary frame meets the minimum change principle for the decomposition action of each action, namely, the motion track of each joint point in the same decomposition action is ensured to be unidirectional as much as possible. As shown in fig. 2, where each complex action contains several key frames.

In practical application, the posture estimation model can adopt a main stream open source model such as Alphapose, openPose or a posture estimation model which is pre-trained on a large-scale posture estimation task data set such as COCO and MPII. In the embodiment, an alpha phase model is adopted as a posture estimation model to extract posture characteristics of a tester.

In practical applications, the determination of the action category may be determined by a machine learning model, a deep learning network, or by normalizing the test sequence of the tester during the test to determine the action category of the current gesture sequence. In the embodiment, a random forest algorithm is adopted to conduct action classification on the test frames, a smoothing window is used to conduct smoothing operation on the classification result sequence, semantic segmentation results of the test actions are obtained, and instance segmentation, namely sub-action decomposition, is completed through a key frame matching algorithm.

In practical application, the key frame is a preferable result of the clustering centers of all the actions of the same type, wherein the number of the clustering centers is about 5-15. The present embodiment sets the number of cluster centers to 10.

In practical application, the decomposition action of each action meets the minimum change principle, and in the hurdle step, the decomposition action can be interpreted as: the tester lifts the test foot from the initial standing posture, the tester strides the test foot across the test line, the tester puts down the test foot while the two legs straighten and straddle, the tester lifts the test foot, the tester withdraws the test foot, and the tester puts down the test foot to recover the initial posture; in a straight bow step, the decomposition action can be interpreted as: the tester straddles the test board with the legs, the tester keeps the upper body bending the legs vertically until the knees of the rear side leg contact the test board, and the tester resumes the straddling state of the legs; in push-ups, the decomposition action can be interpreted as: the testers face the ground, the hands of the testers are placed on the two sides of the head and lie on the ground, the testers support the trunk to straighten the two arms, and the testers bend the two arms to recover the initial state, namely lie on the ground; in rotational stability, the decomposition action can be interpreted as: the tester straightens the double-knee kneeling arms towards the ground, the tester straightens the test arms and the test legs simultaneously, the tester contacts the elbows of the test arms with the knees of the test legs, the tester resumes straightening the test arms and the test legs, and the tester resumes straightening the double-knee kneeling arms towards the ground.

In practical applications, the FMS contains 7 test actions in total, namely squat, hurdle step, shoulder flexibility, straight bow step, active straight knee leg lifting, push-up and rotational stability. As shown in fig. 2, each row corresponds to a key frame of a test action, which is respectively a hurdle step, a straight bow step, a push-up step, an ipsilateral rotation stability and a contralateral rotation stability from top to bottom. In addition, squatting, shoulder mobility, active straight knee leg lifting do not require action resolution and therefore have no keyframes.

And step two, matching the gesture sequence of the test process obtained in the step one with a key frame, wherein the key frame is matched, namely, the distances between the current gesture and all standard gestures are calculated, and the calculation method of the distances comprises Manhattan distance, euclidean distance, cosine similarity and the like. Thus locating the boundaries of the resolved actions, partitioning the test procedure into a sequence of resolved actions, wherein each initial action and the next initial action (not included) are determined as one action instance, and the boundaries of the corresponding resolved actions determine the boundaries of each action instance.

In practical application, the euclidean distance between two frames is used as a distance value. The key frame with the minimum current gesture distance value is the key frame corresponding to the current gesture frame.

In practical application, a sliding window method can be adopted to carry out smoothing operation on a result sequence of action decomposition so as to make up for misjudgment of key frame matching, and the window length is usually 3-7, and in the embodiment, a sliding window with the length of 7 is adopted so as to enable the gesture track to be smoother.

And thirdly, scoring the motions by taking the motion instance as a unit, wherein in seven test motions of the FMS, three motions of squatting deeply, shoulder flexibility and active leg lifting are divided into simple motions, and four motions of hurdle step, straight bow step, push-up and rotation stability are divided into complex motions. The simple actions directly calculate corresponding data according to the scoring standard and score, and the complex actions are obtained through the complex action scoring model. The complex action model consists of a graph convolution network, an LSTM network and a full-connection network, wherein the graph convolution network is responsible for extracting spatial features of a gesture sequence, the LSTM network is responsible for modeling temporal features, the LSTM network sends hidden layer output of a last frame into the full-connection network to obtain action scoring when training, and the full-connection network carries out score prediction on hidden layer output of the LSTM at each frame when testing, so that a scoring sequence of testing actions is obtained.

In practical application, the scoring criteria for squatting are: the horizontal position of the hip joint point is required to be below the knee joint point, otherwise, the action is invalid, and the score is 1; the included angle between the straight line where the ankle joint and the toe are positioned and the ground is more than or equal to 30 degrees, and the score is calculated to be 2 minutes; the included angle between the straight line where the ankle joint and the toe are positioned and the ground is less than 30 degrees, and the score is 3.

In practical applications, the scoring criteria for shoulder flexibility are: the distance between the double fists is smaller than the length of one hand, and the score is 3; the distance between the two fists is less than one half hand length, and the two fists are counted for 2 minutes; the distance between the two fists exceeds one half hand length and counts 1 minute.

In practical application, the scoring criteria for actively straightening the knee and lifting the leg are as follows: the vertical position of the ankle joint is between the ankle joint and the ankle joint, and the score is 3; the distance between the two fists is less than one half hand length, and the two fists are counted for 2 minutes; the distance between the two fists exceeds one half hand length and counts 1 minute.

In practical application, as shown in fig. 3, the complex action scoring model is composed of a graph roll layer, an LSTM layer, an attention layer and a full connection layer. The image convolution layer comprises ten layers of image convolution units for extracting the characteristics of the spatial information of a single gesture frame, the LSTM comprises two hidden layers with the size of 256, the dimension and the size of the characteristics of the outputted hidden layers are the same as those of the output of the image convolution layer, and then the output results of the two layers are spliced and sent to the attention layer; and finally, obtaining a grading classification result of each frame through the full connection layer.

Judging whether the scoring result obtained in the third step reaches full score of the corresponding action;

and step five, if the judgment result of the step four is yes or the current action example is simple action, outputting the grading result of the current action example, and re-executing the step two to evaluate the next action example.

In practical application, as shown in fig. 4, the action category and the current scoring result are displayed in real time in the upper left corner of the test screen.

And step six, if the judgment result of the step four is negative and the current action example is a complex action, carrying out defect analysis on the action example. As shown in fig. 5, the specific steps are as follows:

step 6.1, DTW action alignment: the action to be scored is subjected to sequence alignment based on the decomposition action fragments through a dynamic time alignment algorithm and the full-scale action.

Step 6.2, posture replacement: and in the scoring sequence obtained in the step three, starting from the frame with the score change appearing for the first time, and replacing the frame with the frame of standard action according to the sequence alignment result. Before replacement, the human body gesture in the standard action frame needs to be scaled to the same scale as the replaced gesture through a maximum and minimum distance-based normalization algorithm. The flow of the maximum and minimum value specification algorithm is as follows: firstly, calculating the central point of each human body gesture, namely the average value of all joint point coordinates of the same gesture, then determining the scale of coordinate scaling by calculating a scaling factor, namely the average value of the distances between all joint points and the central point, and finally scaling all coordinates to ensure that the coordinate scale of each gesture is the same.

Step 6.3, action scoring: and (3) re-sending the replaced gesture sequence obtained in the step (6.2) into a complex action scoring model for scoring, and re-executing the step (6.2) and the step (6.3) until a full-score gesture sequence is obtained, wherein the replaced part in the original gesture sequence is the defect action positioned by the algorithm.

Step 6.4, defect part analysis: based on the full-scale replacement gesture sequence obtained in the step 6.3, aiming at the replaced gesture fragments, namely the defect action positioning result, in terms of space, aiming at the defect actions, a defect part analysis algorithm of the defect actions divides the human body joint point into six areas of four limbs, a trunk and a head, and the parts which are more likely to be in error in each area of the body at the moment when the current defect action occurs are ordered for a tester from small to large through the similarity of the joint angles of each area and the standard action. Wherein, the lower the similarity is, the greater the possibility of defects is.

In practical application, the similarity between the joint angles of all the areas and the standard actions is measured by cosine similarity.

Step 6.5, outputting an analysis report: the action evaluation link realizes intelligent scoring of the test action and accurate positioning of the defect action of the tester in time and space, and after the defect analysis result is obtained, the method formats the defect analysis result and prompts the tester in real time, thereby realizing self-learning and self-test of the tester.

The invention fills the blank of the prior art and provides an action scoring and defect analysis method aiming at functional action screening. The method divides actions into two types, namely simple action scoring and complex action scoring, wherein the simple action scoring is obtained through angle and distance calculation, and the complex action scoring is obtained through a graph convolution network and an action scoring network designed by an LSTM network. The method can find out the defect action in the test process, locate and describe the defect part by the technologies of sequence alignment, segment replacement, similarity calculation and the like, guide a tester to perform correct action adjustment in real time, realize FMS independent test, and further improve the motion quality and life quality of the tester.

The principles and embodiments of the present invention are described herein with reference to specific examples, the description of which is provided only to assist in understanding the method of the present invention and its core ideas; also, it is within the scope of the present invention to be modified by those of ordinary skill in the art in light of the present teachings. In view of the foregoing, this description should not be construed as limiting the invention.

Claims (4)

1. An action scoring and defect analysis method for functional action screening, characterized in that: comprises the following steps of the method,

inputting a video of a testing process into a gesture estimation model in real time to obtain a gesture sequence of the current testing process; simultaneously, acquiring key frames of standard actions from training data acquired in the earlier stage through a clustering algorithm, namely taking a single gesture frame as a sample, wherein each gesture frame comprises coordinates of all the nodes of the gesture frame, clustering through the clustering algorithm, and then selecting boundary frames of the decomposition actions from a clustering center of the gesture of the standard actions; the selection standard of the boundary frame is that the decomposition action of each action meets the minimum change principle, namely, the motion track of each joint point in the same decomposition action is ensured to be unidirectional; wherein each complex action includes a number of key frames;

step two, matching the gesture sequence of the testing process obtained in the step one with a key frame;

step three, performing action scoring by taking the action instance as a unit;

judging whether the scoring result obtained in the third step reaches full score of the corresponding action, and judging whether the current action belongs to a simple action or a complex action;

step five, if the judgment result of the step four is yes or the current action example is simple action, outputting the grading result of the current action example, and re-executing the step two to evaluate the next action example;

and step six, if the judgment result of the step four is negative and the current action example is a complex action, carrying out replacement scoring on the action example, and outputting a defect analysis result.

2. A method of action scoring and defect analysis for functional action screening according to claim 1, wherein: the implementation method of the second step is that,

the key frame is matched, namely the distance between the current gesture and all standard gestures is calculated, wherein the distance calculation method comprises Manhattan distance, euclidean distance and cosine similarity; thus locating the boundary of the decomposition action and dividing the testing process into a decomposition action sequence; wherein each initial action and the next initial action are determined as an action instance, and the boundary of the corresponding decomposition action is determined.

3. A method of action scoring and defect analysis for functional action screening according to claim 2, wherein: the implementation method of the third step is that,

in seven test actions of the FMS, three actions of deep squat, shoulder flexibility and active straight knee leg lifting are divided into simple actions, and four actions of hurdle step, straight bow step, push-up and rotation stability are divided into complex actions; the simple actions directly calculate corresponding data according to the scoring standard and score, and the complex actions are obtained through a complex action scoring model; the complex action model consists of a graph convolution network, an LSTM network and a full-connection network, wherein the graph convolution network is responsible for extracting spatial features of a gesture sequence, the LSTM network is responsible for modeling time features, the LSTM network sends hidden layer output of the last frame into the full-connection network to obtain action scoring when training, and the full-connection network carries out score prediction on hidden layer output of the LSTM at each frame when testing, so that a scoring sequence of testing actions is obtained.

4. A method of action scoring and defect analysis for functional action screening as recited in claim 3, wherein: the implementation method of the step six is that,

step 6.1, action alignment: the motion to be scored is subjected to sequence alignment based on the decomposition motion fragments through a dynamic time alignment algorithm and full-scale motion, wherein the dynamic time alignment uses Euclidean distance to calculate the distance between two gesture frames;

step 6.2, posture replacement: in the scoring sequence obtained in the step three, starting from the frame with the score change appearing for the first time, replacing the frame with the frame of standard action according to the sequence alignment result; before replacement, scaling the human body gesture in the standard action frame to the same size of the replaced gesture by a maximum and minimum value normalization algorithm based on the distance; the flow of the maximum and minimum value specification algorithm is as follows: firstly, calculating a central point of each human body gesture, namely an average value of coordinates of all joint points of the same gesture, then determining a coordinate scaling scale by calculating a scaling factor, namely an average value of distances between all joint points and the central point, and finally scaling all coordinates to enable the coordinate scale of each gesture to be the same;

step 6.3, action scoring: re-sending the replaced gesture sequence obtained in the step 6.2 into a complex action scoring model for scoring, and re-executing the step 6.2 and the step 6.3 until a full-score gesture sequence is obtained, wherein the replaced part in the original gesture sequence is the defect action positioned by the algorithm;

step 6.4, defect part analysis: based on the full-scale replacement gesture sequence obtained in the step 6.3, aiming at the replaced gesture fragments, namely the defect action positioning result, in terms of space, aiming at the defect actions, a defect part analysis algorithm of the defect actions divides the human body joint point into six areas of four limbs, a trunk and a head, and the parts which are more likely to be in error in each area of the body at the moment when the current defect action occurs are sequenced to a tester from small to large through the similarity of the joint angles of each area and the standard action; wherein, the lower the similarity is, the greater the possibility of defects is;

step 6.5, outputting an analysis report: the action evaluation link realizes intelligent scoring of the test action and accurate positioning of the defect action of the tester in time and space; after the defect analysis result is obtained, formatting the result and prompting a tester in real time, so that the independent and independent test of the tester is realized, and the requirement of functional action screening is reduced.