CN115393373A - Monocular three-dimensional parabolic ball trajectory tracking method, system, medium and device for court videos - Google Patents
Monocular three-dimensional parabolic ball trajectory tracking method, system, medium and device for court videos Download PDFInfo
- Publication number
- CN115393373A CN115393373A CN202210433272.5A CN202210433272A CN115393373A CN 115393373 A CN115393373 A CN 115393373A CN 202210433272 A CN202210433272 A CN 202210433272A CN 115393373 A CN115393373 A CN 115393373A
- Authority
- CN
- China
- Prior art keywords
- ball
- track
- point
- player
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 41
- 230000033001 locomotion Effects 0.000 claims abstract description 20
- 239000011159 matrix material Substances 0.000 claims abstract description 19
- 238000004590 computer program Methods 0.000 claims description 12
- 238000001514 detection method Methods 0.000 claims description 11
- 238000006243 chemical reaction Methods 0.000 claims description 7
- 238000010801 machine learning Methods 0.000 claims description 6
- 238000012216 screening Methods 0.000 claims description 4
- 238000003860 storage Methods 0.000 claims description 4
- 230000001133 acceleration Effects 0.000 claims 1
- 230000005484 gravity Effects 0.000 claims 1
- 238000009826 distribution Methods 0.000 description 4
- 230000003111 delayed effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000035755 proliferation Effects 0.000 description 1
- 238000004513 sizing Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/168—Segmentation; Edge detection involving transform domain methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Multimedia (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a monocular three-dimensional parabola trajectory tracking method, a monocular three-dimensional parabola trajectory tracking system, a monocular three-dimensional parabola trajectory tracking medium and monocular three-dimensional parabola trajectory tracking equipment for court videos, which comprise the following steps: and detecting a first frame of the ball motion video by using a single-camera calibration algorithm to obtain a projection matrix from a ball field three-dimensional space of the ball video to a camera picture. And extracting 2D positions of the current frame sphere of the video by using four filters. And establishing a three-dimensional parabolic equation with starting point and end point constraints, converting the 2D track of the ball into an equation of a 3D track, and solving the 3D track parameters by using a DLT algorithm. And correcting the speed of the known point and the track by using an EM algorithm according to the known point marked on the track to obtain a final track. According to the invention, the three-dimensional information and the time sequence information of the ball track in the game can be obtained at high precision through the single camera, and the problem that the far end and the near end of the ball track cannot be distinguished is effectively avoided.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a monocular three-dimensional parabolic ball trajectory tracking method, a monocular three-dimensional parabolic ball trajectory tracking system, a monocular three-dimensional parabolic ball trajectory tracking medium and monocular three-dimensional parabolic ball trajectory tracking equipment for court videos.
Background
With the rapid development of multimedia and the increasing demand of the public for it, its proliferation requires the development of content-based multimedia information retrieval automation systems and tools. The sports video field is seen, people are more eager to see the details of all aspects of the ball hitting of players, and are eager to freely choose the viewpoint than ever before. Traditional interactive video viewing for fast browsing, indexing and summarization has not met its requirements.
In a traditional ball game, a plurality of cameras are generally used for deploying and collecting pitching motion tracks of a ball at specific positions in real time, and then a computer analyzes a plurality of lens information of the ball, so that a space motion track of the ball is obtained. However, high demands on the mounting positions and the visible areas of the plurality of imaging lenses limit the practicality of the system thereof. In addition, in the existing track tracking method, the TLD (tracking-learning-detection) target tracking algorithm is slow in running speed and is easily influenced by background colors; the kcf (kernel correlation filters) tracking algorithm is prone to frame loss for tracking of small moving objects. In conclusion, light weight, easy and effective tracking of ball trajectory is a problem to be solved by those skilled in the art.
The paper Chen, HT, tsai, WJ, lee, SY. Et al. Ball tracking and 3D object adaptation with applications to strategies proposes a two-stage volleyball trajectory tracking algorithm that first detects candidate ball two-dimensional coordinates for each frame and then uses them to calculate the trajectory of a two-dimensional ball. And (4) by means of camera calibration, utilizing the physical characteristics of the ball motion to approximate a three-dimensional ball track from a two-dimensional track. The method can effectively realize 3D track labeling by using a single camera.
The method cannot judge the starting point and the end point of the track, and when the plane 2D track is converted into the 3D track, the ball track from far to near can be wrongly identified as the ball track from near to far, so that a larger identification error is caused, and the identification precision is reduced.
Disclosure of Invention
In view of the above, the present invention aims to provide a monocular three-dimensional parabolic ball trajectory tracking method, system, medium and device for court videos, so as to simplify the problems of limited multi-camera field layout, high operation difficulty and large calculation amount. The hitting point of the ball and the time sequence information of the ball track are determined by capturing the posture of the human body, and when the 2D track is converted into the 3D track, the ball from far to near and the ball from near to far can be effectively distinguished. In addition, the invention also provides an optimization scheme of the hitting point and the ball track based on the EM algorithm, which can reduce the detection error and further improve the precision. The specific implementation scheme is as follows:
a monocular three-dimensional parabola trajectory tracking method, system, medium and device facing to a court video comprises:
detecting a first frame of a ball motion video by utilizing a monocular camera calibration algorithm to obtain a projection matrix P from a court three-dimensional space of the ball video to a camera picture;
intercepting a first frame of picture of a target video, and segmenting a field by utilizing color clustering;
obtaining two-dimensional coordinates of 10 ground feature points and two-dimensional coordinates of 4 sphere boundary feature points by using Hough transform and combining sphere prior knowledge;
obtaining the 3D positions of the 14 two-dimensional characteristic point coordinates through the prior knowledge of the sphere, and recording the positions as W 1 ～W 14 ；
Solving the mapping conversion relation from two-dimension to three-dimension of the above characteristic points, and transforming the mapping conversion relation into
m i ＝MW i
Wherein m is i ＝[x i ，y i ，1] T ，W i ＝[x i ，y i ，z i ，1] T The projection matrix M can be solved using the following conversion formula
Solving the projection matrix p by using a DLT algorithm;
after the projection matrix P is obtained, the sphere positions are extracted using a specific filter, including color capture, sizing, shape screening, and fullness filtering.
And a color difference filter. And comparing the pictures of every three frames, screening the area with large RGB difference, and marking the area by using a rectangular frame.
The ball size filter detects that the ball radius satisfies following relational expression:
the shape-screening filter. Assuming that the horizontal-vertical ratio of the ball frame is P, the horizontal-vertical ratio needs to satisfy:
a fullness filter with fullness of D f The size of the detection object is S obj The ball edge interface area is A b-box Selecting objects with fullness smaller than a certain threshold value through the following criteria;
acquiring the hitting time and the three-dimensional coordinates of the two players according to the current human body posture of the video frame;
suppose that the player near the camera is the first player and the player far from the camera is the second player. And for the first player, carrying out human skeleton recognition according to a machine learning algorithm, and extracting human skeleton feature points.
Two-dimensional coordinates of the skeletal points closest to the racket are obtained, and the offset from the skeletal points to the racket is measured from multiple angles.
And acquiring 2D positions of the player feet according to the human skeleton points, and marking the positions as height zero points.
If the item is the hitting of an appliance such as a racket, the vertical distance from the hitting point of the appliance such as the racket to the height zero point is converted into the actual height according to the height zero point position and a measuring algorithm based on monocular vision.
And combining the 2D position and the actual height information of the shot to construct the 3D position and the shot time of the shot point of the first player.
Repeating the steps, and identifying the 3D position and the hitting time of the hitting point of the second player;
from the time of hitting the ball, the ball position is obtained by using the above-mentioned specific filter, and the coordinates of the track point and the time information of each frame obtained are expressed as [ x ] i ，y i ，n i ，1]Wherein x is i ，y i Representing two-dimensional coordinates of track points, n i Representing the time of the track point, and obtaining the value according to the hitting time and recording the value in an x and y plane;
according to the acquired ball position distribution, from the time of hitting the ball to the point adjacent to the next frame in the x direction and the y direction, respectively, the point is added to the selected points of one track.
On the basis of obtaining the position distribution of the ball, if the selected points are more than three, initializing a track prediction equation
y＝a·n 2 +b·n+c，a＜0
x＝d·n+e
Estimating the ball position of the next frame by using the equation according to the estimation coefficient value;
the equation is used to estimate the ball position for the next frame in the x and y directions, respectively, based on the estimation coefficient values. If the errors of the actual ball position and the predicted position of the next frame are smaller than a certain threshold value, adding the next frame point into the selected point, and updating the equation parameters;
if the ball position of the next frame and the predicted ball position are larger than a certain threshold value, judging that the trackball position of the next frame is lost, recording as a position lost frame, recording the trajectory and the corresponding selected point when the number of lost frames is larger than 3, repeating the steps until the batting time of another player is delayed by 20 frames;
scoring the trajectories according to the length and the prediction error within a time period of 20 frames from the first ball hit of the player to the second ball hit of the player, and selecting the trajectories with the scores larger than a threshold value;
according to the obtained two-dimensional tracks, assuming that each independent two-dimensional track contains N ball position points, listing corresponding physical equations to obtain corresponding three-dimensional coordinates, and then detecting the time of the last endpoint of the track as the ball hitting time to indicate that another player hits the ball effectively and obtains the ball hitting position, otherwise, considering that the other player hits the ball inefficiently, the ball hitting position cannot be obtained, and finally obtaining projection conversion equations of all the points;
combining three-dimensional coordinates obtained by a physical equation to obtain a track speed equation of the hitting point, performing landing judgment, and finally solving equations of all the conditions through a DLT algorithm;
if more than two tracks exist, the starting point and the corresponding frame number of the second track solution are taken as the hitting position and the hitting time, and the steps S41 to S45 are repeated.
Correcting the speed of the known point and the track by using an EM (effective velocity) algorithm according to the known point marked on the track, solving the hitting positions of the first player and the second player by using a DLT (digital Living Table) algorithm, and recording the error delta between the new hitting position and the original hitting position 1 The process is repeated until the error Δ 1 ，Δ 2 Are all less than a certain threshold;
and obtaining the three-dimensional coordinates of each point on the corrected track, and reconstructing the three-dimensional track.
Correspondingly, the invention also discloses a light-weight ball track tracking system, which comprises:
the camera calibration module is used for detecting a first frame of a ball motion video to obtain a projection matrix P from a court three-dimensional space of the ball video to a camera picture;
the ball detection module extracts the ball position by using a specific filter, and identifies indexes including color difference, size, shape and fullness;
the human body hitting point identification module is used for judging hitting time and three-dimensional coordinates of a hitting point of a player according to the position of the racket by combining the position of a human body skeleton;
and the three-dimensional track reconstruction module records the distribution of the ball on the two-dimensional coordinates frame by frame, marks the two-dimensional track of the ball, converts the 2D track of the established ball into an equation of a 3D track, solves the equation according to a DTL algorithm to obtain a 3D track parameter, and corrects the speed of a known point and the track by using an EM algorithm to obtain a final track.
Correspondingly, the invention also discloses a computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and the computer program is executed by a processor to realize the steps of the ball motion trail tracking method of the modules.
Correspondingly, the invention also discloses a ball motion trajectory tracking device, which comprises:
memory for storing a computer program
A processor for executing the computer program to realize the steps of the ball motion trajectory tracking method of the above modules when the computer program is executed by the processor.
In the invention, a ball motion trajectory tracking method comprises the following steps: and detecting a first frame of the ball motion video by using a single-camera calibration algorithm to obtain a projection matrix P from a ball field three-dimensional space of the ball video to a camera picture. And extracting 2D positions of the current frame sphere of the video by using four filters. And estimating three-dimensional coordinates of the starting point and the ending point of the parabola. And establishing a three-dimensional parabolic equation with starting point and end point constraints, converting the 2D track of the ball into an equation of a 3D track, and solving the 3D track parameters by using a DLT algorithm. And correcting the speed of the known point and the track by using an EM algorithm according to the known point marked on the track to obtain a final track.
Therefore, in the invention, firstly, a single-camera calibration algorithm is utilized to obtain a two-dimensional projection matrix from a corresponding three-dimensional space to a camera screen, then a two-dimensional track of a ball is obtained through a two-dimensional track tracking algorithm, and then the two-dimensional track is converted into a three-dimensional track through the projection matrix and a parabolic equation. The method for reconstructing the three-dimensional trajectory of the sphere by using the single camera is wider in application range and lower in equipment cost compared with a method for acquiring the 3D trajectory by using a binocular camera in a general algorithm. Secondly, the human body posture is recognized by machine learning through the algorithm, the advantage of high accuracy of the machine learning recognition is fully utilized during training, three-dimensional coordinates of a hitting point and a landing point with high confidence coefficient are obtained, 3D tracks of the ball are constrained by the hitting point and the landing point, time sequence information of the 3D tracks of the ball is obtained, and the ball from near to far and the ball from far to near can be effectively distinguished. Meanwhile, the hitting point and the three-dimensional track are corrected by adopting an EM algorithm, so that the three-dimensional track has higher precision and better track fitting effect compared with the three-dimensional track without constraint. Correspondingly, the invention also discloses a ball motion trajectory tracking system, a medium and equipment, which also have the gain effect.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of a ball trajectory tracking method according to an embodiment of the present invention.
Fig. 2 is a structural diagram of a ball trajectory tracking system disclosed in the present invention.
Fig. 3 is a structural diagram of a ball trajectory tracking device disclosed in the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the invention discloses a ball motion trajectory tracking method, which comprises the following steps of:
step S11: and detecting the first frame of the video by utilizing a monocular camera calibration algorithm to obtain a projection matrix from the three-dimensional space of the court to the picture of the camera.
In this embodiment, first, a first frame of picture of the video is captured, a field is divided by color clustering, two-dimensional coordinates of 10 ground feature points and 4 net boundary feature points are obtained by hough transform, 3D positions of the ground feature point coordinates and the boundary feature point coordinates are obtained by the feature point coordinates in combination with the size of a court, a projection matrix is constructed according to the 2D positions and the 3D positions, and the matrix is solved by a DLT algorithm.
Step S12: and extracting the position of the ball through four filters of color difference, ball size, shape and fullness to obtain a two-dimensional track of the ball.
It can be understood that, in practical applications, there are certain interferents in the motion video of the ball, so in order to eliminate the interference of the static background and some other interferents to the detection of the ball motion trajectory, the position of the ball in the current frame is first extracted through four filters, and then the subsequent steps are performed. It should be noted that, the method for extracting the moving foreground includes, but is not limited to, four filters, which should be used for achieving practical operation, and is not limited herein.
Step S13: according to the human body posture, arm skeleton analysis is carried out, the relative positions of the racket and the human body of the player are obtained, and the hitting time and the hitting point of the two players are judged
In this embodiment, it is assumed that the player close to the camera is the first player and the player far from the camera is the second player. And for the first player, carrying out human skeleton recognition according to a machine learning algorithm, and extracting human skeleton feature points. Two-dimensional coordinates of the skeletal point closest to the racket are obtained, and the offset from the skeletal point to the racket is measured from multiple angles. And acquiring a 2D position of the player foot according to the human skeleton point, and marking the position as a height zero point position. If the item is hit by a racket or other appliance, the vertical distance from the hitting point of the racket or other appliance to the height zero point is converted into the actual height according to the height zero point position and through a measuring algorithm based on monocular vision. And combining the 2D position and the actual height information of the shot to construct the 3D position and the shot time of the shot point of the first player. Repeating steps S31 to S35, the 3D position and the hitting time of the second player hitting point are identified.
Therefore, when the human body posture model is trained, the method has the advantages of high accuracy of machine learning and recognition, and can accurately position the starting point and the end point of the ball track and mark time domain information on the track information by combining multi-angle measurement offset according to specific ball motions.
Step S14: and (3) recording the 2D coordinate distribution of the flying process of the ball frame by frame from the first batting time, and marking a 2D track.
Specifically, in the present embodiment, in order to obtain a 2D trajectory during flight, the ball position is obtained using the above-mentioned specific filter from the time of a shot, and coordinates of a trajectory point and time information obtained for each frame are represented as [ x [ ] i ，y i ，n i ，1]Wherein x is i ，y i Representing two-dimensional coordinates of track points, n i The method comprises the steps of representing time of a track point, obtaining a value according to hitting time, recording the time in an x plane and a y plane, initializing a track prediction equation if the number of selected points is more than three, estimating the ball position of the next frame by using the equation according to an estimation coefficient value, judging that the position of the track ball of the next frame is lost and recording the position of the track ball as a position lost frame if the ball position of the next frame and the predicted ball position are more than a certain threshold value, recording the track and the corresponding selected point when the number of lost frames is more than 3, repeating the process until the hitting time of the other player is delayed by 20 frames, scoring the track with the score larger than the threshold value from the 20 frames after the first hitting of the player and the second hitting of the player according to the length and the prediction error from the track. It should be noted that the method for acquiring the 2D ball trajectory includes, but is not limited to, the above method, and the method is not limited herein, so as to achieve practical operation.
Step S15: and establishing a 2D-to-3D conversion equation of the spherical track according to the projection matrix M and the parabolic equation, and solving by using DTL (delay tolerant l) to finally obtain a 3D spherical track parameter.
Specifically, in this embodiment, two-dimensional spherical track points in a parabolic shape are obtained according to the method for obtaining a 2D track, and the time corresponding to each track point is labeled. Considering more than two parabolic equations of the ball trajectory can be recurrently obtained by two parabolic equations of the adjacent ball trajectory, and only considering two adjacent ball trajectories includes: combining the three-dimensional spherical parabolic equation, assume A M×N A constant matrix representing M rows and N columns, when more than two tracks are detected according to a two-dimensional detection method, the situation that a floor point exists in the middle is indicated, if the hitting of the other player is invalid, a frame corresponding to the intersection point of the two-dimensional tracks is taken as the end time of a first track, and x is calculated 2 ，y 2 ，z 2 Combining three dimensions, for all points on the trajectory, assume
In order to track a speed of the vehicle,
for track two speeds, there are:
if another player hits effectively, x 2 ，y 2 ，z 2 Which can be represented by another player hitting position, for all points on the trajectory:
if no landing spot exists, the following steps are carried out: a. The 2N×3 [v x ，v y ，v z ] T ＝A 3×1 The equations for all of the above cases can be solved by the DLT algorithm. If more than two tracks exist, the starting point and the corresponding frame number of the second track solution are taken as the hitting position and the hitting time, and the steps S41 to S45 are repeated. The known three-dimensional reconstruction equation can be reconstructed through the known hitting positions and the time sequence, and the reconstructed three-dimensional trajectory has high precision through constraint on the equation solution space. Meanwhile, by marking the time sequence information of the track, the track conditions from far to near and from near to far can be distinguished,thereby reducing the tracking error.
Step S16: and correcting the speed of the known point and the track by using an EM algorithm according to the known point marked on the track to obtain a final track.
It can be understood that due to the fact that shielding and interference exist in an actual scene, and a high-speed fuzzy phenomenon exists when a human body and a ball are in contact, errors inevitably exist in the detected three-dimensional position of the hitting point and the detected speed of the ball track, and therefore the hitting point and the speed of the ball track are corrected by adopting an EM algorithm, and the track is corrected. Specifically, in the present embodiment, the trajectory speed is solved as a known quantity and the hitting position is solved as an unknown quantity, assuming [ x ] x 1 ，y 1 ，z 1 ] T For the first stroke position of the player, [ x ] 2 ，y 2 ，z 2 ] T And constructing a trajectory equation for the second hit position of the player according to the existence of the landing. The hitting positions of the first player and the second player can be solved through a DLT algorithm, and the error delta between the new hitting position and the original hitting position is recorded 1 Then, the trajectory speed is recalculated by the method of step S15 based on the new hitting position, and the new trajectory speed and the original trajectory speed are recorded. The above process is repeated until the error Δ is reached 1 ，Δ 2 Are all less than a certain threshold.
And the camera calibration module 21 is configured to extract a projection matrix P from the three-dimensional space of the ball field of the ball video to the camera image.
And the ball detection module 22 is configured to extract a 2D position of a ball of the current frame of the video.
And the human hitting point identification module 23 is used for judging hitting time and three-dimensional coordinates of a hitting point of a player according to the position of the racket by combining the position of the human skeleton.
And a three-dimensional track reconstruction module 24 for reconstructing a three-dimensional track.
Correspondingly, the invention also discloses a computer readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to realize the steps of the monocular three-dimensional parabolic ball trajectory tracking method for court video disclosed in the foregoing.
Correspondingly, the invention also discloses badminton motion trail tracking equipment, which comprises the following components as shown in figure 3:
a memory 31 for storing a computer program;
a processor 32 for implementing the steps of the course video-oriented monocular three-dimensional parabolic ball trajectory tracking method as disclosed in the foregoing when executing the computer program.
Claims (8)
1. A monocular three-dimensional parabola trajectory tracking method, system, medium and device for court videos are characterized by comprising the following steps:
step S1: and detecting a first frame of the ball motion video by using a single-camera calibration algorithm to obtain a projection matrix P from a three-dimensional space of a court of the ball video to a camera picture.
Step S2: and extracting 2D positions of the current frame sphere of the video by using four filters.
And step S3: and estimating the three-dimensional coordinates of the starting point and the end point of the parabola.
And step S4: and establishing a three-dimensional parabolic equation with starting point and end point constraints, converting the 2D track of the ball into an equation of a 3D track, and solving the 3D track parameters by using a DLT algorithm.
Step S5: and correcting the speed of the known point and the track by using an EM algorithm according to the known point marked on the track to obtain a final track.
2. The method according to claim 1, wherein the process of step S1 comprises:
step S11: and intercepting a first frame picture of the video.
Step S12: and for the picture, dividing the field by utilizing color clustering, and obtaining two-dimensional coordinates of 10 ground characteristic points and 4 net boundary characteristic points by using Hough transform.
Step S13: and acquiring the 3D positions of the coordinates of the ground features and the boundary feature points by combining the sizes of the court through the feature point coordinates, and recording the positions as W 1 ～W 14
Step S14: for the two-dimensional coordinates and the three-dimensional coordinates of each characteristic point, there is a conversion relation
m i ＝PW i
M is said i ＝[x i ，y i ，1] T ，W i ＝[x i ，y i ，z i ，1] T ，
Step S15: the projection matrix P can be solved using the following conversion equation
Wherein said P j Represents the jth row of the projection matrix P, which is solved using the DLT algorithm.
3. The method according to claim 1, wherein the step S2 process comprises:
step S21: passing through a color difference filter. And comparing the pictures of every three frames, screening the area with large RGB difference, and marking the area by using a rectangular frame.
Step S22: through a ball size filter. Suppose D is the true sphere radius, L is the true court line length, L n To measure the length of the course, L f For the length of the far-end court line, R is the radius of the detection ball, and Delta R is the detection error, the radius of the detection ball satisfies the following relational expression:
step S23: the filter is screened through a shape. Assuming that the horizontal-vertical ratio of the ball frame is P, the horizontal-vertical ratio needs to satisfy:
step S24: through a fullness filter. Suppose the fullness is D f The size of the detection object is S obj The ball edge interface area is A b-box The fullness is defined as:
and selecting the object with the fullness smaller than a certain threshold value.
4. The method according to claim 1, wherein the step S3 procedure comprises:
step S31: suppose that the player closer to the camera is the first player and the player farther from the camera is the second player. And for the first player, carrying out human skeleton recognition according to a machine learning algorithm, and extracting human skeleton feature points.
Step S32: two-dimensional coordinates of the skeletal point closest to the racket are obtained, and the offset from the skeletal point to the racket is measured from multiple angles.
Step S33: and acquiring a 2D position of the player foot according to the human skeleton point, and marking the position as a height zero point position.
Step S34: if the item is hit by a racket or other appliance, the vertical distance from the hitting point of the racket or other appliance to the height zero point is converted into the actual height according to the height zero point position and through a measuring algorithm based on monocular vision.
Step S35: and combining the 2D position and the actual height information of the shot to construct the 3D position and the shot time of the shot point of the first player.
Step S36: repeating steps S31 to S35, the 3D position and the hitting time of the second player hitting point are identified.
5. The method according to claim 1, wherein the step S4 comprises:
step S41: and acquiring parabolic two-dimensional ball track points according to the method of claim 2 within a 20-frame time period from the first ball hit of the player to the second ball hit of the player, wherein the sequence number of each track point is i.
Step S42:suppose [ v ] x ，v y ，v z ]Velocity components of the locus points in the x, y and z axes, t i The difference between the number of frames corresponding to the track point and the number of frames at the beginning of the track, [ x ] o ，y o ，z o ] T For the coordinates of the hitting point, g is the acceleration of gravity, the following physical equations can be listed:
step S43: the method of claim 4, wherein if the time of the last end point of the detected trajectory is the hitting time, which indicates that another player hits the ball effectively, the hitting position can be obtained according to the method of step S3, otherwise, the hitting position cannot be obtained because another player is considered to be ineffective.
Step S44: according to the method of step S1, the following equation exists for each point i:
step S45:
considering more than two parabolic equations of the ball trajectory can be recurrently obtained by two parabolic equations of the adjacent ball trajectory, and only considering two adjacent ball trajectories includes: combining the three-dimensional coordinates in step S42, assume A M×N A constant matrix representing M rows and N columns, wherein when two tracks obtained in step S41 indicate that a floor point exists in the middle, if another player does not hit the ball, the frame corresponding to the intersection point of the two-dimensional tracks is used as the first trackEnd time of (c), calculate x 2 ，y 2 ，z 2 Combining the equations of step S42 and step S44, assuming that all points on the trajectory are located
In order to track a speed of the vehicle,
for track two speeds, there are:
if the other player hits the ball effectively, x is determined according to the formula of step S42 2 ，y 2 ，z 2 Which may be represented by the other player hitting position, for all points on the trajectory:
if there is no landing point, there are:
A 2N×3 [v x ，v y ，v z ] T ＝A 3×1
the equations for all of the above cases can be solved by the DLT algorithm.
Step S46: if more than two tracks exist, taking the starting point and the corresponding frame number of the second track solution as the hitting position and the hitting time, and repeating the steps S41 to S45.
6. The method of claim 1, wherein the step S5 procedure comprises:
s51: solving the trajectory speed as a known quantity and the hitting position as an unknown quantity, assuming [ x ] according to the case described in step S45 and the equation in conjunction with step S42 1 ，y 1 ，z 1 ] T For the first stroke position of the player, [ x ] 2 ，y 2 ，z 2 ] T For the second player, there are:
when a landing spot is present:
A 2N×6 [x 1 ，y 1 ，z 1 ，x 2 ，y 2 ，z 2 ] T ＝A 6×1
when there is no landing point
A 2N×3 [x 1 ，y 1 ，z 1 ] T ＝A 3×1
The hitting positions of the first player and the second player can be solved through a DLT algorithm, and the error delta between the new hitting position and the original hitting position is recorded 1 。
S52: according to S45, the hitting position in the step S51 is used, the track speed is solved through a DLT algorithm, and the error delta between the new track speed and the original track speed is recorded 2 。
S53: repeating steps S51 to S52 until the error Δ 1 ，Δ 2 Are all less than a certain threshold.
7. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the ball movement trajectory tracking method according to any one of claims 1 to 6.
8. A ball trajectory tracking device, comprising:
a memory for storing a computer program; a processor for implementing the steps of the ball movement trajectory tracking method according to any one of claims 1 to 6 when executing said computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210433272.5A CN115393373A (en) | 2022-04-22 | 2022-04-22 | Monocular three-dimensional parabolic ball trajectory tracking method, system, medium and device for court videos |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210433272.5A CN115393373A (en) | 2022-04-22 | 2022-04-22 | Monocular three-dimensional parabolic ball trajectory tracking method, system, medium and device for court videos |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115393373A true CN115393373A (en) | 2022-11-25 |
Family
ID=84115408
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210433272.5A Pending CN115393373A (en) | 2022-04-22 | 2022-04-22 | Monocular three-dimensional parabolic ball trajectory tracking method, system, medium and device for court videos |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115393373A (en) |
-
2022
- 2022-04-22 CN CN202210433272.5A patent/CN115393373A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106780620B (en) | Table tennis motion trail identification, positioning and tracking system and method | |
| US10769480B2 (en) | Object detection method and system | |
| US9330307B2 (en) | Learning based estimation of hand and finger pose | |
| CN104408725B (en) | A kind of target reacquisition system and method based on TLD optimized algorithms | |
| CN109064484B (en) | Crowd movement behavior identification method based on fusion of subgroup component division and momentum characteristics | |
| CN111444890A (en) | Sports data analysis system and method based on machine learning | |
| KR20220040433A (en) | Creating player trajectories with multi-camera player tracking | |
| CN101826155B (en) | Method for identifying act of shooting based on Haar characteristic and dynamic time sequence matching | |
| KR101428922B1 (en) | Golf swing classification apparatus and method using fuzzy system | |
| CN109684919B (en) | Badminton service violation distinguishing method based on machine vision | |
| CN114119739A (en) | Binocular vision-based hand key point space coordinate acquisition method | |
| CN110298864B (en) | Visual sensing method and device for golf push rod equipment | |
| CN111754549B (en) | Badminton player track extraction method based on deep learning | |
| BR102019000927A2 (en) | DESIGN A BEAM PROJECTION FROM A PERSPECTIVE VIEW | |
| CN115100744A (en) | Badminton game human body posture estimation and ball path tracking method | |
| CN111744161A (en) | Table tennis falling point detection and edge ball wiping judgment system | |
| CN110910410A (en) | Court positioning system and method based on computer vision | |
| WO2022232302A1 (en) | Methods and systems to automatically record relevant action in a gaming environment | |
| CN107564037A (en) | A kind of multirobot detection and tracking based on local feature | |
| CN110910489B (en) | Monocular vision-based intelligent court sports information acquisition system and method | |
| CN114037923A (en) | Target activity hotspot graph drawing method, system, equipment and storage medium | |
| JP2020107071A (en) | Object tracking device and program thereof | |
| CN108269265A (en) | Billiard ball batter's box assay method and its device based on deep learning | |
| CN115393373A (en) | Monocular three-dimensional parabolic ball trajectory tracking method, system, medium and device for court videos | |
| CN115845349A (en) | General training method for ball game items for moving target detection based on deep learning technology and auxiliary referee system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |