CN101794451A - Tracing method based on motion track - Google Patents

Abstract

The invention relates to a tracing method based on a motion track, belonging to the technical field of video image processing. The tracing method comprises the following steps of detecting a candidate ball; producing tracks; selecting tracks; and connecting the tracks. In the method, a color distance is calculated by extracting the chrominance component and the saturation degree component in an image, and the brightness component is neglected, therefore, influences caused by illumination variation to a detection result can be eliminated; the accuracy and the effectiveness of object rough selection is efficiently enhanced by utilizing the correlation of space background color information; by utilizing the length characteristic to select the tracks, the complex degree for selecting tracks is simplified, and the treatment speed of the whole tracing detection system is enhanced. Objects which are undetected can be more accurately predicted by adopting different strategies to connect the tracks under different conditions.

Description

Tracking based on movement locus

Technical field

What the present invention relates to is a kind of method of technical field of video image processing, specifically is a kind of tracking based on movement locus that is used for football.

Background technology

Based on the sport video analysis of computer vision, always be the research focus that enjoys attention.Football has mass foundation and vast football fan colony widely as most popular in the world ball game.Therefore, the automatic analysis of section of football match video there is good application prospects, has very practical value.The football video analysis mainly comprises video frequency searching, video index, video annotation, video frequency abstract, critical event detection and strategy and tactics analysis or the like.

The detection and tracking ball is a very challenging job in section of football match video, and there are all drawbacks in disposal route in the past, and it is lower to show as accuracy of detection, is subjected to external environment influence such as light big etc.This mainly contains, and following some reason causes: take the position of video camera of football match and direction always in the variation that does not stop, the motion of football not only comprises the motion of ball self in the video of therefore competing, also comprise the motion of video camera.In the detection and tracking process of ball, this two aspect all must be considered; Because the influence of on-the-spot light and ball movement velocity, characteristic informations such as the color of ball, size, shape can often change, so to be difficult to single ball be that object is set up an effective model and directly detected ball; In football match, through the situation that regular meeting takes place that ball contacts with the court line with the sportsman or blocked by the sportsman, this can cause very big difficulty to the detection of ball.

Find through retrieval prior art, document Tong X F, Wang T, Li W L, with Zhang Y M, " Athree-level scheme for real-time ball tracking ", MCAM 2007.LNCS, vol.4577, SpringerHeidelberg 2007 be by based on the ball object, in the track and the tri-layer model between track realized detection and tracking to ball, obtained certain effect, but processing more complicated in the prior art between track and needs are artificial auxiliary, greatly reduce the efficient of system's operation, and practical value is not high; And the completion process of track also is to realize that by simple interpolation accuracy is not high.

Summary of the invention

The present invention is directed to the prior art above shortcomings, a kind of tracking based on movement locus is provided, can improve football detects and follows the tracks of in the match video efficient and accuracy rate.

The present invention is achieved by the following technical solutions, the present invention includes following steps:

Step 1: the detection of candidate's ball: the raw video image that receives in real time by Network Transmission is carried out Object Segmentation handle, extract the candidate's ball set in the video image;

The concrete steps that described Object Segmentation is handled comprise:

1.1) extract zone, non-meadow in the panorama sketch by color characteristic.

1.2) use Hough transformation to detect the court cathetus to remove these straight lines then, so just eliminated the noise of markings generation in the court.

1.3) bianry image cut apart after, utilize the shape and the size characteristic elimination partial noise of ball.

1.4) utilize the correlativity of space background colouring information, the noise that the auditorium of elimination background is effectively produced.

Described candidate's ball set is meant all object similar to the ball visual signature and object balls to be tracked in the raw video image.

Step 2: the generation of track: in the time-space domain of raw video image, seek the seed tlv triple, and the seed tlv triple is initialized as the course of action track, adopt Kalman filter that the course of action track is predicted then, and utilize the mode of predicting checking to carry out the track growth, obtain some prediction routes;

Described seed tlv triple is meant: candidate's ball object all occurred near the continuous three two field picture positions in video.

Described searching seed tlv triple is meant:

A) with second frame as initial, be the center with the object's position of the candidate's ball in this frame, with the region of search of the range of movement of candidate's ball in every frame as this frame;

B) whether have candidate's ball to fall in the determined region of search in the front and back consecutive frame of searching present frame:

B1) fall into the region of search when candidate's ball, judge then whether this frame is included in the known trajectory, in not being included in known trajectory, then be initialized as a new track, and write down the position of each tracing point in described three frames with present frame and front and back consecutive frame thereof;

B2) when candidate's ball not in the region of search, then as the object repeating step a) with the back frame of present frame.

Described course of action track is meant: same candidate's ball object occurs in sequence of frames of video continuously, writes down the particular location of its ball in every frame, and resulting location sets is exactly the course of action track.

The parameter of described Kalman filter is:

State-transition matrix: $A=\left[\begin{array}{cccc}1& 1& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& 1\\ 0& 0& 0& 1\end{array}\right]$ Measure matrix: $H=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 0& 1& 0\end{array}\right]$

The mode of described prediction checking is meant: by Kalman filter the object of next frame in the track is predicted earlier, be the center with this prediction result then, in certain scope search candidate ball, verify out that with this this candidate's ball is whether on this track.

Step 3: the selection of track:, obtain the movement locus fragment of football scattered interruption in video by time span comparison and cancelling noise track to some prediction routes;

Described time span relatively is meant: at first to the prediction route of all generations, according to the time sequencing of the starting point of predicting route it is lined up in an orderly manner; To two adjacent prediction routes, utilize the context of these two prediction route starting points and end point then, judge whether they intersect on the time-space domain; Last according to predicting putting in order of route, adjacent prediction route is handled in twos: non-intersect on the time-space domain when adjacent two prediction routes, then two prediction routes all keep as the movement locus fragment.

Described cancelling noise track is meant: according to putting in order of prediction route, when adjacent two prediction routes intersect in the time-space domain, then obtain the time span of two prediction routes, then that time span is long prediction route keeps as the movement locus fragment, and the short prediction route of time span is rejected as noise track.

Step 4: the connection of track: some movement locus fragments are connected one by one, obtain complete football track, thereby realized the detection and the tracking of football.

The described connection one by one is meant:

4.1) directly utilize the Kalman filtering prediction, some the movement locus fragments that step 3 obtains are carried out forward prediction prolongation calculating and back forecast prolongation calculating, obtain prolonging back movement locus fragment;

4.2) calculate after per two adjacent prolongations the movement locus fragment in forecast interval at a distance of nearest spacing, and write down after corresponding two prolongations of this spacing two corresponding future positions on the movement locus fragment;

4.3) movement locus fragment after two adjacent prolongations is smoothly filled, obtain complete football track.

Described forecast interval is meant: the time period between two movement locus that separate, and promptly between first frame of the last frame of last track and back one track.

Described level and smooth filling is meant: by the mode of smooth trajectory, replenish the particular location of object between two separated tracks:

I) when the pairing time point of two future positions that obtains is identical, with this time point of this time point of satisfying movement locus fragment after the prolongation of predicted condition forward movement locus fragment with forward part and after satisfying the prolongation of predicted condition backward with the rear section respectively as the level and smooth stuffer in the front and back of this time point, obtain complete football track; The value of this time point object on track represented in the average of the prediction of this time point by two tracks;

The pairing time point of two future positions that ii) ought obtain will satisfy the part before the last time point of movement locus fragment after the prolongation of predicted condition forward and satisfy the later part of after the prolongation of the postcondition back time point of movement locus fragment respectively as the level and smooth stuffer in the front and back of this time point not simultaneously; Part between pairing two time points of two future positions adopts the mode of simple one-dimensional linear interpolation to fill, and two end points of interpolation are exactly two above-mentioned future positions, obtain complete football track.

Compared with prior art, the present invention has following beneficial effect:

1. the present invention calculates color distance by chromatic component and the saturation degree component that extracts in the image, ignores luminance component, can eliminate illumination variation like this to influence that testing result caused;

2. the present invention utilizes the correlativity of space background colouring information, has improved the degree of accuracy and the validity of object rough effectively;

3. the present invention utilizes length characteristic that track is selected, and has simplified the complexity that track is selected, and has improved the processing speed of whole detection tracker;

4. the present invention adopts different strategies that track is connected under different situations, can dope the object of omission more accurately.

Description of drawings

Fig. 1 is a schematic flow sheet of the present invention.

Fig. 2 is candidate's ball processing procedure picture figure.

Fig. 3 is video track product process figure.

Fig. 4 handles picture figure for video track.

Fig. 5 is the testing result figure of system.

Embodiment

Below embodiments of the invention are elaborated, present embodiment is being to implement under the prerequisite with the technical solution of the present invention, provided detailed embodiment and concrete operating process, but protection scope of the present invention is not limited to following embodiment.

As shown in Figure 1, present embodiment may further comprise the steps:

1, detects the candidate's ball set that comprises in every two field picture in the section of football match video.

Four kinds of visual angle types are arranged: full shot, medium camera lens, close-up shot and grandstand camera lens in the section of football match video.Because football mainly appears in the full shot, so this patent is also mainly considered the ball in the full shot is carried out detection and tracking.

After filtering out full shot, extract the zone, non-meadow in the image earlier.In football video, the color on meadow has very high stability and unicity.The HSV that counts two field picture by the statistics front accumulates color histogram, obtains the main color component of image at an easy rate, just the color component on meadow.Calculate the color of each pixel and the distance between the main color, judge according to this distance value whether this pixel belongs to the zone, meadow.In system, chromatic component and the saturation degree component got in the hsv color space calculate color distance, ignore luminance component, reduce the influence that illumination variation is brought.Like this, just obtained panoramic picture and handled bianry image later, shown in Fig. 1 (a).Wherein black region is represented zone, non-meadow, and it had both comprised object ball, has also comprised many noises.

Then, detect the court cathetus with Hough transformation and remove these straight lines then, be used for eliminating the noise that markings produce in the court, shown in Fig. 1 (b).

Then, bianry image carried out image segmentation after, utilize the shape and the size characteristic elimination partial noise of ball.Concrete feature has: the size of the major axis of (1) cut zone; (2) area of cut zone; (3) form parameter F=4* π * A/P ², wherein, A represents the area in zone, P represents the girth in zone.Generally speaking, the approaching more circle in zone, form parameter F is just more near 1; (4) eccentricity, the ratio of regional major axis and minor axis dimension.E=D _L/ D _S, D _LAnd D _SRepresent the length of regional minimum boundary rectangle and wide respectively.Eccentricity is big more, and the zone is that the probability of ball is just more little.

At last, utilize the correlativity of color, consider the space background colouring information.For the object ball on the court, major part all is the meadow around it, and then color of point all is near main color around it.Regulation when having the color that surpasses four points close with main color in eight neighbor points around the zone, is thought to comprise ball in this zone.Utilize the space background colouring information, the noise that the auditorium of elimination background is effectively produced.

By above four steps, just obtained candidate's ball object of every two field picture, shown in Fig. 1 (c).Utilize these candidate's balls just to generate track and track is handled accordingly.

2, the generation of track and selection

Video is divided into small pieces, and each segment all comprises the frame of video of specific quantity.In native system, be that base unit is handled track with the segment, that is to say that the length of the track of processing can not surpass the number of video frames of segment.

After obtaining candidate's ball of every two field picture, at first in the time-space domain, seek the seed tlv triple.So-called seed tlv triple candidate's ball object all occurred near the continuous three two field picture positions exactly in video.Whether with the second frame object position is the center, have candidate's ball to fall near this position in the frame before and after seeking.After finding the seed tlv triple, judge earlier whether it is included in the already present track.If no, just with new track of this tlv triple initialization, and the position of good each tracing point of record.

After obtaining new track, track is predicted with Kalman filter.Kalman filter is a state prediction device commonly used in the discrete time process, and inside mainly comprises following two equations:

The system motion equation X _K+1=AX _k+ w _k(1);

Systematic observation equation Z _k=HX _k+ v _k(2);

Wherein: X _k(X _K+1) be system state vector at moment K (K+1), Z _kFor the system state at moment K is measured vector.w _kAnd v _kBe respectively the motion of normal distribution and measure noise vector, both are separate.A and H are respectively state-transition matrix and measure matrix.The center of getting ball is that system state is measured vector, and center and its movement velocity of getting ball are the system state vector.Therefore have

$X=\left[\begin{array}{c}x\\ \stackrel{\‾}{x}\\ y\\ \stackrel{\‾}{y}\end{array}\right],$ $Z=\left[\begin{array}{c}x\\ y\end{array}\right],$ $A=\left[\begin{array}{cccc}1& 1& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& 1\\ 0& 0& 0& 1\end{array}\right],$ $H=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 0& 1& 0\end{array}\right]---\left(3\right);$

Wherein: (x, the y) center of expression ball, (x, y) expression ball difference movement velocity in the x and y direction.

Predict the position at track place in a new two field picture by Kalman filter, and in this two field picture, search for candidate's ball object near this position.If exist, then prolong track to this frame, and with this candidate ball center as track position in this frame.If do not search corresponding candidate's ball, the object that this track representative then is described is in this frame omission.When the omission frame number also surpasses threshold values, still prolong track to this frame, and with the predicted value of Kalman filter as the position of track at this frame; When the omission frame number surpasses threshold values, think that then this track disappears in video, so the track that stops growing.Detailed process as shown in Figure 3.

By the track growth, just can from video clips, obtain the track that many candidate's ball objects (comprising true football and other noise) generate, shown in Fig. 4 (a).Wherein, some is the track that true football generates, and some is the track that produces owing to the noise of introducing.So must therefrom select the track of true football.

Define a set C who forms by true football track earlier _fPair set C _fInitialization, thus with track all as the set C _fIn element, i.e. C _f={ T _i, i=1,2 ..., N}, wherein, T _iI bar track in the expression current video segment, N represents the sum of track in the current video segment.

Get two track T in the video clips _uAnd T _v, track T wherein _uStart frame less than track T _vStart frame, i.e. K _{Min, u}＜=K _{Min, v}, K wherein _{Min, u}(K _{Min, v}) be track u (start frame v).As track T _uEnd frame greater than track T _vStart frame the time, i.e. K _{Max, u}＞=K _{Min, v}, then two tracks intersect in the time-space domain, i.e. T _u∩ T _vOtherwise, think that then these two tracks separate.In video clips, true football track is all long usually, and the track that noise produces is shorter.Therefore when two intersection of locus, get the track of the long track of course length as true ball.Have:

Wherein: L _u=K _{Max, u}-K _{Min, u}, L _v=K _{Max, v}-K _{Min, v}

By track is selected, finally obtained the set C that true football track is formed _f, shown in Fig. 4 (b).C _fIn comprised a section track that is separated from each other, promptly also exist the frame of omission between the track.Obtain a complete track in video clips, just must the track of these separation be connected.

3, the connection of track

Obtaining by after the sphere path curve set, next will populated per two tracks between the frame of omission.By observing, see that omission frame between two tracks mainly is to overlap with the sportsman or blocked by the sportsman and the reason of ball travel direction and speed flip-flop causes owing to football on the court.Compensate the omission point in conjunction with Kalman filter prediction and approach based on linear interpolation.

As two track T _uAnd T _v, K wherein _{Max, u}＜K _{Min, v}At first, obtain track T respectively by Kalman filter _uAnd T _vIn the interval

In predicted value, use

With Represent, wherein Shown in Fig. 4 (c).Find out 2 points when two tracks are nearest in forecast interval then, correspond to track T respectively _uOn a frame and T _vOn the b frame.That is:

$(a,b)=\underset{a.b}{\arg \min }\mathrm{dist}({\hat{p}}_{a,u},{\hat{p}}_{b,v})$

$s.t.\left\{\begin{array}{c}a\≤b,\\ K_{\max ,u}\≤a\≤K_{\min ,v},\\ K_{\max ,u}\≤b\≤K_{\min ,v},\end{array}\right.---\left(5\right);$

Wherein:

$\mathrm{dist}({\hat{p}}_{a,u},{\hat{p}}_{b,v})=\sqrt{{({\hat{x}}_{a,u}-{\hat{x}}_{b,v})}^2+{({\hat{y}}_{a,u}-{\hat{y}}_{b,v})}^2},$

${\hat{p}}_{b,v}=({\hat{x}}_{b,v},{\hat{y}}_{b,v}).;$

By finding the solution following formula, just obtained the value of a and b.If the value of a and b equates, determine that then football still keeps motion in omission image duration, and, perhaps be subjected to blocking of sportsman and can not correctly detect object ball at motion process because the motion of ball speed is too fast; If a is less than b, then football static state occurred in omission image duration, and this situation often occurs in the sportsman and receives teammate's pass, and trapping moments later spreads out of ball during this period of time again.When this sportsman catches, because ball combines with the people, thereby omission has appearred.

Like this, also divide following two kinds of situations to replenishing of omission frame.

When a equates with b, the omission ball position track T before a frame then _uRepresent the omission ball position track T after a frame in this interval predicted value _vPredicted value represent that and a frame ball position is track T _uAnd T _vAverage in this frame predicted value.That is:

$p_k=\left\{\begin{array}{cc}{\hat{p}}_{k,u}& K_{\max ,u}\&le;k<a\\ ({\hat{p}}_{k,u}+{\hat{p}}_{k,v})/2& k=a\\ {\hat{p}}_{k,v}& a<k\&le;K_{\min ,v}\end{array}\right.---\left(6\right);$

As a during less than b, the omission ball position before a frame is still used track T _uRepresent that in this interval predicted value the omission ball position after a frame is still used track T _vPredicted value represent that and ball motion is less in the frame between a and the b, so just can obtain comparatively accurate target ball position with simple linear interpolation.

$p_k=\left\{\begin{array}{cc}{\hat{p}}_{k,u}& K_{\max ,u}\&le;k\&le;a\\ (k-a)({\hat{p}}_{b,v}-{\hat{p}}_{a,u})/(b-a)& a<k<b\\ {\hat{p}}_{k,v}& b\&le;k\&le;K_{\min ,v}\end{array}\right.---\left(7\right);$

Like this, by said method, the omission ball position between populated exactly track forms a complete football track, thereby has realized the detection and tracking of football, shown in Fig. 4 (d).

Detect with tracking results as shown in Figure 5, wherein left column is former figure, the right side is classified as through track and is handled the testing result figure that obtains.

The detection of football is adopted said method with tracking, and by experimental result is added up, obtaining accurate rate is 90.25%, and looking into the rate of depositing is 77.8%.Compared with prior art, all improve.In addition, the native system image processing speed reaches 60.36 (fps), and the speed of processing is greatly improved.

Claims (8)

1. the tracking based on movement locus is characterized in that, may further comprise the steps:

Step 1: the detection of candidate's ball: the raw video image that receives in real time by Network Transmission is carried out Object Segmentation handle, extract the candidate's ball set in the video image;

Step 2: the generation of track: in the time-space domain of raw video image, seek the seed tlv triple, and the seed tlv triple is initialized as the course of action track, adopt Kalman filter that the course of action track is predicted then, and utilize the mode of predicting checking to carry out the track growth, obtain some prediction routes;

Step 3: the selection of track:, obtain the movement locus fragment of football scattered interruption in video by time span comparison and cancelling noise track to some prediction routes;

Step 4: the connection of track: some movement locus fragments are connected one by one, obtain complete football track, thereby realized the detection and the tracking of football.

2. the tracking based on movement locus according to claim 1 is characterized in that, described candidate's ball set is meant all object similar to the ball visual signature and object balls to be tracked in the raw video image.

3. the tracking based on movement locus according to claim 1 is characterized in that, described seed tlv triple is meant: candidate's ball object all occurred near the continuous three two field picture positions in video.

4. the tracking based on movement locus according to claim 1 is characterized in that, described searching seed tlv triple is meant:

A) with second frame as initial, be the center with the object's position of the candidate's ball in this frame, with the region of search of the range of movement of candidate's ball in every frame as this frame;

B) whether have candidate's ball to fall in the determined region of search in the front and back consecutive frame of searching present frame:

B1) fall into the region of search when candidate's ball, judge then whether this frame is included in the known trajectory, in not being included in known trajectory, then be initialized as a new track, and write down the position of each tracing point in described three frames with present frame and front and back consecutive frame thereof;

B2) when candidate's ball not in the region of search, then as the object repeating step a) with the back frame of present frame.

5. the tracking based on movement locus according to claim 1, it is characterized in that, the mode of described prediction checking is meant: by Kalman filter the object of next frame in the track is predicted earlier, be the center with this prediction result then, in certain scope search candidate ball, verify out that with this this candidate's ball is whether on this track.

6. the tracking based on movement locus according to claim 1, it is characterized in that, described cancelling noise track is meant: according to putting in order of prediction route, when adjacent two prediction routes intersect in the time-space domain, then obtain the time span of two prediction routes, then that time span is long prediction route keeps as the movement locus fragment, and the short prediction route of time span is rejected as noise track.

7. the tracking based on movement locus according to claim 1 is characterized in that, the described connection one by one is meant:

4.1) directly utilize the Kalman filtering prediction, some the movement locus fragments that step 3 obtains are carried out forward prediction prolongation calculating and back forecast prolongation calculating, obtain prolonging back movement locus fragment;

4.2) calculate after per two adjacent prolongations the movement locus fragment in forecast interval at a distance of nearest spacing, and write down after corresponding two prolongations of this spacing two corresponding future positions on the movement locus fragment;

4.3) movement locus fragment after two adjacent prolongations is smoothly filled, obtain complete football track.

8. the tracking based on movement locus according to claim 1 is characterized in that, described level and smooth filling is meant: by the mode of smooth trajectory, replenish the particular location of object between two separated tracks:

I) when the pairing time point of two future positions that obtains is identical, with this time point of this time point of satisfying movement locus fragment after the prolongation of predicted condition forward movement locus fragment with forward part and after satisfying the prolongation of predicted condition backward with the rear section respectively as the level and smooth stuffer in the front and back of this time point, obtain complete football track; The value of this time point object on track represented in the average of the prediction of this time point by two tracks;

The pairing time point of two future positions that ii) ought obtain will satisfy the part before the last time point of movement locus fragment after the prolongation of predicted condition forward and satisfy the later part of after the prolongation of the postcondition back time point of movement locus fragment respectively as the level and smooth stuffer in the front and back of this time point not simultaneously; Part between pairing two time points of two future positions adopts the mode of simple one-dimensional linear interpolation to fill, and two end points of interpolation are exactly two above-mentioned future positions, obtain complete football track.

Images