The goal detection method
Technical field
The present invention relates to a kind of goal and detect and the football video highlight event detecting method, in particular, relate to a kind of football video highlight event detecting method that detects based on the goal.
Background technology
Excellent incident is extracted in the fast browsing of sports video and plays an important role.Spectators can pay close attention to some critical segments rather than the whole video section in the sports video usually.The excellent incident of extraction that can be flexible automatic helps to understand rapidly and accurately video content.In the prior art, the extracting method of excellent incident mainly is divided into two classes: based on resetting and rule-based.Think that based on the method for resetting excellent incident is the incident of resetting in the television relay,, determine the generation of excellent incident by identification to replay mode.Rule-based excellent event detecting method finds corresponding excellent incident according to this rule then by rule manual or that automatic, automanual training finds excellent incident to take place.The general training pattern parameters such as hidden Markov model, dynamic bayesian network that adopt, though can obtain quite good detecting effectiveness, the cost of total system is also higher, institute is time-consuming also longer, is not suitable for the excellent incident of real-time detection.
Football match is as one of the highest sports tournament of attendance and popularity rate, if can the exciting part in the football match be extracted automatically, is that football fan or trainer, sportsman are very helpful.The football video highlight incident extracting method of traditional complexity adopts hidden Markov model to detect excellent incident in document 1 " Soccer HighlightsDetection and Recognition using HMMs " as people such as Assfalg.The motion feature that is characterized as camera that HMM training is used, the motion feature of camera itself is difficult to extract, and the HMM that obtains by training is 80% to the detection accuracy rate of excellent event detection, and is both time-consuming, and the accuracy rate of detection is not high yet.
Near the forbidden zone zone is crucial zone in the football match, and a lot of critical incidents and excellent camera lens are such as shooting, penalty kick, and free kick etc. all occur near the zone the forbidden zone.This shows, if can in frame of video, find out near the forbidden zone zone, just can be on this basis further analysis video content, thereby on semantic hierarchies, analyze the generation of excellent incident.
Summary of the invention
The purpose of this invention is to provide a kind of event detecting method of football video highlight fast and accurately that detects based on the goal.
To achieve these goals, the present invention takes following technical scheme:
A kind of goal detection method, described goal is meant football pitch, comprises the steps:
(1) at first extract the colored football video image of a frame, following step is all based on this two field picture.
(2) adopt straw hat conversion (top-hat) to extract edge image to above-mentioned video frame images;
(3) on the basis of edge image, search for the line segment that the vertical direction neighborhood meets certain threshold value;
(4) in the line segment that previous step obtains suddenly, adopt hough transform (hough) to choose two line segments the longest as candidate's goal post;
(5) adopt second heuristic rule to determine whether the candidate goal post of previous step in rapid is the goal.
Further, in the above-mentioned steps (1), can be to extract frame by frame for the extraction of a two field picture, can be to extract every frame, also extract every 5~20 fixing frames.
Further, the step of extraction edge image is as follows in the above-mentioned steps (2):
(a) coloured image is done straw hat conversion (top-hat) and extract the colour edging image;
(b) the colour edging image is carried out greyscale transformation and obtain gray level image;
(c) adopt adaptive method with the gray level image binaryzation, obtain two-value black and white edge image.
Further, adopt adaptive method with the gray level image binaryzation in the above-mentioned steps (c), be meant: adopt the 2-mean cluster, with the histogram of gray scale edge image poly-be two classes, obtain one high and one low two class cluster centres, for each pixel in the gray level image, if the cluster centre from high pixel value is nearer, then it is changed to white,, then it is changed to black if instead nearer from the cluster centre of low-pixel value.
Further, above-mentioned steps (3) is searched for the line segment that the vertical direction neighborhood meets certain threshold value on the basis of edge image, be meant: on the basis of the two-value black and white edge image that obtains, for each white pixel, search for its vertical direction up and down neighborhood be the scope of 8 to 20 pixels, if be white pixel entirely in the described scope, the line segment that then described scope is represented is candidate's goal post.
Further, described second heuristic rule is meant:
(a) length of two ball door pillars all should meet certain threshold value requirement, should not be the goal if meet then think in the scope of 5~100 pixels;
(b) distance between the two ball door pillars should meet certain threshold value requirement, should not be the goal if meet then think in the scope of 20~160 pixels.
(c) satisfying on the basis of above-mentioned rule, should meet certain proportion requirement between distance between the two ball door pillars and the long ball door pillar yet, scale-up factor should not be the goal if meet then think in 0.5~3.5 scope.
(d) satisfying on the basis of above-mentioned rule, the line segment of goal post should keep certain continuity, and the difference of the difference of the bound of each goal post and door pillar length should be the goal if meet then not think in the scope of 0~25 pixel.
(e) satisfying on the basis of above-mentioned rule, the center of the minimum rectangle that two ball door pillars constitute should be in certain zone, and the ordinate of its central point should not be the goal if meet then think in 0~240 scope.
(f) satisfying on the basis of above-mentioned rule, the position of two ball door pillars should meet certain threshold value requirement, have the upper bound of goal post of minimum lower bound and the distance that has between the lower bound of goal post in the maximum upper bound and should in the scope of 0~6 pixel, not be the goal if meet then think.
A kind of football video highlight event detecting method that detects based on the goal comprises the steps:
1) goal is detected;
2) detecting on the basis at goal, in conjunction with the excellent incident in first heuristic rule detection football video.
Further, the step 1) goal is detected and is comprised the steps:
(1) at first extract the colored football video image of a frame, following step is all based on this two field picture.
(2) adopt straw hat conversion (top-hat) to extract edge image to above-mentioned video frame images;
(3) on the basis of edge image, search for the line segment that the vertical direction neighborhood meets certain threshold value;
(4) in the line segment that previous step obtains suddenly, adopt hough transform (hough) to choose two line segments the longest as candidate's goal post;
(5) adopt second heuristic rule to determine whether the candidate goal post of previous step in rapid is the goal.
Further, in the above-mentioned steps (1), can be to extract frame by frame for the extraction of a two field picture, can be to extract every frame, also extract every 5~20 fixing frames.
Further, the step of extraction edge image is as follows in the above-mentioned steps (2):
(a) coloured image is done straw hat conversion (top-hat) and extract the colour edging image;
(b) the colour edging image is carried out greyscale transformation and obtain gray level image;
(c) adopt adaptive method with the gray level image binaryzation, it is black in edge image to obtain two-value.
Further, adopt adaptive method with the gray level image binaryzation in the above-mentioned steps (c), be meant: adopt the 2-mean cluster, with the histogram of gray scale edge image poly-be two classes, obtain one high and one low two class cluster centres, for each pixel in the gray level image, if the cluster centre from high pixel value is nearer, then it is changed to white,, then it is changed to black if instead nearer from the cluster centre of low-pixel value.
Further, above-mentioned steps (3) is searched for the line segment that the vertical direction neighborhood meets certain threshold value on the basis of edge image, be meant: on the basis of the two-value black and white edge image that obtains, for each white pixel, search for its vertical direction up and down neighborhood be the scope of 8 to 20 pixels, if be white pixel entirely in the described scope, the line segment that then described scope is represented is candidate's goal post.
Further, described second heuristic rule is meant:
(a) length of two ball door pillars all should meet certain threshold value requirement, should not be the goal if meet then think in the scope of 5~100 pixels;
(b) distance between the two ball door pillars should meet certain threshold value requirement, should not be the goal if meet then think in the scope of 20~160 pixels.
(c) satisfying on the basis of above-mentioned rule, should meet certain proportion requirement between distance between the two ball door pillars and the long ball door pillar yet, scale-up factor should not be the goal if meet then think in 0.5~3.5 scope.
(d) satisfying on the basis of above-mentioned rule, the line segment of goal post should keep certain continuity, and the difference of the difference of the bound of each goal post and door pillar length should be the goal if meet then not think in the scope of 0~25 pixel.
(e) satisfying on the basis of above-mentioned rule, the center of the minimum rectangle that two ball door pillars constitute should be in certain zone, and the ordinate of its central point should not be the goal if meet then think in 0~240 scope.
(f) satisfying on the basis of above-mentioned rule, the position of two ball door pillars should meet certain threshold value requirement, have the upper bound of goal post of minimum lower bound and the distance that has between the lower bound of goal post in the maximum upper bound and should in the scope of 0~6 pixel, not be the goal if meet then think.
Further, described first heuristic rule is meant: excellent incident occurs near the forbidden zone, goal mostly, after finding the goal, lens type information in conjunction with frame before and after the camera lens of goal, be that distant view, middle scape, feature information just can detect dissimilar excellent incidents, as goal event, corner-kick incident etc.
Compared with prior art, the invention has the advantages that:
At goal detection technique in the past, the present invention has accurately, fast, advantage that false drop rate is low.Adopt the edge image of top-hat conversion extraction, wherein represent more clear the showing of white pixel at goal, help further reliably, extract accurately goal post; Adopt the method for neighborhood search to extract candidate's goal post, greatly reduced the false drop rate that goal post detects, having avoided background flase drops such as net or billboards is the situation of goal post.The present invention is more efficient to the detection of excellent incident, quick.On the basis that only needs to detect, just can detect excellent incident fast, avoid classic method to need large-scale training data and complex features to extract in conjunction with some simple heuristic rules at the goal.
Description of drawings
Fig. 1 is the process flow diagram that goal event detects in the football video of embodiment 1.
Embodiment
Below in conjunction with the drawings and specific embodiments the present invention is described in further detail:
In the football race, the most tangible sign is exactly the goal near the zone the forbidden zone, if can accurately detect the goal, and on this basis in conjunction with some heuristic rules, but the excellent incident of accurate detection in the football video just.Football video highlight incident extracting method with respect to traditional complexity, the present invention only need occur by detecting the goal, and in conjunction with some simple heuristic rules the football video of complexity is handled, find and locate the generation of excellent incident exactly, obtained good testing result.The complexity of traditional excellent event detecting method, time-consuming drawback had both been avoided, the higher detection of having got back accuracy rate.
The present invention has at first proposed a kind of goal detection method of novelty, promptly extract the edge by the original color video frame images being done the top-hat conversion, on the basis of the edge image that obtains, way by neighborhood search finds the candidate goal, extract goal post by the hough conversion then, utilize at last whether some didactic rule judgment are real goals.Detecting on the basis at goal, find excellent incident by some didactic rules, with in the football match the most attractive attention excellent incident---goal event is an example, when goal event takes place when, what at first occur is near the camera lens in forbidden zone, must have this moment one section long goal camera lens to occur, it is the sportsman's of a period of time feature that shooting finishes the back.Just can accurately detect goal event according to these simple heuristic rules, recall ratio is 91.5%.Excellent incident such as corner-kick, free kick etc. for other have also been obtained the quite good detecting result.
Embodiment 1
Present embodiment is to be example to one section football video image processing process, and the invention will be further described.The present invention mainly was divided into for two steps: detect the appearance at goal, and on the basis of detecting the goal, in conjunction with the excellent incident in some heuristic rules detection football videos.Below each step is described in detail:
1, the goal is detected
The present invention proposes the method at goal in a kind of automatic detection football video.As most important sign in the football field, the present invention extracts two ball door pillars as the sign that finds the goal, and the appearance at goal indicates a kind of potential shooting or goal incident.The present invention can judge in any frame (being mainly distant view image, middle scape image) in the football video whether include the goal, if the goal is arranged, then can obtain the parameter information of two ball door pillars simultaneously, comprises the length information and the positional information of every ball door pillar.
The main thought of this method is at first to be that the original image frame is extracted edge image, on the basis of edge image, each pixel is done neighborhood search on the vertical direction then, the pixel that satisfies prior preset threshold condition is possible goal post pixel (being referred to as candidate's goal post), after obtaining candidate's goal post figure, utilize the hough conversion to find two line segments the longest, finally judge whether it is real goal in conjunction with some didactic rules then as the goal.Describe each step below in detail:
1.1, extract edge image:
This step mainly is the edge image that obtains original image, so that further find real goal on the basis of this edge image.
The present invention adopts the top-hat conversion to extract the colour edging image, and the top-hat conversion is a kind of morphological transformation method of extracting the edge at gray level image, also is called peak detector.Because the color at goal is white, just the gray-scale value of its red (R), green (G), blue (B) three passages is all very high, is equivalent to the crest on the gray scale curved surface, so can play the purpose that strengthens white pixel with the top-hat conversion.In view of this advantage of top-hat conversion, the present invention is used for the edge extracting of coloured image with the top-hat conversion, just can be among the former figure significantly white pixel extract.
The top-hat conversion process of this coloured image is, at first be to original image, do morphologic opening operation, both earlier image was done morphologic erosion operation, on the basis of the corrosion diagram picture that obtains, make the morphology dilation operation, so just obtain the result images of an opening operation.It is poor to do point-to-point pixel with original image and this result images, and the error image that obtains just is the colour edging image of original image.Wherein the convolution template template of opening operation used 5 * 5 is:
For erosion operation wherein, each passage gray-scale value of the RGB of each pixel of corrosion diagram picture is to be the minimum value of corresponding each passage gray-scale value of 25 pixels in 5 * 5 neighborhoods at center with this point in the original image, and for dilation operation, each passage gray-scale value of the RGB of each pixel of expansion plans picture is to be the maximal value of corresponding each passage gray-scale value of 25 pixels in 5 * 5 neighborhoods at center with this point in the corrosion diagram picture.
We must make binary conversion treatment to the colour edging image in order further to extract candidate's goal post, obtain a two-value black white image.Here the present invention adopts adaptive method to obtain a two-value black white image, and its core is for to be divided into two classes with all pixels, and a class is the edge pixel point, and a class is non-edge pixel point.Be specially the gray level image of at first trying to achieve the colour edging image, extract the color histogram of this gray scale edge image then, then the color histogram to this gray level image carries out k (present embodiment is k=2 here) mean cluster, obtain the gray-scale value at two class centers, the higher central value for edge pixel point class of gray-scale value wherein, what gray-scale value was lower is the central value of non-edge pixel point class.Obtain the Euclidean distance of the gray-scale value of each gray values of pixel points and two class centers in the gray level image respectively, if the centre distance of this pixel isolated edge pixel class is near, it is 255 that the gray-scale value that then will put in bianry image is composed, promptly this is the edge pixel point, in bianry image, show as white, if this pixel is near from the centre distance of non-edge pixel point class, it is 0 that the gray-scale value that then will put in bianry image is composed, promptly this is non-edge pixel point, shows as black in bianry image.
1.2, extract candidate's goal post
This step mainly is on the basis of 1.1 binary edge maps that obtain, and finds candidate's goal post by the neighborhood search on the vertical direction.
Be specially each white pixel point on the edge image, do the neighborhood search on the vertical direction.The hunting zone is [10,10], and promptly this white pixel point is up and down in the scope of each 10 pixel.If totally 20 grey scale pixel values of the neighborhood up and down of the vertical direction of this pixel all are 255 (both being white pixel), then this pixel may be the goal post pixel.The gray-scale value of 20 pixels of neighborhood is 255 all about why wanting, and is for fear of the net erroneous judgement is goal post, because the net zone is discontinuous in the white pixel of vertical direction.
1.3, extract the goal
This step mainly is to adopt the hough conversion to extract two line segments the longest as the goal post line from candidate's goal post, uses some heuristic rules then, further judges whether to be real goalpost lines, and the line segment of some error extraction is got rid of.
The present invention adopts the hough conversion of simplification to extract goal post.No matter from what angle shot, goal post is vertical all the time in picture frame in football video.If show with polar coordinates, the polar angle of the line segment of goal post correspondence should be 0 degree.If the coordinate of a pixel in rectangular coordinate system be (i, j), the coordinate in the then corresponding polar coordinate system with it be (ρ, θ), the conversion formula between them is:
ρ=icosθ+jsinθ
Accumulation array corresponding to the Hough conversion is [ρ, θ] like this, is a two-dimensional array.The polar angle of known goal post is 0 degree, and then conversion formula becomes:
ρ=icos0+jsin0=i
Be that goal post line utmost point in polar coordinate system directly is worth the abscissa value for its each point in rectangular coordinate system.The accumulation array of hough conversion just becomes one dimension like this, only needs accumulation ρ value can find two line segments the longest from candidate's goal post.Very big like this computing time of accelerating the hough conversion.
Obtain two line segments the longest by the hough conversion of simplifying, but can not determine still whether these two line segments are final goal posts, this situation that just needs some didactic rules wherein to judge by accident is got rid of.These didactic rules are:
A) length of two door pillars all should meet certain threshold value requirement, promptly should be within rational length.Be limited to 5 pixels under the door pillar height that present embodiment is set, on be limited to 0.3 times of the frame of video height, if do not meet then be not the goal.
B) two door pillars between distance also should meet certain threshold value requirement, promptly should be within rational width.Distance following is limited to 20 pixels between two door pillars that present embodiment is set, on be limited to 0.4 times of the video width, if do not meet then be not the goal.
C) satisfying on the basis of above-mentioned rule, also should satisfy certain proportion requirement between distance between two door pillars and the longest door pillar.The following of scale-up factor is limited to 0.5, on be limited to 2.5, if do not meet then for the goal.
D) satisfying on the basis of above-mentioned rule, for fear of net in the feature or sportsman are treated as door pillar, requiring the door pillar pixel to have certain continuity, promptly too many interruption can not appear in each goal post.The difference that requires the difference of bound of each door pillar and door pillar length is less than 20 pixels, if do not meet then be not the goal.
E) satisfying on the basis of above-mentioned rule, should be within certain scope by the center of the minimum rectangle that constitutes of two door pillars.Because the goal generally appears at the middle and upper part branch of video frame image, so the center of this rectangle should be not less than 0.8 times of picture altitude, if do not meet then be not the goal.
F) when above-mentioned condition all meets, those can occur edge erroneous judgement that staggered sportsman who occurs in distant view extracts is the situation of goal post, appearance for fear of this situation, the distance that must guarantee two door pillars up-and-down boundary in vertical direction meets certain threshold value requirement, promptly also should be within rational distance.Here the distance that has between the lower bound of the upper bound and the goal post with maximum upper bound of goal post of minimum lower bound of present embodiment regulation should be less than 4 pixels, if do not meet then be not the goal.
In conjunction with these didactic rules, finally obtain goal post.
2, excellent event detection
The present invention is detecting on the basis at goal, just can detect excellent incident in the football video accurately in conjunction with some didactic rules.After finding the goal frame, lens type information in conjunction with frame before and after the camera lens of goal, be that distant view, middle scape, feature information just can detect dissimilar excellent incidents, the extraction of these lens type information can be adopted existing technology, the lens type extracting method of mentioning in can " Automatic Soccer Video Analysis andSummarization " referring to people such as Ekin.
The present invention with in the football match the goal event of attractive attention be that example specifies the excellent event detecting method that detects based on the goal.When goal event takes place when, near zone, the forbidden zone of Chu Xianing at first, this is generally the long shot that comprises clear goal, and after such camera lens continued for some time, goal event was finished, and can occur sportsman or referee's close-up shot usually.The flow process that goal event detects as shown in Figure 1, video flowing sequence for input, at first detect and whether have the goal in the present frame, if have the goal and occur the above long shot that includes the goal of 100 frames continuously, then might be a goal event, whether the frame of video that continues to detect thereafter backward exists close-up shot, if close-up shot outnumber 40 frames, then this section video flowing is an excellent incident of shooting, otherwise the video flowing that continues is rearwards searched excellent goal event.
Embodiment 2
Detect on the basis at goal at embodiment 1,, then can detect the corner-kick incident in conjunction with the detection of corner-kick position.
It should be noted last that above embodiment is only unrestricted in order to technical scheme of the present invention to be described.Although the present invention is had been described in detail with reference to embodiment, those of ordinary skill in the art is to be understood that, technical scheme of the present invention is made amendment or is equal to replacement, do not break away from the spirit and scope of technical solution of the present invention, it all should be encompassed in the middle of the claim scope of the present invention.