CN105721860A - Video lens boundary detection method based on hypergraph - Google Patents

Abstract

The invention discloses a video lens boundary detection method based on hypergraph. The video lens boundary detection method is characterized in that video characteristics can be extracted, and a k neighbor hypergraph model can be constructed; the video lens boundary can be searched; a minimum scoring threshold of a scoring vector can be set according to the k neighbor hypergraph; the label vector of the searching vertex is 1, and the label vectors of other vertexes are 0; the scoring vectors of the vertexes can be calculated, and the continuous vertex positions of the scoring vectors greater than the minimum scoring threshold can be recorded, and less than a half of vertexes can be selected randomly, and can be recorded as the set, and the label vector is 1; the scoring vectors of the vertexes can be recalculated according to the label vector; the process of setting the label vector of the searching vertex as 1, and setting the label vectors of other vertexes as 0 can be repeated; the video lens boundary and the type can be determined; the process of setting the label vector of the searching vertex as 1, and setting the label vectors of other vertexes as 0 can be repeated, till all of the video lens boundaries are completely determined. The retrieval of the video frames can be carried out by the hypergraph model, and the hypergraph model is used for the video lens boundary detection field.

Description

A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph

Technical field

The present invention relates to a kind of Methods for Shot Boundary Detection of Video Sequences.Particularly relate to the characteristic of seriality that a kind of content in same video lens has and similarity, by hypergraph model to the analysis of picture frame in video lens, it is determined that the Methods for Shot Boundary Detection of Video Sequences based on hypergraph of each shot boundary of video.

Background technology

Video lens is often referred to the video segment of video camera one-time continuous shooting, and video shot boundary is often referred between video lens consecutive frame and occurs in that change in some sense.Video shot boundary detection is used to find a kind of technology on border between multiple cinestrip.When two video lens change, it will usually occur that some significantly change, for instance the change etc. of color characteristic.

Video shot boundary generally comprises two types.One is abrupt shot (abruptshot), refers to that frame of video jumps to suddenly another camera lens from a camera lens.Abrupt shot usually occurs between two frames, and front and back two frame is belonging respectively to former and later two camera lenses.Another kind is gradual shot (gradualshot), refers to that frame of video is transitioned into another camera lens gradually slowly from a camera lens, and camera lens has a kind of edit effect over time and space.Gradual shot generally includes and is fade-in gradually to go out (fadeinandfadeout), dissolve (dissolve) etc..Gradual shot usually occurs in a few frame between tens frames, is the transition of former and later two camera lenses.Wherein abrupt shot is easier to detect, and the more difficult detection of gradual shot, it is the emphasis of shot boundary detector.

Video shot boundary detection mainly includes the similarity measurement between the feature extraction of frame of video, video features, shot boundary determines three steps.Existing video shot boundary detection technique mainly has the method based on edge and the method based on movable information.The edge of image and gradient information can well show the visual information of image, therefore can use as the feature of video image.Shot detection method based on edge is generally more sensitive to the motion ratio of camera, object, is not therefore very accurate to the detection of gradual shot.Being based in camera lens frame of video based on the method for movable information is basic assumption smooth, that boundary is sudden change.Therefore based on the method for movable information, the detection of abrupt shot is comparatively accurate, the detection for gradual shot neither be very accurate.Method based on movable information also has the shortcoming that calculating time complexity is high.The maximum challenge of current video shot boundary detection technique is how to detect gradual shot and how to eliminate the impact on video shot boundary place of the high-speed motion of illumination or camera and object.

Summary of the invention

The technical problem to be solved is to provide a kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph that hypergraph model can be utilized to determine video shot boundary one by one.

The technical solution adopted in the present invention is: a kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph, comprises the steps:

1) video features is extracted；

2) k neighbour's hypergraph model is built:

$H(v,e)=\left\{\begin{array}{ccc}1,& if& v\∈e\\ 0,& if& v\∉e\end{array}\right.---\left(1\right)$

Wherein, v represents the summit of hypergraph, and e represents the super limit of hypergraph, and (v, e) for incidence matrix for H；

3) video shot boundary is found, including:

(1) first according to k neighbour's hypergraph model, when calculating a given inquiry summit, other summits are relative to the score vector on this summit:

F=(1-γ) (I-γ Θ)^-1y(2)

Wherein, γ is constant factor；I is unit matrix；Y is the label vector on n summit of hypergraph, and dimension is n × 1 dimension, and f is the score vector relative to query point, and dimension is n × 1 dimension；Θ is the Laplacian Matrix of n × n, Θ=D_v ^-1/2HWD_e ^-1H^TD_v ^-1/2, wherein, W is weight matrix, is the weight matrix being constituted super limit with the weight w (e) on limit for diagonal:

$w\left(e\right)={\mathrm{\Σ}}_{v_i,v_j\∈e}\left|\right|v_i-v_j|{|}_2---\left(3\right)$

D_vFor the degree matrix on summit, it is the degree matrix being constituted summit with degree d (v) on summit for diagonal:

D (v)=∑_e∈Ew(e)·H(v,e)(4)

D_eFor the degree matrix on super limit, it is the degree matrix being constituted super limit with degree d (e) on limit for diagonal:

D (e)=∑_v∈eH(v,e)(5)

(2) the minimum score threshold δ of score vector f is set；

(3) label vector y (j)=1 on order inquiry summit, the label vector on other summits is set to 0；

(4) calculate each summit score vector f, and record the continuous vertex position of the f > δ comprising jth position, and from described continuous vertex position, randomly select the summit within half as feedback point, be designated as set F '_k, and make label vector y (F '_k)=1, k is the label of set；

(5) according to label vector y (F '_k)=1, calculates each summit score vector f again, and record comprises set F '_kThe continuous vertex position of f > δ, be designated as set F_k, now F_kRepresent all frames in the same camera lens comprising jth position；

(6) j=F is made_k(last)+1, k=k+1, label vector y (j)=1 is set, calculate each summit score vector f, and record the continuous vertex position of the f > δ comprising jth position, and from described continuous vertex position, randomly select the summit within half as feedback point, it is designated as set F '_k+1, and make label vector y (F '_k+1)=1, according to label vector y (F '_k+1)=1, calculates each summit score vector f again, and record comprises set F '_k+1The continuous vertex position of f > δ, obtain set F_k+1, wherein, F_k(last) for set F_kLast summit；

(7) video shot boundary and type are determined；

(8) F is made_k=F_k+1, return (6th) step, until all video shot boundaries are determined complete.

Step 3) in (3rd) step represent summit label vector y=[0 ..., 1 ... 0]^T, wherein 1 in jth position.

Step 3) in determination video shot boundary described in (7th) step and type include:

Take set F_kWith set F_k+1Common factor F_△, i.e. F_△=F_k∩F_k+1If,Then it is determined that at F_kAnd F (last)_k+1(first) some place is a shot boundary, and this shot boundary is abrupt shot, i.e. F_kAnd F (last)_k+1(first) up-and-down boundary of respectively abrupt shot is put；IfSo may determine that at F_△And F (first)_△(last) some place is a shot boundary, and this shot boundary is gradual shot, i.e. F_△And F (first)_△(last) up-and-down boundary of respectively gradual shot is put, wherein, F_k+1(first) for set F_k+1First summit, F_△(first) for set F_△First summit, F_△(last) for set F_△Last summit.

A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph of the present invention, by the hypergraph model retrieval to frame of video, is used for hypergraph model the detection field of video shot boundary, has following distinguishing feature:

1, hypergraph model is applied to above shot boundary detector by the present invention first, it is by all frame of video in a camera lens are all retrieved thus determining video shot boundary, the different and conventional thinking being found shot boundary by the change of shot boundary place color or movable information.

2, experiments verify that, video shot boundary can be detected by the present invention fast and effectively, is therefore a kind of effective video lens boundary detection method.

3, the present invention is simple, excellent effect.It is usable in the pretreatment stage in the fields such as video analysis, video frequency abstract, video frequency searching.

Accompanying drawing explanation

Fig. 1 is the present invention flow chart based on the Methods for Shot Boundary Detection of Video Sequences of hypergraph；

Fig. 2 is an abrupt shot (frame of video is continuous videos), the wherein border of 5 frames and 6 frames respectively former and later two camera lenses；

Fig. 3 is a gradual shot (frame of video is continuous videos), and wherein 6 frames are the bounds of former and later two camera lenses to 9 frames.

Detailed description of the invention

Below in conjunction with embodiment and accompanying drawing, a kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph of the present invention is described in detail.

A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph of the present invention, comprises the steps:

1) video features is extracted；

2) k neighbour's hypergraph model is built:

$H(v,e)=\left\{\begin{array}{ccc}1,& if& v\∈e\\ 0,& if& v\∉e\end{array}\right.---\left(1\right)$

Wherein, v represents the summit of hypergraph, and e represents the super limit of hypergraph, and (v, e) for incidence matrix for H；

3) video shot boundary is found, including:

(1) first according to k neighbour's hypergraph model, when calculating a given inquiry summit, other summits are relative to the score vector on this summit:

F=(1-γ) (I-γ Θ)^-1y(2)

Wherein, γ is constant factor；I is unit matrix；Y is the label vector on n summit of hypergraph, and dimension is n × 1 dimension, and f is the score vector relative to query point, and dimension is n × 1 dimension；Θ is the Laplacian Matrix of n × n, Θ=D_v ^-1/2HWD_e ^-1H^TD_v ^-1/2, wherein, W is weight matrix, is the weight matrix being constituted super limit with the weight w (e) on limit for diagonal:

$w\left(e\right)={\mathrm{\Σ}}_{v_i,v_j\∈e}\left|\right|v_i-v_j|{|}_2---\left(3\right)$

D_vFor the degree matrix on summit, it is the degree matrix being constituted summit with degree d (v) on summit for diagonal:

D (v)=∑_e∈Ew(e)·H(v,e)(4)

D_eFor the degree matrix on super limit, it is the degree matrix being constituted super limit with degree d (e) on limit for diagonal:

D (e)=∑_v∈eH(v,e)(5)

(2) the minimum score threshold δ of score vector f is set；

(3) order inquiry summit label vector y (j)=1, the label vector on other summits is set to 0, represent summit label vector y=[0 ..., 1 ... 0]^T, wherein 1 in jth position；

(4) calculate each summit score vector f, and record the continuous vertex position of the f > δ comprising jth position, and from described continuous vertex position, randomly select the summit within half as feedback point, be designated as set F '_k, and make label vector y (F '_k)=1, k is the label of set；

(5) according to label vector y (F '_k)=1, calculates each summit score vector f again, and record comprises set F '_kThe continuous vertex position of f > δ, be designated as set F_k, now F_kRepresent all frames in the same camera lens comprising jth position；

(6) j=F is made_k(last)+1, k=k+1, label vector y (j)=1 is set, calculate each summit score vector f, and record the continuous vertex position of the f > δ comprising jth position, and from described continuous vertex position, randomly select the summit within half as feedback point, it is designated as set F '_k+1, and make label vector y (F '_k+1)=1, according to label vector y (F '_k+1)=1, calculates each summit score vector f again, and record comprises set F '_k+1The continuous vertex position of f > δ, obtain set F_k+1, wherein, F_k(last) for set F_kLast summit；

(7) video shot boundary and type are determined, including:

Take set F_kWith set F_k+1Common factor F_△, i.e. F_△=F_k∩F_k+1If,Then it is determined that at F_kAnd F (last)_k+1(first) some place is a shot boundary, and this shot boundary is abrupt shot, i.e. F_kAnd F (last)_k+1(first) up-and-down boundary of respectively abrupt shot is put；IfSo may determine that at F_△And F (first)_△(last) some place is a shot boundary, and this shot boundary is gradual shot, i.e. F_△And F (first)_△(last) up-and-down boundary of respectively gradual shot is put, wherein, F_k+1(first) for set F_k+1First summit, F_△(first) for set F_△First summit, F_△(last) for set F_△Last summit；

(8) F is made_k=F_k+1, return (6th) step, until all video shot boundaries are determined complete.

Fig. 2 and Fig. 3 is the judgement example of abrupt shot and gradual shot.Wherein Fig. 2 is abrupt shot, and the result of note query point 1 is set F₁={ 1,2,3,4,5}；So note query point F₁(last) result of+1 (namely putting 6) is set F₂={ 6,7,8,9,10}；So can be determined that a F₁(last)=5 and some F₂(first)=6 is the bound of abrupt shot；Fig. 3 is gradual shot, and the result of note query point 1 is set F₁={ 1,2,3,4,5,6,7,8,9}；So note query point F₁(last) result of+1 (namely putting 10) is set F₂={ 6,7,8,9,10,11,12,13,14}；F_△=F₁∩F₂={ 6,7,8,9}；So can be determined that a F_△(first)=6 and some F_△(last)=9 is the bound of gradual shot.

A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph of the present invention, in the specific implementation, it is possible to be divided into coarseness range detection and fine granularity range detection.First pass through coarseness range detection and determine the probable ranges of video boundaries, save and calculate the time in a large number；Then pass through fine granularity range detection and carry out precise video shot boundary scope.

Coarseness range detection step is as follows:

1) frame of video is sampled, for instance two frames of can sampling for 1 second；

2) feature of video is extracted；

3) k neighbour's hypergraph model is built；

4) perform according to lens boundary detection method step, and according to F_△Judge lens type and bounds.

The detection of above-mentioned coarseness has simply detected in sampling interframe, only determines a probable ranges, then also needs to carry out fine granularity range detection, and fine granularity range detection includes:

(1) all frame of video between sampling coarseness range detection result；

(2) repeat coarseness range detection step 2), 3), 4), thus obtaining accurate video shot boundary scope.

Claims (3)

1. the Methods for Shot Boundary Detection of Video Sequences based on hypergraph, it is characterised in that comprise the steps:

1) video features is extracted；

2) k neighbour's hypergraph model is built:

$H(v,e)=\left\{\begin{array}{ccc}1,& if& v\∈e\\ 0,& if& v\∉e\end{array}\right.---\left(1\right)$

Wherein, v represents the summit of hypergraph, and e represents the super limit of hypergraph, and (v, e) for incidence matrix for H；

3) video shot boundary is found, including:

(1) first according to k neighbour's hypergraph model, when calculating a given inquiry summit, other summits are relative to the score vector on this summit:

F=(1-γ) (I-γ Θ)^-1y(2)

Wherein, γ is constant factor；I is unit matrix；Y is the label vector on n summit of hypergraph, and dimension is n × 1 dimension, and f is the score vector relative to query point, and dimension is n × 1 dimension；Θ is the Laplacian Matrix of n × n, Θ=D_v ^-1/2HWD_e ^-1H^TD_v ^-1/2, wherein, W is weight matrix, is the weight matrix being constituted super limit with the weight w (e) on limit for diagonal:

$w\left(e\right)={\mathrm{\Σ}}_{v_i,v_j\∈e}\left|\right|v_i-v_j|{|}_2---\left(3\right)$

D_vFor the degree matrix on summit, it is the degree matrix being constituted summit with degree d (v) on summit for diagonal:

D (v)=∑_e∈Ew(e)·H(v,e)(4)

D_eFor the degree matrix on super limit, it is the degree matrix being constituted super limit with degree d (e) on limit for diagonal:

D (e)=∑_v∈eH(v,e)(5)

(2) the minimum score threshold δ of score vector f is set；

(3) label vector y (j)=1 on order inquiry summit, the label vector on other summits is set to 0；

(4) calculate each summit score vector f, and record the continuous vertex position of the f > δ comprising jth position, and from described continuous vertex position, randomly select the summit within half as feedback point, be designated as set F '_k, and make label vector y (F '_k)=1, k is the label of set；

(5) according to label vector y (F '_k)=1, calculates each summit score vector f again, and record comprises set F '_kThe continuous vertex position of f > δ, be designated as set F_k, now F_kRepresent all frames in the same camera lens comprising jth position；

(6) j=F is made_k(last)+1, k=k+1, label vector y (j)=1 is set, calculate each summit score vector f, and record the continuous vertex position of the f > δ comprising jth position, and from described continuous vertex position, randomly select the summit within half as feedback point, it is designated as set F '_k+1, and make label vector y (F '_k+1)=1, according to label vector y (F '_k+1)=1, calculates each summit score vector f again, and record comprises set F '_k+1The continuous vertex position of f > δ, obtain set F_k+1, wherein, F_k(last) for set F_kLast summit；

(7) video shot boundary and type are determined；

(8) F is made_k=F_k+1, return (6th) step, until all video shot boundaries are determined complete.

2. a kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph according to claim 1, it is characterised in that step 3) in (3rd) step represent summit label vector y=[0 ..., 1 ... 0]^T, wherein 1 in jth position.

3. a kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph according to claim 1, it is characterised in that step 3) in determination video shot boundary described in (7th) step and type include:

Take set F_kWith set F_k+1Common factor F_Δ, i.e. F_Δ=F_k∩F_k+1If,Then it is determined that at F_kAnd F (last)_k+1(first) some place is a shot boundary, and this shot boundary is abrupt shot, i.e. F_kAnd F (last)_k+1(first) up-and-down boundary of respectively abrupt shot is put；IfSo may determine that at F_ΔAnd F (first)_Δ(last) some place is a shot boundary, and this shot boundary is gradual shot, i.e. F_ΔAnd F (first)_Δ(last) up-and-down boundary of respectively gradual shot is put, wherein, F_k+1(first) for set F_k+1First summit, F_Δ(first) for set F_ΔFirst summit, F_Δ(last) for set F_ΔLast summit.