CN105721860B - A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph - Google Patents
A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph Download PDFInfo
- Publication number
- CN105721860B CN105721860B CN201610037058.2A CN201610037058A CN105721860B CN 105721860 B CN105721860 B CN 105721860B CN 201610037058 A CN201610037058 A CN 201610037058A CN 105721860 B CN105721860 B CN 105721860B
- Authority
- CN
- China
- Prior art keywords
- summit
- mrow
- vector
- video
- shot
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N17/00—Diagnosis, testing or measuring for television systems or their details
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
Abstract
A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph:Extract video features;Build k neighbour's hypergraph models;Video shot boundary is found, including:The minimum score threshold of score vector is set according to k neighbours hypergraph model;The label vector on order inquiry summit is 1, and the label vector on other summits is arranged to 0;Each summit score vector is calculated, and records score vector more than the continuous vertex position in minimum score threshold, and randomly selects the summit within half and is designated as gathering, and it is 1 to make label vector;According to label vector, each summit score vector is calculated again;Label vector from order inquiry summit is 1, and the label vector on other summits is arranged to 0 and repeated;Determine video shot boundary and type;The label vector from order inquiry summit is 1 again, and the label vector on other summits is arranged to 0 and repeated, until all video shot boundaries determine to finish.The present invention is by retrieval of the hypergraph model to frame of video, by detection field of the hypergraph model for video shot boundary.
Description
Technical field
The present invention relates to a kind of Methods for Shot Boundary Detection of Video Sequences.More particularly to a kind of with content in same video lens
The continuity and the characteristic of similitude having, by analysis of the hypergraph model to picture frame in video lens, determine that video is each
The Methods for Shot Boundary Detection of Video Sequences based on hypergraph of shot boundary.
Background technology
Video lens are often referred to the video segment of video camera one-time continuous shooting, and video shot boundary is often referred to video lens
Occurs change in some sense between consecutive frame.Video shot boundary detection is for finding side between multiple cinestrip
A kind of technology on boundary.When two video lens change, it will usually there are some and significantly change, such as color characteristic
Change etc..
Video shot boundary generally comprises two types.One kind is abrupt shot (abrupt shot), refers to frame of video from one
Individual camera lens jumps to another camera lens suddenly.Abrupt shot generally occurs between two frames, and front and rear two frame is belonging respectively to former and later two
Camera lens.Another kind is gradual shot (gradual shot), refers to frame of video and is gradually slowly transitioned into another from a camera lens
Camera lens, camera lens have a kind of edit effect over time and space.Gradual shot, which generally includes to be fade-in, gradually goes out (fade in
And fade out), dissolve (dissolve) etc..Gradual shot generally occurs in several frames between more than ten frames, is former and later two
The transition of camera lens.Wherein abrupt shot is easier to detect, and the more difficult detection of gradual shot, is the emphasis of shot boundary detector.
Video shot boundary detection mainly includes similarity measurement, the camera lens between the feature extraction of frame of video, video features
Border determines three steps.Existing video shot boundary detection technique mainly has method based on edge and based on movable information
Method.The edge and gradient information of image can be very good to show the visual information of image, therefore can be used as video image
Feature use.Shot detection method based on edge is generally more sensitive to the motion ratio of camera, object, therefore to gradual change
The detection of camera lens is not very accurate.It based on frame of video in camera lens is smooth that method based on movable information, which is, boundary is
The basic assumption of mutation.Therefore it is more accurate based on detection of the method for movable information to abrupt shot, for gradual shot
Detection is nor very accurate.Method based on movable information, which also has, calculates the shortcomings that time complexity is high.Video lens at present
The maximum challenge of bound test technology is the high speed for how detecting gradual shot and how eliminating illumination or camera and object
Motion is to the influence at video shot boundary.
The content of the invention
The technical problem to be solved by the invention is to provide one kind to determine video mirror one by one in front using hypergraph model
The Methods for Shot Boundary Detection of Video Sequences based on hypergraph on boundary.
The technical solution adopted in the present invention is:A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph, including it is as follows
Step:
1) video features are extracted;
2) k neighbour's hypergraph models are built:
Wherein, v represents the summit of hypergraph, and e represents the super side of hypergraph, and H (v, e) is incidence matrix;
3) video shot boundary is found, including:
(1) first according to k neighbour's hypergraph models, when calculating given inquiry summit, other summits are relative to the summit
Score vector:
F=(1- γ) (I- γ Θ)-1y (2)
Wherein, γ is constant factor;I is unit matrix;Y is the label vector on n summit of hypergraph, and dimension is that n × 1 is tieed up, f
For relative to the score vector of query point, dimension is that n × 1 is tieed up;Θ be n × n Laplacian Matrix, Θ=Dv -1/2HWDe - 1HTDv -1/2, wherein, W is weight matrix, be with while weight w (e) be diagonal is formed surpass while weight matrix:
DvIt is the degree matrix for being formed summit for diagonal with the degree d (v) on summit for the degree matrix on summit:
D (v)=∑e∈Ew(e)·H(v,e) (4)
DeFor the degree matrix on super side, be with while degree d (e) be that diagonal is formed degree matrix when surpassing:
D (e)=∑v∈eH(v,e) (5)
(2) the minimum score threshold δ of score vector f is set;
(3) label vector y (j)=1 on order inquiry summit, the label vector on other summits are arranged to 0;
(4) each summit score vector f is calculated, and is recorded comprising the f including j-th of position>δ continuous vertex position,
And the summit within half is randomly selected from described continuous vertex position as feedback point, it is designated as set F 'k, and make label
Vectorial y (F 'k)=1, k is the label of set;
(5) according to label vector y (F 'k)=1, each summit score vector f is calculated again, and record and include set F 'k's
f>δ continuous vertex position, is designated as set Fk, now FkRepresent all frames in the same camera lens comprising j-th of position;
(6) j=F is madek(last)+1, k=k+1, label vector y (j)=1 is set, calculates each summit score vector f, and
And record includes the f including j-th of position>δ continuous vertex position, and randomly select one from described continuous vertex position
Summit within half is designated as set F ' as feedback pointk+1, and make label vector y (F 'k+1)=1, according to label vector y
(F′k+1)=1, each summit score vector f is calculated again, and record and include set F 'k+1F>δ continuous vertex position, is obtained
To set Fk+1, wherein, Fk(last) it is set FkLast summit;
(7) video shot boundary and type are determined;
(8) F is madek=Fk+1, (6) step is returned to, until all video shot boundaries determine to finish.
In step 3) (3) step represent summit label vector y=[0 ..., 1 ... 0]T, wherein 1 j-th of position.
Determination video shot boundary and type in step 3) described in (7) step include:
Take set FkWith set Fk+1Common factor F△, i.e. F△=Fk∩Fk+1IfThen it is determined that in Fk(last)
And Fk+1(first) it is a shot boundary at point, and the shot boundary is abrupt shot, i.e. FkAnd F (last)k+1
(first) point is respectively the up-and-down boundary of abrupt shot;IfIt can so determine in F△And F (first)△
(last) it is a shot boundary at point, and the shot boundary is gradual shot, i.e. F△And F (first)△(last) point difference
For the up-and-down boundary of gradual shot, wherein, Fk+1(first) it is set Fk+1First summit, F△(first) it is set F△'s
First summit, F△(last) it is set F△Last summit.
A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph of the present invention, passes through inspection of the hypergraph model to frame of video
Rope, hypergraph model is used for the detection field of video shot boundary, there is following distinguishing feature:
1st, hypergraph model is applied to above shot boundary detector by the present invention first, is by being regarded all in a camera lens
Frequency frame is all retrieved so as to determine video shot boundary, different with passing through color at shot boundary or movable information in the past
Change to find the thinking of shot boundary.
2nd, experiments verify that, the present invention fast and effectively can detect video shot boundary, therefore be that one kind has
The Methods for Shot Boundary Detection of Video Sequences of effect.
3rd, simple and easy, excellent effect of the invention.The fields such as video analysis, video frequency abstract, video frequency searching can be used in
Pretreatment stage.
Brief description of the drawings
Fig. 1 is the flow chart of the Methods for Shot Boundary Detection of Video Sequences of the invention based on hypergraph;
Fig. 2 is an abrupt shot (frame of video is continuous videos), wherein 5 frames and 6 frames are respectively former and later two camera lenses
Border;
Fig. 3 is a gradual shot (frame of video is continuous videos), wherein 6 frames to 9 frames are the border of former and later two camera lenses
Scope.
Embodiment
A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph of the present invention is made with reference to embodiment and accompanying drawing
Describe in detail.
A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph of the present invention, comprises the following steps:
1) video features are extracted;
2) k neighbour's hypergraph models are built:
Wherein, v represents the summit of hypergraph, and e represents the super side of hypergraph, and H (v, e) is incidence matrix;
3) video shot boundary is found, including:
(1) first according to k neighbour's hypergraph models, when calculating given inquiry summit, other summits are relative to the summit
Score vector:
F=(1- γ) (I- γ Θ)-1y (2)
Wherein, γ is constant factor;I is unit matrix;Y is the label vector on n summit of hypergraph, and dimension is that n × 1 is tieed up, f
For relative to the score vector of query point, dimension is that n × 1 is tieed up;Θ be n × n Laplacian Matrix, Θ=Dv -1/2HWDe - 1HTDv -1/2, wherein, W is weight matrix, be with while weight w (e) be diagonal is formed surpass while weight matrix:
DvIt is the degree matrix for being formed summit for diagonal with the degree d (v) on summit for the degree matrix on summit:
D (v)=∑e∈Ew(e)·H(v,e) (4)
DeFor the degree matrix on super side, be with while degree d (e) be that diagonal is formed degree matrix when surpassing:
D (e)=∑v∈eH(v,e) (5)
(2) the minimum score threshold δ of score vector f is set;
(3) label vector y (j)=1 on order inquiry summit, the label vector on other summits are arranged to 0, represent the mark on summit
Sign vectorial y=[0 ..., 1 ... 0]T, wherein 1 j-th of position;
(4) each summit score vector f is calculated, and is recorded comprising the f including j-th of position>δ continuous vertex position,
And the summit within half is randomly selected from described continuous vertex position as feedback point, it is designated as set F 'k, and make label
Vectorial y (F 'k)=1, k is the label of set;
(5) according to label vector y (F 'k)=1, each summit score vector f is calculated again, and record and include set F 'k's
f>δ continuous vertex position, is designated as set Fk, now FkRepresent all frames in the same camera lens comprising j-th of position;
(6) j=F is madek(last)+1, k=k+1, label vector y (j)=1 is set, calculates each summit score vector f, and
And record includes the f including j-th of position>δ continuous vertex position, and randomly select one from described continuous vertex position
Summit within half is designated as set F ' as feedback pointk+1, and make label vector y (F 'k+1)=1, according to label vector y
(F′k+1)=1, each summit score vector f is calculated again, and record and include set F 'k+1F>δ continuous vertex position, is obtained
To set Fk+1, wherein, Fk(last) it is set FkLast summit;
(7) video shot boundary and type are determined, including:
Take set FkWith set Fk+1Common factor F△, i.e. F△=Fk∩Fk+1IfThen it is determined that in Fk(last)
And Fk+1(first) it is a shot boundary at point, and the shot boundary is abrupt shot, i.e. FkAnd F (last)k+1
(first) point is respectively the up-and-down boundary of abrupt shot;IfIt can so determine in F△And F (first)△
(last) it is a shot boundary at point, and the shot boundary is gradual shot, i.e. F△And F (first)△(last) point difference
For the up-and-down boundary of gradual shot, wherein, Fk+1(first) it is set Fk+1First summit, F△(first) it is set F△'s
First summit, F△(last) it is set F△Last summit;
(8) F is madek=Fk+1, (6) step is returned to, until all video shot boundaries determine to finish.
Fig. 2 and Fig. 3 is the judgement example of abrupt shot and gradual shot.Wherein Fig. 2 is abrupt shot, note query point 1
As a result it is set F1={ 1,2,3,4,5 };So remember query point F1(last) result of+1 (putting 6) is set F2=6,7,8,
9,10};It so can be determined that point F1=5 and point F (last)2(first) it is=6 the upper and lower of abrupt shot
Boundary;Fig. 3 is gradual shot, and the result of note query point 1 is set F1={ 1,2,3,4,5,6,7,8,9 };So remember query point F1
(last) result of+1 (putting 10) is set F2={ 6,7,8,9,10,11,12,13,14 };F△=F1∩F2=6,7,8,
9};It so can be determined that point F△=6 and point F (first)△(last)=9 it is the bound of gradual shot.
A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph of the present invention, in the specific implementation, can be divided into coarse grain
Spend range detection and fine granularity range detection.The probable ranges of video boundaries are determined by coarseness range detection first, are saved
It is a large amount of to calculate the time;Then by fine granularity range detection come precise video shot boundary scope.
Coarseness range detection step is as follows:
1) frame of video is sampled, such as can be with 1 second two frame of sampling;
2) feature of video is extracted;
3) k neighbour's hypergraph models are built;
4) performed according to lens boundary detection method step, and according to F△To judge lens type and bounds.
Above-mentioned coarseness detection is simply detected in sampling interframe, and a probable ranges are only determined, then also need
Fine granularity range detection is carried out, fine granularity range detection includes:
(1) all frame of video between coarseness range detection result are sampled;
(2) repeat coarseness range detection step 2), 3), 4), so as to obtain accurate video shot boundary scope.
Claims (2)
1. a kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph, it is characterised in that comprise the following steps:
1) video features are extracted;
2) k neighbour's hypergraph models are built:
<mrow>
<mi>H</mi>
<mrow>
<mo>(</mo>
<mi>v</mi>
<mo>,</mo>
<mi>e</mi>
<mo>)</mo>
</mrow>
<mo>=</mo>
<mfenced open = "{" close = "">
<mtable>
<mtr>
<mtd>
<mrow>
<mn>1</mn>
<mo>,</mo>
</mrow>
</mtd>
<mtd>
<mrow>
<mi>i</mi>
<mi>f</mi>
</mrow>
</mtd>
<mtd>
<mrow>
<mi>v</mi>
<mo>&Element;</mo>
<mi>e</mi>
</mrow>
</mtd>
</mtr>
<mtr>
<mtd>
<mrow>
<mn>0</mn>
<mo>,</mo>
</mrow>
</mtd>
<mtd>
<mrow>
<mi>i</mi>
<mi>f</mi>
</mrow>
</mtd>
<mtd>
<mrow>
<mi>v</mi>
<mo>&NotElement;</mo>
<mi>e</mi>
</mrow>
</mtd>
</mtr>
</mtable>
</mfenced>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
</mrow>
Wherein, v represents the summit of hypergraph, and e represents the super side of hypergraph, and H (v, e) is incidence matrix;
3) video shot boundary is found, including:
(1) first according to k neighbour's hypergraph models, when calculating given inquiry summit, other summits obtain relative to the summit
Divide vector:
F=(1- γ) (I- γ Θ)-1y (2)
Wherein, γ is constant factor;I is unit matrix;Y is the label vector on n summit of hypergraph, and dimension is that n × 1 is tieed up, and f is phase
For the score vector of query point, dimension is that n × 1 is tieed up;Θ be n × n Laplacian Matrix, Θ=Dv -1/2HWDe -1HTDv -1/2,
Wherein, W is weight matrix, be with while weight w (e) be diagonal is formed surpass while weight matrix:
<mrow>
<mi>w</mi>
<mrow>
<mo>(</mo>
<mi>e</mi>
<mo>)</mo>
</mrow>
<mo>=</mo>
<msub>
<mi>&Sigma;</mi>
<mrow>
<msub>
<mi>v</mi>
<mi>i</mi>
</msub>
<mo>,</mo>
<msub>
<mi>v</mi>
<mi>j</mi>
</msub>
<mo>&Element;</mo>
<mi>e</mi>
</mrow>
</msub>
<mo>|</mo>
<mo>|</mo>
<msub>
<mi>v</mi>
<mi>i</mi>
</msub>
<mo>-</mo>
<msub>
<mi>v</mi>
<mi>j</mi>
</msub>
<mo>|</mo>
<msub>
<mo>|</mo>
<mn>2</mn>
</msub>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>3</mn>
<mo>)</mo>
</mrow>
</mrow>
DvIt is the degree matrix for being formed summit for diagonal with the degree d (v) on summit for the degree matrix on summit:
D (v)=∑e∈Ew(e)·H(v,e) (4)
DeFor the degree matrix on super side, be with while degree d (e) be that diagonal is formed degree matrix when surpassing:
D (e)=∑v∈eH(v,e) (5)
(2) the minimum score threshold δ of score vector f is set;
(3) label vector y (j)=1 on order inquiry summit, the label vector on other summits are arranged to 0;
(4) each summit score vector f is calculated, and is recorded comprising the f including j-th of position>δ continuous vertex position, and from
The summit within half is randomly selected in described continuous vertex position as feedback point, is designated as set F 'k, and make label vector
y(F′k)=1, k is the label of set;
(5) according to label vector y (F 'k)=1, each summit score vector f is calculated again, and record and include set F 'kF>δ
Continuous vertex position, be designated as set Fk, now FkRepresent all frames in the same camera lens comprising j-th of position;
(6) j=F is madek(last)+1, k=k+1, label vector y (j)=1 is set, calculates each summit score vector f, and record
Include the f including j-th of position>δ continuous vertex position, and randomly selected from described continuous vertex position within half
Summit as feedback point, be designated as set F 'k+1, and make label vector y (F 'k+1)=1, according to label vector y (F 'k+1)=1,
Each summit score vector f is calculated again, and is recorded and included set F 'k+1F>δ continuous vertex position, obtains set Fk+1,
Wherein, Fk(last) it is set FkLast summit;
(7) video shot boundary and type are determined, including:
Take set FkWith set Fk+1Common factor F△, i.e. F△=Fk∩Fk+1IfThen it is determined that in FkAnd F (last)k+1
(first) it is a shot boundary at point, and the shot boundary is abrupt shot, i.e. FkAnd F (last)k+1(first) point minute
Not Wei abrupt shot up-and-down boundary;IfIt can so determine in F△And F (first)△(last) it is one at point
Shot boundary, and the shot boundary is gradual shot, i.e. F△And F (first)△(last) point is respectively the upper and lower of gradual shot
Border, wherein, Fk+1(first) it is set Fk+1First summit, F△(first) it is set F△First summit, F△
(last) it is set F△Last summit;
(8) F is madek=Fk+1, (6) step is returned to, until all video shot boundaries determine to finish.
A kind of 2. Methods for Shot Boundary Detection of Video Sequences based on hypergraph according to claim 1, it is characterised in that step 3)
In (3) step represent summit label vector y=[0 ..., 1 ... 0]T, wherein 1 j-th of position.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610037058.2A CN105721860B (en) | 2016-01-20 | 2016-01-20 | A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610037058.2A CN105721860B (en) | 2016-01-20 | 2016-01-20 | A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph |
Publications (2)
Publication Number | Publication Date |
---|---|
CN105721860A CN105721860A (en) | 2016-06-29 |
CN105721860B true CN105721860B (en) | 2017-12-08 |
Family
ID=56147431
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201610037058.2A Active CN105721860B (en) | 2016-01-20 | 2016-01-20 | A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN105721860B (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110874833B (en) * | 2019-10-15 | 2022-03-22 | 合肥工业大学 | SAR image change detection method based on hypergraph matching |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7177470B2 (en) * | 2002-11-13 | 2007-02-13 | Koninklijke Philips Electronics N. V. | Method of and system for detecting uniform color segments |
CN101650830B (en) * | 2009-08-06 | 2012-08-15 | 中国科学院声学研究所 | Combined automatic segmentation method for abrupt change and gradual change of compressed domain video lens |
CN104881681B (en) * | 2015-05-22 | 2018-08-14 | 浙江大学 | Image sequence type labeling based on mixing graph model |
-
2016
- 2016-01-20 CN CN201610037058.2A patent/CN105721860B/en active Active
Also Published As
Publication number | Publication date |
---|---|
CN105721860A (en) | 2016-06-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Zhao et al. | Text from corners: a novel approach to detect text and caption in videos | |
KR101165359B1 (en) | Apparatus and method for analyzing relation with image and image or video | |
TW200536389A (en) | Intelligent key-frame extraction from a video | |
CN106687990B (en) | For the method based on gradual improvement from video sequence selection frame | |
CN105578034A (en) | Control method, control device and system for carrying out tracking shooting for object | |
US8467611B2 (en) | Video key-frame extraction using bi-level sparsity | |
JP2010518673A (en) | Method and system for video indexing and video synopsis | |
US20140226861A1 (en) | Person-based Video Summarization by Tracking and Clustering Temporal Face Sequences | |
Ren et al. | Best frame selection in a short video | |
CN110210379A (en) | A kind of lens boundary detection method of combination critical movements feature and color characteristic | |
CN103617263A (en) | Television advertisement film automatic detection method based on multi-mode characteristics | |
Wan et al. | A new technique for summarizing video sequences through histogram evolution | |
Osian et al. | Video shot characterization | |
Li et al. | An efficient spatiotemporal attention model and its application to shot matching | |
CN105721860B (en) | A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph | |
Cricri et al. | Salient event detection in basketball mobile videos | |
CN104504162B (en) | A kind of video retrieval method based on robot vision platform | |
KR101822443B1 (en) | Video Abstraction Method and Apparatus using Shot Boundary and caption | |
Baber et al. | Video segmentation into scenes using entropy and SURF | |
Luo et al. | Salient object detection in videos by optimal spatio-temporal path discovery | |
Piramanayagam et al. | Shot boundary detection and label propagation for spatio-temporal video segmentation | |
Baabou et al. | A comparative study and state-of-the-art evaluation for pedestrian detection | |
CN110830734B (en) | Abrupt change and gradual change lens switching identification method and system | |
KR102151748B1 (en) | Method, apparatus and computer-readable recording medium for seperating the human and the background from the video | |
Potnurwar et al. | Visual Attention Key Frame Extraction for Video Annotations |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |