CN105721860B - A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph - Google Patents

A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph Download PDF

Info

Publication number
CN105721860B
CN105721860B CN201610037058.2A CN201610037058A CN105721860B CN 105721860 B CN105721860 B CN 105721860B CN 201610037058 A CN201610037058 A CN 201610037058A CN 105721860 B CN105721860 B CN 105721860B
Authority
CN
China
Prior art keywords
summit
mrow
vector
video
shot
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201610037058.2A
Other languages
Chinese (zh)
Other versions
CN105721860A (en
Inventor
冀中
樊帅飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianjin University
Original Assignee
Tianjin University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianjin University filed Critical Tianjin University
Priority to CN201610037058.2A priority Critical patent/CN105721860B/en
Publication of CN105721860A publication Critical patent/CN105721860A/en
Application granted granted Critical
Publication of CN105721860B publication Critical patent/CN105721860B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N17/00Diagnosis, testing or measuring for television systems or their details

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Analysis (AREA)

Abstract

A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph:Extract video features;Build k neighbour's hypergraph models;Video shot boundary is found, including:The minimum score threshold of score vector is set according to k neighbours hypergraph model;The label vector on order inquiry summit is 1, and the label vector on other summits is arranged to 0;Each summit score vector is calculated, and records score vector more than the continuous vertex position in minimum score threshold, and randomly selects the summit within half and is designated as gathering, and it is 1 to make label vector;According to label vector, each summit score vector is calculated again;Label vector from order inquiry summit is 1, and the label vector on other summits is arranged to 0 and repeated;Determine video shot boundary and type;The label vector from order inquiry summit is 1 again, and the label vector on other summits is arranged to 0 and repeated, until all video shot boundaries determine to finish.The present invention is by retrieval of the hypergraph model to frame of video, by detection field of the hypergraph model for video shot boundary.

Description

A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph
Technical field
The present invention relates to a kind of Methods for Shot Boundary Detection of Video Sequences.More particularly to a kind of with content in same video lens The continuity and the characteristic of similitude having, by analysis of the hypergraph model to picture frame in video lens, determine that video is each The Methods for Shot Boundary Detection of Video Sequences based on hypergraph of shot boundary.
Background technology
Video lens are often referred to the video segment of video camera one-time continuous shooting, and video shot boundary is often referred to video lens Occurs change in some sense between consecutive frame.Video shot boundary detection is for finding side between multiple cinestrip A kind of technology on boundary.When two video lens change, it will usually there are some and significantly change, such as color characteristic Change etc..
Video shot boundary generally comprises two types.One kind is abrupt shot (abrupt shot), refers to frame of video from one Individual camera lens jumps to another camera lens suddenly.Abrupt shot generally occurs between two frames, and front and rear two frame is belonging respectively to former and later two Camera lens.Another kind is gradual shot (gradual shot), refers to frame of video and is gradually slowly transitioned into another from a camera lens Camera lens, camera lens have a kind of edit effect over time and space.Gradual shot, which generally includes to be fade-in, gradually goes out (fade in And fade out), dissolve (dissolve) etc..Gradual shot generally occurs in several frames between more than ten frames, is former and later two The transition of camera lens.Wherein abrupt shot is easier to detect, and the more difficult detection of gradual shot, is the emphasis of shot boundary detector.
Video shot boundary detection mainly includes similarity measurement, the camera lens between the feature extraction of frame of video, video features Border determines three steps.Existing video shot boundary detection technique mainly has method based on edge and based on movable information Method.The edge and gradient information of image can be very good to show the visual information of image, therefore can be used as video image Feature use.Shot detection method based on edge is generally more sensitive to the motion ratio of camera, object, therefore to gradual change The detection of camera lens is not very accurate.It based on frame of video in camera lens is smooth that method based on movable information, which is, boundary is The basic assumption of mutation.Therefore it is more accurate based on detection of the method for movable information to abrupt shot, for gradual shot Detection is nor very accurate.Method based on movable information, which also has, calculates the shortcomings that time complexity is high.Video lens at present The maximum challenge of bound test technology is the high speed for how detecting gradual shot and how eliminating illumination or camera and object Motion is to the influence at video shot boundary.
The content of the invention
The technical problem to be solved by the invention is to provide one kind to determine video mirror one by one in front using hypergraph model The Methods for Shot Boundary Detection of Video Sequences based on hypergraph on boundary.
The technical solution adopted in the present invention is:A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph, including it is as follows Step:
1) video features are extracted;
2) k neighbour's hypergraph models are built:
Wherein, v represents the summit of hypergraph, and e represents the super side of hypergraph, and H (v, e) is incidence matrix;
3) video shot boundary is found, including:
(1) first according to k neighbour's hypergraph models, when calculating given inquiry summit, other summits are relative to the summit Score vector:
F=(1- γ) (I- γ Θ)-1y (2)
Wherein, γ is constant factor;I is unit matrix;Y is the label vector on n summit of hypergraph, and dimension is that n × 1 is tieed up, f For relative to the score vector of query point, dimension is that n × 1 is tieed up;Θ be n × n Laplacian Matrix, Θ=Dv -1/2HWDe - 1HTDv -1/2, wherein, W is weight matrix, be with while weight w (e) be diagonal is formed surpass while weight matrix:
DvIt is the degree matrix for being formed summit for diagonal with the degree d (v) on summit for the degree matrix on summit:
D (v)=∑e∈Ew(e)·H(v,e) (4)
DeFor the degree matrix on super side, be with while degree d (e) be that diagonal is formed degree matrix when surpassing:
D (e)=∑v∈eH(v,e) (5)
(2) the minimum score threshold δ of score vector f is set;
(3) label vector y (j)=1 on order inquiry summit, the label vector on other summits are arranged to 0;
(4) each summit score vector f is calculated, and is recorded comprising the f including j-th of position>δ continuous vertex position, And the summit within half is randomly selected from described continuous vertex position as feedback point, it is designated as set F 'k, and make label Vectorial y (F 'k)=1, k is the label of set;
(5) according to label vector y (F 'k)=1, each summit score vector f is calculated again, and record and include set F 'k's f>δ continuous vertex position, is designated as set Fk, now FkRepresent all frames in the same camera lens comprising j-th of position;
(6) j=F is madek(last)+1, k=k+1, label vector y (j)=1 is set, calculates each summit score vector f, and And record includes the f including j-th of position>δ continuous vertex position, and randomly select one from described continuous vertex position Summit within half is designated as set F ' as feedback pointk+1, and make label vector y (F 'k+1)=1, according to label vector y (F′k+1)=1, each summit score vector f is calculated again, and record and include set F 'k+1F>δ continuous vertex position, is obtained To set Fk+1, wherein, Fk(last) it is set FkLast summit;
(7) video shot boundary and type are determined;
(8) F is madek=Fk+1, (6) step is returned to, until all video shot boundaries determine to finish.
In step 3) (3) step represent summit label vector y=[0 ..., 1 ... 0]T, wherein 1 j-th of position.
Determination video shot boundary and type in step 3) described in (7) step include:
Take set FkWith set Fk+1Common factor F, i.e. F=Fk∩Fk+1IfThen it is determined that in Fk(last) And Fk+1(first) it is a shot boundary at point, and the shot boundary is abrupt shot, i.e. FkAnd F (last)k+1 (first) point is respectively the up-and-down boundary of abrupt shot;IfIt can so determine in FAnd F (first) (last) it is a shot boundary at point, and the shot boundary is gradual shot, i.e. FAnd F (first)(last) point difference For the up-and-down boundary of gradual shot, wherein, Fk+1(first) it is set Fk+1First summit, F(first) it is set F's First summit, F(last) it is set FLast summit.
A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph of the present invention, passes through inspection of the hypergraph model to frame of video Rope, hypergraph model is used for the detection field of video shot boundary, there is following distinguishing feature:
1st, hypergraph model is applied to above shot boundary detector by the present invention first, is by being regarded all in a camera lens Frequency frame is all retrieved so as to determine video shot boundary, different with passing through color at shot boundary or movable information in the past Change to find the thinking of shot boundary.
2nd, experiments verify that, the present invention fast and effectively can detect video shot boundary, therefore be that one kind has The Methods for Shot Boundary Detection of Video Sequences of effect.
3rd, simple and easy, excellent effect of the invention.The fields such as video analysis, video frequency abstract, video frequency searching can be used in Pretreatment stage.
Brief description of the drawings
Fig. 1 is the flow chart of the Methods for Shot Boundary Detection of Video Sequences of the invention based on hypergraph;
Fig. 2 is an abrupt shot (frame of video is continuous videos), wherein 5 frames and 6 frames are respectively former and later two camera lenses Border;
Fig. 3 is a gradual shot (frame of video is continuous videos), wherein 6 frames to 9 frames are the border of former and later two camera lenses Scope.
Embodiment
A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph of the present invention is made with reference to embodiment and accompanying drawing Describe in detail.
A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph of the present invention, comprises the following steps:
1) video features are extracted;
2) k neighbour's hypergraph models are built:
Wherein, v represents the summit of hypergraph, and e represents the super side of hypergraph, and H (v, e) is incidence matrix;
3) video shot boundary is found, including:
(1) first according to k neighbour's hypergraph models, when calculating given inquiry summit, other summits are relative to the summit Score vector:
F=(1- γ) (I- γ Θ)-1y (2)
Wherein, γ is constant factor;I is unit matrix;Y is the label vector on n summit of hypergraph, and dimension is that n × 1 is tieed up, f For relative to the score vector of query point, dimension is that n × 1 is tieed up;Θ be n × n Laplacian Matrix, Θ=Dv -1/2HWDe - 1HTDv -1/2, wherein, W is weight matrix, be with while weight w (e) be diagonal is formed surpass while weight matrix:
DvIt is the degree matrix for being formed summit for diagonal with the degree d (v) on summit for the degree matrix on summit:
D (v)=∑e∈Ew(e)·H(v,e) (4)
DeFor the degree matrix on super side, be with while degree d (e) be that diagonal is formed degree matrix when surpassing:
D (e)=∑v∈eH(v,e) (5)
(2) the minimum score threshold δ of score vector f is set;
(3) label vector y (j)=1 on order inquiry summit, the label vector on other summits are arranged to 0, represent the mark on summit Sign vectorial y=[0 ..., 1 ... 0]T, wherein 1 j-th of position;
(4) each summit score vector f is calculated, and is recorded comprising the f including j-th of position>δ continuous vertex position, And the summit within half is randomly selected from described continuous vertex position as feedback point, it is designated as set F 'k, and make label Vectorial y (F 'k)=1, k is the label of set;
(5) according to label vector y (F 'k)=1, each summit score vector f is calculated again, and record and include set F 'k's f>δ continuous vertex position, is designated as set Fk, now FkRepresent all frames in the same camera lens comprising j-th of position;
(6) j=F is madek(last)+1, k=k+1, label vector y (j)=1 is set, calculates each summit score vector f, and And record includes the f including j-th of position>δ continuous vertex position, and randomly select one from described continuous vertex position Summit within half is designated as set F ' as feedback pointk+1, and make label vector y (F 'k+1)=1, according to label vector y (F′k+1)=1, each summit score vector f is calculated again, and record and include set F 'k+1F>δ continuous vertex position, is obtained To set Fk+1, wherein, Fk(last) it is set FkLast summit;
(7) video shot boundary and type are determined, including:
Take set FkWith set Fk+1Common factor F, i.e. F=Fk∩Fk+1IfThen it is determined that in Fk(last) And Fk+1(first) it is a shot boundary at point, and the shot boundary is abrupt shot, i.e. FkAnd F (last)k+1 (first) point is respectively the up-and-down boundary of abrupt shot;IfIt can so determine in FAnd F (first) (last) it is a shot boundary at point, and the shot boundary is gradual shot, i.e. FAnd F (first)(last) point difference For the up-and-down boundary of gradual shot, wherein, Fk+1(first) it is set Fk+1First summit, F(first) it is set F's First summit, F(last) it is set FLast summit;
(8) F is madek=Fk+1, (6) step is returned to, until all video shot boundaries determine to finish.
Fig. 2 and Fig. 3 is the judgement example of abrupt shot and gradual shot.Wherein Fig. 2 is abrupt shot, note query point 1 As a result it is set F1={ 1,2,3,4,5 };So remember query point F1(last) result of+1 (putting 6) is set F2=6,7,8, 9,10};It so can be determined that point F1=5 and point F (last)2(first) it is=6 the upper and lower of abrupt shot Boundary;Fig. 3 is gradual shot, and the result of note query point 1 is set F1={ 1,2,3,4,5,6,7,8,9 };So remember query point F1 (last) result of+1 (putting 10) is set F2={ 6,7,8,9,10,11,12,13,14 };F=F1∩F2=6,7,8, 9};It so can be determined that point F=6 and point F (first)(last)=9 it is the bound of gradual shot.
A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph of the present invention, in the specific implementation, can be divided into coarse grain Spend range detection and fine granularity range detection.The probable ranges of video boundaries are determined by coarseness range detection first, are saved It is a large amount of to calculate the time;Then by fine granularity range detection come precise video shot boundary scope.
Coarseness range detection step is as follows:
1) frame of video is sampled, such as can be with 1 second two frame of sampling;
2) feature of video is extracted;
3) k neighbour's hypergraph models are built;
4) performed according to lens boundary detection method step, and according to FTo judge lens type and bounds.
Above-mentioned coarseness detection is simply detected in sampling interframe, and a probable ranges are only determined, then also need Fine granularity range detection is carried out, fine granularity range detection includes:
(1) all frame of video between coarseness range detection result are sampled;
(2) repeat coarseness range detection step 2), 3), 4), so as to obtain accurate video shot boundary scope.

Claims (2)

1. a kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph, it is characterised in that comprise the following steps:
1) video features are extracted;
2) k neighbour's hypergraph models are built:
<mrow> <mi>H</mi> <mrow> <mo>(</mo> <mi>v</mi> <mo>,</mo> <mi>e</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfenced open = "{" close = ""> <mtable> <mtr> <mtd> <mrow> <mn>1</mn> <mo>,</mo> </mrow> </mtd> <mtd> <mrow> <mi>i</mi> <mi>f</mi> </mrow> </mtd> <mtd> <mrow> <mi>v</mi> <mo>&amp;Element;</mo> <mi>e</mi> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <mn>0</mn> <mo>,</mo> </mrow> </mtd> <mtd> <mrow> <mi>i</mi> <mi>f</mi> </mrow> </mtd> <mtd> <mrow> <mi>v</mi> <mo>&amp;NotElement;</mo> <mi>e</mi> </mrow> </mtd> </mtr> </mtable> </mfenced> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>1</mn> <mo>)</mo> </mrow> </mrow>
Wherein, v represents the summit of hypergraph, and e represents the super side of hypergraph, and H (v, e) is incidence matrix;
3) video shot boundary is found, including:
(1) first according to k neighbour's hypergraph models, when calculating given inquiry summit, other summits obtain relative to the summit Divide vector:
F=(1- γ) (I- γ Θ)-1y (2)
Wherein, γ is constant factor;I is unit matrix;Y is the label vector on n summit of hypergraph, and dimension is that n × 1 is tieed up, and f is phase For the score vector of query point, dimension is that n × 1 is tieed up;Θ be n × n Laplacian Matrix, Θ=Dv -1/2HWDe -1HTDv -1/2, Wherein, W is weight matrix, be with while weight w (e) be diagonal is formed surpass while weight matrix:
<mrow> <mi>w</mi> <mrow> <mo>(</mo> <mi>e</mi> <mo>)</mo> </mrow> <mo>=</mo> <msub> <mi>&amp;Sigma;</mi> <mrow> <msub> <mi>v</mi> <mi>i</mi> </msub> <mo>,</mo> <msub> <mi>v</mi> <mi>j</mi> </msub> <mo>&amp;Element;</mo> <mi>e</mi> </mrow> </msub> <mo>|</mo> <mo>|</mo> <msub> <mi>v</mi> <mi>i</mi> </msub> <mo>-</mo> <msub> <mi>v</mi> <mi>j</mi> </msub> <mo>|</mo> <msub> <mo>|</mo> <mn>2</mn> </msub> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>3</mn> <mo>)</mo> </mrow> </mrow>
DvIt is the degree matrix for being formed summit for diagonal with the degree d (v) on summit for the degree matrix on summit:
D (v)=∑e∈Ew(e)·H(v,e) (4)
DeFor the degree matrix on super side, be with while degree d (e) be that diagonal is formed degree matrix when surpassing:
D (e)=∑v∈eH(v,e) (5)
(2) the minimum score threshold δ of score vector f is set;
(3) label vector y (j)=1 on order inquiry summit, the label vector on other summits are arranged to 0;
(4) each summit score vector f is calculated, and is recorded comprising the f including j-th of position>δ continuous vertex position, and from The summit within half is randomly selected in described continuous vertex position as feedback point, is designated as set F 'k, and make label vector y(F′k)=1, k is the label of set;
(5) according to label vector y (F 'k)=1, each summit score vector f is calculated again, and record and include set F 'kF>δ Continuous vertex position, be designated as set Fk, now FkRepresent all frames in the same camera lens comprising j-th of position;
(6) j=F is madek(last)+1, k=k+1, label vector y (j)=1 is set, calculates each summit score vector f, and record Include the f including j-th of position>δ continuous vertex position, and randomly selected from described continuous vertex position within half Summit as feedback point, be designated as set F 'k+1, and make label vector y (F 'k+1)=1, according to label vector y (F 'k+1)=1, Each summit score vector f is calculated again, and is recorded and included set F 'k+1F>δ continuous vertex position, obtains set Fk+1, Wherein, Fk(last) it is set FkLast summit;
(7) video shot boundary and type are determined, including:
Take set FkWith set Fk+1Common factor F, i.e. F=Fk∩Fk+1IfThen it is determined that in FkAnd F (last)k+1 (first) it is a shot boundary at point, and the shot boundary is abrupt shot, i.e. FkAnd F (last)k+1(first) point minute Not Wei abrupt shot up-and-down boundary;IfIt can so determine in FAnd F (first)(last) it is one at point Shot boundary, and the shot boundary is gradual shot, i.e. FAnd F (first)(last) point is respectively the upper and lower of gradual shot Border, wherein, Fk+1(first) it is set Fk+1First summit, F(first) it is set FFirst summit, F (last) it is set FLast summit;
(8) F is madek=Fk+1, (6) step is returned to, until all video shot boundaries determine to finish.
A kind of 2. Methods for Shot Boundary Detection of Video Sequences based on hypergraph according to claim 1, it is characterised in that step 3) In (3) step represent summit label vector y=[0 ..., 1 ... 0]T, wherein 1 j-th of position.
CN201610037058.2A 2016-01-20 2016-01-20 A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph Active CN105721860B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610037058.2A CN105721860B (en) 2016-01-20 2016-01-20 A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610037058.2A CN105721860B (en) 2016-01-20 2016-01-20 A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph

Publications (2)

Publication Number Publication Date
CN105721860A CN105721860A (en) 2016-06-29
CN105721860B true CN105721860B (en) 2017-12-08

Family

ID=56147431

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610037058.2A Active CN105721860B (en) 2016-01-20 2016-01-20 A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph

Country Status (1)

Country Link
CN (1) CN105721860B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110874833B (en) * 2019-10-15 2022-03-22 合肥工业大学 SAR image change detection method based on hypergraph matching

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7177470B2 (en) * 2002-11-13 2007-02-13 Koninklijke Philips Electronics N. V. Method of and system for detecting uniform color segments
CN101650830B (en) * 2009-08-06 2012-08-15 中国科学院声学研究所 Combined automatic segmentation method for abrupt change and gradual change of compressed domain video lens
CN104881681B (en) * 2015-05-22 2018-08-14 浙江大学 Image sequence type labeling based on mixing graph model

Also Published As

Publication number Publication date
CN105721860A (en) 2016-06-29

Similar Documents

Publication Publication Date Title
Zhao et al. Text from corners: a novel approach to detect text and caption in videos
KR101165359B1 (en) Apparatus and method for analyzing relation with image and image or video
TW200536389A (en) Intelligent key-frame extraction from a video
CN106687990B (en) For the method based on gradual improvement from video sequence selection frame
CN105578034A (en) Control method, control device and system for carrying out tracking shooting for object
US8467611B2 (en) Video key-frame extraction using bi-level sparsity
JP2010518673A (en) Method and system for video indexing and video synopsis
US20140226861A1 (en) Person-based Video Summarization by Tracking and Clustering Temporal Face Sequences
Ren et al. Best frame selection in a short video
CN110210379A (en) A kind of lens boundary detection method of combination critical movements feature and color characteristic
CN103617263A (en) Television advertisement film automatic detection method based on multi-mode characteristics
Wan et al. A new technique for summarizing video sequences through histogram evolution
Osian et al. Video shot characterization
Li et al. An efficient spatiotemporal attention model and its application to shot matching
CN105721860B (en) A kind of Methods for Shot Boundary Detection of Video Sequences based on hypergraph
Cricri et al. Salient event detection in basketball mobile videos
CN104504162B (en) A kind of video retrieval method based on robot vision platform
KR101822443B1 (en) Video Abstraction Method and Apparatus using Shot Boundary and caption
Baber et al. Video segmentation into scenes using entropy and SURF
Luo et al. Salient object detection in videos by optimal spatio-temporal path discovery
Piramanayagam et al. Shot boundary detection and label propagation for spatio-temporal video segmentation
Baabou et al. A comparative study and state-of-the-art evaluation for pedestrian detection
CN110830734B (en) Abrupt change and gradual change lens switching identification method and system
KR102151748B1 (en) Method, apparatus and computer-readable recording medium for seperating the human and the background from the video
Potnurwar et al. Visual Attention Key Frame Extraction for Video Annotations

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant