CN103093458A - Detecting method and detecting device for key frame - Google Patents

Abstract

The invention provides a detecting method for a key frame and a detecting device for the key frame. The method comprises the following steps of conducting uneven partitioning to an input current video frame and a video frame adjacent to the current video frame, conducting statistics to histogram distribution probability and joint histogram distribution probability of two adjacent video frames in each partitioning and each color component, computing partitioning weighting mutual information quantity between the two adjacent video frames according to the histogram distribution probability and the joint histogram distribution probability of the two adjacent video frames, computing video frame differences of the two adjacent video frames according to the partitioning weighting mutual information quantity, conducting first key frame detecting to the current video frame according to the frame differences so as to obtain a primary detecting result of the current video frame and conducting second key frame detecting according to the primary detecting result of the current video frame so as to obtain a final detecting result of the current video frame. According to the detecting method, computing speed is fast, recall ratio and precision ratio are high, and expanding capacity and portability are strong.

Description

The detection method of key frame and device

Technical field

The present invention relates to technical field of computer vision, particularly a kind of detection method of key frame and device.

Background technology

Video is made of different scenes, and scene is comprised of different camera lenses, and each camera lens has comprised and severally do not wait frame of video to hundreds of more than even, and the detection of key frame refers to detect these and do not wait the shot boundary of video, corresponding frame of video when finding out shot change.General shot change comprises lens mutation, the type of being fade-in fade-out gradual change and lysotype gradual change.

The detection method of the key frame of prior art is that to utilize image histogram distance to characterize the frame of consecutive frame poor, thereby corresponding frame of video when finding out shot change, the minimum frame difference method of histogram for example, the problem that this method exists is, image histogram is just by simple cumulative statistics, is not enough to definite realization and goes out feature difference between adjacent two frame histograms.

Prior art can adopt the dual threshold method to carry out the detection of key frame, can detect simultaneously saltant and the gradually changeable key frame of video, the problem that this method exists is, the motion artifacts to illumination and large object on the one hand, there is obvious false retrieval, on the other hand for the key frame of part gradation type, lysotype gradual change key frame for example, can not correctly detect, false retrieval and undetected situation occur.

Summary of the invention

Purpose of the present invention is intended to solve at least one of above-mentioned technological deficiency.

For achieving the above object, an aspect of of the present present invention proposes a kind of detection method of key frame, comprise the following steps: S1: the current video frame of input and the adjacent video frames of described current video frame are carried out non-homogeneous piecemeal, and add up histogram distribution probability and the joint histogram distribution probability of described two adjacent frame of video on each piecemeal and each color component; S2: according to the divided group transinformation content of described two adjacent frame of video between the histogram distribution probability on each piecemeal and each color component and described two the adjacent frame of video of joint histogram distribution probability calculating, and poor according to the frame of described two the adjacent frame of video of described divided group transinformation content calculating; S3: poor according to the frame of described two adjacent frame of video described current video frame is carried out the initial survey result that key frame for the first time detects to obtain described current video frame; S4: described current video frame is carried out the final detection result that key frame for the second time detects to obtain described current video frame according to the initial survey result of described current video frame.

According to the detection method of the key frame of the embodiment of the present invention, have the following advantages: (1) computing velocity is fast, treatment effeciency is high: the video for arbitrary resolution can carry out sampling processing, can realize that real-time key frame detects; (2) recall ratio and precision ratio are high: locator key frame exactly has very high recall ratio and precision ratio; (3) good expansion and transplantability, easy to use: can combine with other key frame detection method and other application, have good expansion and wide application space.

For realizing said method, another aspect of the present invention also proposes a kind of pick-up unit of key frame, comprise: the distribution probability statistical module, be used for the current video frame of input and the adjacent video frames of described current video frame are carried out non-homogeneous piecemeal, and add up histogram distribution probability and the joint histogram distribution probability of described two adjacent frame of video on each piecemeal and each color component; The poor computing module of frame, be connected with described distribution probability statistical module, be used for according to the divided group transinformation content of described two adjacent frame of video between the histogram distribution probability on each piecemeal and each color component and described two the adjacent frame of video of joint histogram distribution probability calculating, and poor according to the frame of described two the adjacent frame of video of described divided group transinformation content calculating; First detection module is connected with the poor computing module of described frame, is used for poor according to the frame of described two adjacent frame of video described current video frame being carried out the initial survey result that key frame for the first time detects to obtain described current video frame; The second detection module is connected with described the first key frame detection module, is used for according to the initial survey result of described current video frame, described current video frame being carried out the final detection result that key frame for the second time detects to obtain described current video frame.。

According to the pick-up unit of the key frame of the embodiment of the present invention, easy to use, treatment effeciency is high, and recall ratio and precision ratio are high, and have good expansion and transplantability.

The aspect that the present invention adds and advantage part in the following description provide, and part will become obviously from the following description, or recognize by practice of the present invention.

Description of drawings

Above-mentioned and/or the additional aspect of the present invention and advantage will become from the following description of the accompanying drawings of embodiments and obviously and easily understand, wherein:

Fig. 1 is the detection method process flow diagram of the key frame of one aspect of the present invention embodiment;

Fig. 2 is the non-homogeneous piecemeal of the embodiment of the present invention and the schematic diagram of respective weights;

Fig. 3 is the principle schematic that the key frame for the first time of the embodiment of the present invention detects;

Fig. 4 is the distribution schematic diagram of the pixel sampled point of the embodiment of the present invention;

Fig. 5 is the present invention's structural representation of the pick-up unit of the key frame of embodiment on the other hand;

Fig. 6 is the structural representation of the distribution probability statistical module 100 of the embodiment of the present invention;

Fig. 7 is the structural representation of the poor computing module 200 of the frame of the embodiment of the present invention;

Fig. 8 is the structural representation of the first detection module 300 of the embodiment of the present invention; And

Fig. 9 is the structural representation of the second detection module 400 of the embodiment of the present invention.

Embodiment

The below describes embodiments of the invention in detail, and the example of described embodiment is shown in the drawings, and wherein same or similar label represents same or similar element or the element with identical or similar functions from start to finish.Be exemplary below by the embodiment that is described with reference to the drawings, only be used for explaining the present invention, and can not be interpreted as limitation of the present invention.

For the detection method of key frame of the explanation embodiment of the present invention and the effect quality of device have defined recall ratio and precision ratio.Recall ratio refer to video lens cut apart in correct key frame detect number divided by actual number of key frames, precision ratio refer to video lens cut apart in correct key frame detect number divided by the number that detects of total key frame.

Fig. 1 is the detection method process flow diagram of the key frame of one aspect of the present invention embodiment.As shown in Figure 1, the detection method according to the key frame of the embodiment of the present invention comprises the following steps:

Step S101 carries out non-homogeneous piecemeal to the current video frame of input and the adjacent video frames of current video frame, and adds up histogram distribution probability and the joint histogram distribution probability of two adjacent frame of video on each piecemeal and each color component.

Particularly, at first, to two adjacent frame of video according to the ratio of 1:3:1 respectively to long and the wide piecemeal that carries out, obtain 9 non-homogeneous piecemeal m (m=1,2 ..., 9), and give weights W according to the position of non-homogeneous piecemeal, wherein,

$W=\left[\begin{array}{ccc}{w}_1& w_2& w_3\\ w_4& w_5& w_6\\ w_7& w_8& w_9\end{array}\right]=\left[\begin{array}{ccc}1& 1& 1\\ 2& 4& 2\\ 1& 1& 1\end{array}\right].$

Global color that can only the reflecting video two field picture according to traditional histogram calculation method distributes, and is difficult to the spatial information of reflecting video two field picture.Due in ordinary video, advertisement or captions appear in top and bottom through the frame of video of being everlasting, the frequent variations of advertisement or captions detects the formation interference to the key frame of camera lens, therefore embodiments of the invention carry out inhomogeneous piecemeal to frame of video, and give different weights according to the position of non-homogeneous piecemeal, embody the variation of main contents in frame of video, improve thus recall ratio and precision ratio.The non-homogeneous piecemeal that is illustrated in figure 2 as the embodiment of the present invention reaches the schematic diagram of each piecemeal being given weights, according to the ratio of 1:3:1 with frame of video be divided into 3 * 3 do not wait piecemeal, wherein, in Fig. 2, numeral weights in non-homogeneous piecemeal, W represents the width of video frame images, and H represents the height of video frame images.

Then, adding up two adjacent frame of video value on m piecemeal, R color component is the number of pixels of i, and with the sum of all pixels order of number of pixels divided by the m piecemeal, obtains the histogram distribution probability

With

And obtain successively histogram distribution probability on G, B color component

With

With

Wherein t represents current video frame, and t-1 represents the adjacent video frames of current video frame.

At last, add up two adjacent frame of video pixels that value is respectively i and j on corresponding m piecemeal, R color component to number, and with pixel to the sum of all pixels order of number divided by the m piecemeal, obtain the joint histogram distribution probability And obtain successively joint histogram distribution probability on G, B color component

Step S102, according to the divided group transinformation content of two adjacent frame of video between the histogram distribution probability on each piecemeal and each color component and two adjacent frame of video of joint histogram distribution probability calculating, and poor according to the frame of two adjacent frame of video of divided group transinformation content calculating.

Particularly, at first, according to the histogram distribution probability of two adjacent frame of video

And joint histogram distribution probability

Calculate the transinformation content of two adjacent frame of video on m piecemeal, R color component according to following formula

$I_{t,t-1}^m\left(R\right)=-\underset{i=0}{\overset{N-1}{\mathrm{\Σ}}}\underset{j=0}{\overset{N-1}{\mathrm{\Σ}}}{p}_{t,t-1}^m(R_i,R_j)*{\log }_2\frac{p_{t,t-1}^m(R_i,R_j)}{p_t^m\left(R_i\right)*p_{t-1}^m\left(R_j\right)},$

And calculate successively two adjacent frame of video of acquisition at the m piecemeal, the transinformation content on G, B color component $I_{t,t-1}^m\left(B\right).$

Then, according to two adjacent frame of video at the m piecemeal, the transinformation content on R, G, B color component

And

Calculate total color transinformation content according to following formula

$I_{t,t-1}^m(R,G,B)=\frac{1}{3}(I_{t,t-1}^m\left(R\right)+I_{t,t-1}^m\left(G\right)+I_{t,t-1}^m\left(B\right)).$

According to weight matrix W and total color transinformation content

Calculate divided group transinformation content I according to following formula _{T, t-1},

$I_{t,t-1}=\underset{m=1}{\overset{9}{\mathrm{\Σ}}}(w_m*I_{t,t-1}^m(R,G,B))/\underset{m=1}{\overset{9}{\mathrm{\Σ}}}{w}_m.$

At last, according to divided group transinformation content I _{T, t-1}, according to the poor dist of frame of two adjacent frame of video of following formula calculating _{T, t-1},

dist _t，t-1=1-I _t，t-1。

Step S103 poorly according to the frame of two adjacent frame of video carries out to current video frame the initial survey result that key frame for the first time detects to obtain current video frame.

Be illustrated in figure 3 as the principle schematic of the detection of key frame for the first time of the embodiment of the present invention, detect respectively sudden change key frame and gradual change key frame by first threshold TH and the first threshold TL that two poor sizes of judgment frame are set.

Particularly, at first, compare with first threshold TH and first threshold TL the frame of two adjacent frame of video is poor.

In one embodiment of the invention, at first arranging of first threshold TH and Second Threshold TL specifically comprise the steps:, calculates that between the adjacent video frames of the previous key frame of current video frame and current video frame, video sequence length is the poor dist of frame of the frame of video of S _{I, i1}(i=1 ..., S-1), be then the poor dist of frame of the frame of video of S according to video sequence length _{I, i-1}(i=1 ..., S-1), calculate the poor average μ of frame, wherein,

$\mathrm{\μ}=\frac{1}{S}\underset{i=1}{\overset{i=S-1}{\mathrm{\Σ}}}{\mathrm{dist}}_{i,i-1}.$

Average μ poor according to frame calculates described threshold value according to following formula at last,

TH=5μ，TL=3μ。

It should be understood that the setting of first threshold TH and Second Threshold TL, can carry out the self-adaptation adjustment according to different video contents in computation process according to the embodiment of the present invention, also can arrange as required voluntarily.

If the frame of two adjacent frame of video is poor greater than first threshold TH, current video frame may be the sudden change key frame, the key frame for the first time that carries out following steps detects: poor according to annular array storage frames centered by current video frame, two adjacent video frames each r frame of left and right, wherein, r=3 ~ 5; The frame of two adjacent frame of video of judgement poor with annular array in the frame extent of all 2r+1 adjacent video frames, if the frame of two adjacent frame of video poor be maximal value, continue next step judgement, otherwise current video frame is not the key frame that suddenlys change; Continue the frame of two adjacent frame of video of judgement poor with annular array in the frame of all 2r+1 adjacent video frames poor in time large frame extent, if during the poor frame than all 2r+1 adjacent video frames in annular array of the frame of two adjacent frame of video is poor, time large frame is poor large 3 times, current video frame is the sudden change key frame, otherwise current video frame is not the sudden change key frame.

If the frame of two adjacent frame of video is poor less than first threshold TH and greater than Second Threshold TL, current video frame is the gradual change start frame, and the gradual change end frame of carrying out the key frame for the first time of following steps detects: the poor dist of frame that calculates current video frame and k frame of video thereafter _t,k, k=t+1 wherein, t+2 ...; The poor dist of frame of judgement current video frame and k frame of video thereafter _t,kWith the size of first threshold, if the poor dist of frame of current video frame and k frame of video thereafter _t,kGreater than first threshold, the k frame is candidate's end frame; Continue to calculate the poor dist of frame of k frame and a frame of video after it _{K+j+1, k+j}, wherein, j=0,1,2 ..., a-1; Continue the poor dist of frame of judgement k frame and a frame of video after it _t,kWith the size of Second Threshold, if the poor dist of frame of k frame and a frame of video after it _{K+j+1, k+j}Less than Second Threshold, the k frame is the gradual change end frame.

The first detecting method of step S103 claims again the dual threshold method, can satisfy the requirement of different video content change by two parameter values of threshold value TH and TL are set, and can detect also detection of gradual transitions key frame well of sudden change key frame, the gradual change key frame of particularly being fade-in fade-out.

Step S104 carries out to current video frame the final detection result that key frame for the second time detects to obtain current video frame according to the initial survey result of current video frame.

Particularly, at first, obtain the initial survey result of current video frame.

If the initial survey result of current video frame detects for sudden change key frame, the key frame for the second time that carries out following steps: and pixel difference histogram variance poor according to current video frame calculating blocked histogram; Definite first change threshold poor according to blocked histogram, and determine the second change threshold according to pixel difference histogram variance; The size of judgement pixel difference histogram variance and the first change threshold and blocked histogram is poor and the size of the second change threshold; If pixel difference histogram variance is poor less than the second change threshold greater than the first change threshold or blocked histogram, current video frame is not the sudden change key frame, if pixel difference histogram variance is less than or equal to the first change threshold and blocked histogram is poor more than or equal to the second change threshold, current video frame is the sudden change key frame.

More specifically, the poor BHDM of blocked histogram is that D (t, t-1) computation process is as follows,

$D(t,t-1)=\frac{\left[\underset{k=1}{\overset{m}{\mathrm{\Σ}}}\mathrm{DB}(t,t-1,k)\right]-\mathrm{Max}\left(\mathrm{DB}(t,t-1,k)\right)}{m-1},$

$\mathrm{DB}(t,t-1,k)=\frac{\underset{j=1}{\overset{n}{\mathrm{\Σ}}}|H_{t,k}\left(j\right)-H_{t-1,k}\left(j\right)|}{n},$

Wherein, H _{T, k}(j) be the value of normalization histogram on grey level j of the k piecemeal of current video frame t, after current video frame t is carried out non-homogeneous piecemeal as shown in step S101, get its gray-scale map, statistics current video frame t is at the k piecemeal, gray-scale value is the number of pixels of j and with the sum of all pixels order of number of pixels divided by the k piecemeal, obtains H _{T, k}(j), n represents the quantity of gray level, and m represents the piecemeal number of frame of video, and calculates the H of the adjacent video frames t-1 that obtains current video frame t according to these computing method _{T-1, k}(j).

Pixel difference histogram variance (VDHM) is that V (t, t-1) computation process is as follows,

$V(t,t-1)=\frac{\underset{j=1}{\overset{n}{\mathrm{\Σ}}}{({\mathrm{DH}}_{t,t-1}\left(j\right)-\stackrel{\‾}{\mathrm{DH}})}^2}{n}$

Wherein, DH _{T, t-1}(j) value of normalization pixel difference histogram on difference rank j of two adjacent video frames t of expression and t-1, get the gray-scale map of two adjacent video frames t and t-1, then the gray-scale map with two adjacent video frames subtracts each other according to the pixel correspondence position and capture element absolute difference, obtain new gray-scale map, adding up this new gray-scale map is the number of pixels of j in the gray scale value, and with number of pixels divided by the total number of the pixel of gray-scale map, obtain DH _{T, t-1}(j). DH _{T, t-1}(j) mean value, n represent the quantity of gray level.

In computation process, j rounds numerical value, n=256 in [0,255] interval.

Hence one can see that, 0≤D (t, t-1)≤1, blocked histogram poor (BHDM) reflection be color distortion between the image of frame of video, 0≤V (t, t-1)≤1, pixel difference histogram variance (VDHM) reflection be spatial diversity between the image of frame of video.If current video frame t is not the sudden change key frame, its corresponding pixel difference histogram variance (VDHM) V (t, t-1) is larger and poor (BHDM) D of blocked histogram (t, t-1) is smaller.If current video frame t is the sudden change key frame, its corresponding pixel difference histogram variance (VDHM) value V (t, t-1) is smaller and blocked histogram poor (BHDM) value D (t, t-1) is larger.

The first change threshold T that blocked histogram poor (BHDM) is corresponding _bThe second change threshold T corresponding with pixel difference histogram variance (VDHM) _vCan determine by the adaptive threshold method based on moving window, wherein,

$T_v=\frac{1}{3}*\frac{1}{S}\underset{i=1}{\overset{i=S-1}{\mathrm{\Σ}}}V(t,t-1),$ $T_b=3*\frac{1}{S}\underset{i=1}{\overset{i=S-1}{\mathrm{\Σ}}}D(t,t-1).$

If the initial survey result of current video frame is the gradual change key frame, carry out respectively following (a) and (b) shown in key frame for the second time detect.

(a) according to the initial survey result of step S103, be that the gradual change start frame carries out the sampling of pixel R, G, B to sequence of frames of video between the gradual change end frame to current video frame, be illustrated in figure 4 as the distribution plan of pixel sampled point, stain is pixel sampling point position, and it is respectively the mid point of each line segment; Judge whether there is entirely black sampled point in the sampled point sequence, if there is complete black sampled point, current video frame and gradual change end frame are the gradual change key frame of being fade-in fade-out, and if there is no entirely deceive sampled point, and the initial survey result is flase drop.

In the general shot change process of being fade-in fade-out, always there are a frame or a few frame all black picture, can determine whether it is the camera lens of being fade-in fade-out by detecting black frame of video.

(b) according to the initial survey result of described step S103, judge two initial survey key frame S _aAnd S _bBetween sequence of frames of video length whether greater than 30 frames, if sequence of frames of video length greater than 30 frames, continues next step calculating and judgement, if sequence of frames of video length is not more than 30 frames, two initial survey key frame S _aAnd S _bBetween do not have lysotype gradual change key frame; Add up two initial survey key frame S _a, S _bBetween the poor average λ of frame of adjacent video frames; Judge two initial survey key frame S _a, S _bBetween sequence of frames of video S _a, S _a+1, S _a+2..., S _b-1, S _bIn whether have certain frame S _k, after it, frame of two adjacent video frames of a frame is poor all greater than described average λ and less than Second Threshold, if exist S _kBe candidate's start frame of lysotype gradual change, frame sequence S detected this moment _{K+ α+1}, continue next step calculating and judgement, otherwise continue to detect candidate's start frame of lysotype gradual change, wherein a=5 ~ 8; Judgement S _{K+ α+1}..., S _b-1, S _bWhether middle existence exists certain frame S _r, after it, frame of two adjacent video frames of ω frame is poor all less than average λ, if exist S _rBe the end frame of lysotype gradual change, make k=r+ ω change (b) over to and continue to detect, as k detection of end, wherein ω=5 ~ 8 during b.

Detection method according to the key frame of the embodiment of the present invention, fully utilized the initial survey of non-homogeneous piecemeal, transinformation content, improvement dual threshold method and rechecked scheduling algorithm, specifically have the following advantages: (1) computing velocity is fast, treatment effeciency is high: the method for the embodiment of the present invention can be carried out sampling processing for the video of arbitrary resolution, can realize that real-time key frame detects; (2) recall ratio and precision ratio are high: the present invention has fully utilized the initial survey of non-homogeneous piecemeal, transinformation content, improvement dual threshold method, the advantage of reinspection scheduling algorithm, the method that makes the embodiment of the present invention is the locator key frame exactly, has very high recall ratio and precision ratio; (3) good expansion and transplantability, easy to use: the method for the embodiment of the present invention can combine with other key frame detection method and other application, has good expansion and wide application space.

Fig. 5 is the present invention's structural representation of the pick-up unit of the key frame of embodiment on the other hand.As shown in Figure 5, the pick-up unit according to the key frame of the embodiment of the present invention comprises distribution probability statistical module 100, the poor computing module 200 of frame, first detection module 300 and the second detection module 400.

Wherein, distribution probability statistical module 100 is used for the current video frame of input and the adjacent video frames of current video frame are carried out non-homogeneous piecemeal, and adds up histogram distribution probability and the joint histogram distribution probability of two adjacent frame of video on each piecemeal and each color component.

As shown in Figure 6, distribution probability statistical module 100 comprises non-homogeneous minute module unit 110, histogram distribution probability statistics unit 120 and joint histogram distribution probability statistic unit 130.Wherein, non-homogeneous minute module unit 110 is used for two adjacent frame of video according to the ratio of 1:3:1 respectively to long and the wide piecemeal that carries out, obtain 9 non-homogeneous piecemeal m (m=1,2 ..., 9), and give weights W according to the position of non-homogeneous piecemeal.Histogram distribution probability calculation unit 120 is connected with non-homogeneous minute module unit 110, being used for two adjacent frame of video of statistics value on m piecemeal, R color component is the number of pixels of i, and with the sum of all pixels order of number of pixels divided by the m piecemeal, obtain the histogram distribution probability

With

And obtain successively histogram distribution probability on G, B color component

With

With

Wherein t represents current video frame, and t-1 represents the adjacent video frames of current video frame.Joint histogram distribution probability computing unit 130 is connected with histogram distribution probability calculation unit 120, be used for two adjacent frame of video of statistics value on corresponding m piecemeal, R color component and be respectively the pixel of i and j to number, and with pixel to the sum of all pixels order of number divided by the m piecemeal, obtain the joint histogram distribution probability

And obtain successively described joint histogram distribution probability on G, B color component

The poor computing module 200 of frame is connected with distribution probability computing module 100, be used for according to the divided group transinformation content of two adjacent frame of video between the histogram distribution probability on each piecemeal and each color component and two adjacent frame of video of joint histogram distribution probability calculating, and poor according to the frame of two adjacent frame of video of divided group transinformation content calculating.

As shown in Figure 7, the poor computing module 200 of frame comprises single color transinformation content computing unit 210, total color transinformation content computing unit 220, divided group transinformation content computing unit 230 and the poor computing unit 240 of frame.Wherein, single color transinformation content computing unit 210 is for the histogram distribution probability according to two adjacent frame of video

And joint histogram distribution probability

Calculate the transinformation content of two adjacent frame of video on m piecemeal, R color component

And calculate successively two adjacent frame of video at the m piecemeal, the transinformation content on G, B color component

Total color transinformation content computing unit 220 is connected with single color transinformation content computing unit 210, for according to two adjacent frame of video at the m piecemeal, the transinformation content on R, G, B color component

And

Calculate total color transinformation content

Divided group transinformation content computing unit 230 is connected with total color transinformation content computing unit 220, is used for according to weight matrix W and total color transinformation content

Calculate divided group transinformation content I _{T, t-1}The poor computing unit 240 of frame is connected with divided group transinformation content computing unit 230, is used for according to divided group transinformation content I _{T, t-1}, calculate the poor dist of frame of adjacent two frame of video _{T, t-1}

Detailed computation process can be with reference to the computation process of step S102 in the detection method of the key frame of the embodiment of the present invention.

Detection module 300 is connected with the poor computing module 200 of frame for the first time, is used for poor according to the frame of two adjacent frame of video current video frame being carried out the initial survey result that key frame for the first time detects to obtain current video frame.

As shown in Figure 8, detection module 300 comprises comparing unit 310, the first judging unit 320, the first sudden change key frame detecting unit 330 and the first gradual change key frame detecting unit 340 for the first time.Wherein, comparing unit 310 is used for comparing with first threshold and Second Threshold the frame of two adjacent frame of video is poor.The first judging unit 320 is connected with comparing unit 310, be used for the comparative result according to comparing unit, the first detection mode of judgement current video frame, if the frame of two adjacent frame of video is poor greater than described first threshold, current video frame may be the sudden change key frame, enter the first sudden change key frame detecting unit 330, if the frame of two adjacent frame of video is poor less than first threshold and greater than Second Threshold, current video frame is the gradual change start frame, enters the first gradual change key frame detecting unit 340 and carries out the detection of gradual change end frame.The first sudden change key frame detecting unit 330 is connected with the first judging unit 310, is used for judging whether current video frame is the sudden change key frame.The first gradual change key frame detecting unit 340 is connected with the first judging unit 310, is used for continuing to detect the gradual change end frame of current video frame.

The detailed testing process of first sudden change key frame detecting unit 330 and the first gradual change key frame detecting unit 340 can be with reference to the testing process of step S103 in the detection method of the key frame of one aspect of the present invention embodiment.

In one embodiment of the invention, first detection module 300 also comprises threshold value setting unit 350, threshold value setting unit 350 is connected with the first gradual change key frame detecting unit 340 with the first sudden change key frame detecting unit 330, be used for arranging first threshold and Second Threshold, the setting steps that concrete setting up procedure can comprise with reference to step S103 in the detection method of the key frame of one aspect of the present invention embodiment.

The second detection module 400 is connected with first detection module 300, is used for according to the initial survey result of current video frame, current video frame being carried out the final detection result that key frame for the second time detects to obtain current video frame.

As shown in Figure 9, the second detection module 400 comprises acquiring unit 410, the second judging unit 420, the second sudden change key frame detecting unit 430, second be fade-in fade-out gradual change key frame detecting unit 440 and the second lysotype gradual change key frame detecting unit 450.Wherein, acquiring unit 410 is connected with first detection module 300, is used for obtaining the initial survey result of current video frame.The second judging unit 420 is connected with acquiring unit 410, be used for the initial survey result according to acquiring unit 410, the second detection mode of judgement current video frame, if the initial survey result of current video frame is the sudden change key frame, entering the second sudden change key frame detecting unit 430 detects, if the initial survey result of current video frame is the gradual change key frame, enter the second be fade-in fade-out gradual change key frame detecting unit 440 and the second lysotype gradual change key frame detecting unit 450.The second sudden change key frame detecting unit 430 is connected with the second judging unit 420, for the reinspection of the sudden change key frame that carries out current video frame.The second gradual change key frame detecting unit 440 of being fade-in fade-out is connected with the second judging unit 420, is used for carrying out the reinspection of the key frame of being fade-in fade-out of current video frame.The second lysotype gradual change key frame detecting unit 450 is connected with the second judging unit 420, for the reinspection of the lysotype gradual change key frame that carries out current video frame.

The second sudden change key frame detecting unit 430, second is fade-in fade-out the concrete testing process of gradual change key frame detecting unit 440 and the second lysotype gradual change key frame detecting unit 450 can be with reference to the testing process of the step S104 in the detection method of the key frame of one aspect of the present invention embodiment.

According to the pick-up unit of the key frame of the embodiment of the present invention, easy to use, treatment effeciency is high, and recall ratio and precision ratio are high, and have good expansion and transplantability.

Detection method and device according to the key frame of the embodiment of the present invention have following beneficial effect at least:

(1) computing velocity is fast, treatment effeciency is high: can carry out sampling processing to the video of arbitrary resolution according to the method for the embodiment of the present invention, realize that real-time key frame detects, the device treatment effeciency of the embodiment of the present invention is high.

(2) recall ratio and precision ratio are high: the method synthesis of the embodiment of the present invention has utilized non-homogeneous piecemeal, transinformation content, improved dual threshold method, camera lens to recheck the scheduling algorithm advantage, and the locator key frame, have very high recall ratio and precision ratio exactly.

(3) good expansion and transplantability, easy to use: the method for the embodiment of the present invention can combine with other key frame detection method and other application, has good expansion and wide application space.The device of the embodiment of the present invention has good expansion and wide application space.

Although illustrated and described embodiments of the invention, for the ordinary skill in the art, be appreciated that without departing from the principles and spirit of the present invention and can carry out multiple variation, modification, replacement and modification to these embodiment, scope of the present invention is by claims and be equal to and limit.

Claims (12)

1. the detection method of a key frame, is characterized in that, comprises the following steps:

S1: the current video frame of input and the adjacent video frames of described current video frame are carried out non-homogeneous piecemeal, and add up histogram distribution probability and the joint histogram distribution probability of described two adjacent frame of video on each piecemeal and each color component;

S2: according to the divided group transinformation content of described two adjacent frame of video between the histogram distribution probability on each piecemeal and each color component and described two the adjacent frame of video of joint histogram distribution probability calculating, and poor according to the frame of described two the adjacent frame of video of described divided group transinformation content calculating;

S3: poor according to the frame of described two adjacent frame of video described current video frame is carried out the initial survey result that key frame for the first time detects to obtain described current video frame;

S4: described current video frame is carried out the final detection result that key frame for the second time detects to obtain described current video frame according to the initial survey result of described current video frame.

2. the detection method of key frame according to claim 1, is characterized in that, described step S1 further comprises:

S11: to described two adjacent frame of video according to the ratio of 1:3:1 respectively to long and the wide piecemeal that carries out, obtain 9 non-homogeneous piecemeal m (m=1,2 ..., 9), and give weights W according to the position of described non-homogeneous piecemeal, wherein,

$W=\left[\begin{array}{ccc}{w}_1& w_2& w_3\\ w_4& w_5& w_6\\ w_7& w_8& w_9\end{array}\right]=\left[\begin{array}{ccc}1& 1& 1\\ 2& 4& 2\\ 1& 1& 1\end{array}\right];$

S12: adding up described two adjacent frame of video value on m piecemeal, R color component is the number of pixels of i, and with the sum of all pixels order of described number of pixels divided by the m piecemeal, obtains described histogram distribution probability

With

And obtain successively described histogram distribution probability on G, B color component

With With

Wherein t represents described current video frame, and t-1 represents the adjacent video frames of described current video frame;

S13: add up described two adjacent frame of video pixel that value is respectively i and j on corresponding m piecemeal, R color component to number, and with described pixel to the sum of all pixels order of number divided by the m piecemeal, obtain described joint histogram distribution probability And obtain successively described joint histogram distribution probability on G, B color component

3. the detection method of key frame according to claim 2, is characterized in that, described step S2 further comprises:

S21: according to the histogram distribution probability of described two adjacent frame of video

And described joint histogram distribution probability

Calculate the transinformation content of described two adjacent frame of video on m piecemeal, R color component according to following formula

$I_{t,t-1}^m\left(R\right)=-\underset{i=0}{\overset{N-1}{\mathrm{\Σ}}}\underset{j=0}{\overset{N-1}{\mathrm{\Σ}}}{p}_{t,t-1}^m(R_i,R_j)*{\log }_2\frac{p_{t,t-1}^m(R_i,R_j)}{p_t^m\left(R_i\right)*p_{t-1}^m\left(R_j\right)},$

And calculate successively described two the adjacent frame of video of acquisition at the m piecemeal, the transinformation content on G, B color component $I_{t,t-1}^m\left(G\right),I_{t,t-1}^m\left(B\right);$

S22: according to described two adjacent frame of video at the m piecemeal, the transinformation content on R, G, B color component And

Calculate total color transinformation content according to following formula

$I_{t,t-1}^m(R,G,B)=\frac{1}{3}(I_{t,t-1}^m\left(R\right)+I_{t,t-1}^m\left(G\right)+I_{t,t-1}^m\left(B\right));$

S23: according to described weight matrix W and described total color transinformation content

Calculate described divided group transinformation content I according to following formula _{T, t-1},

$I_{t,t-1}=\underset{m=1}{\overset{9}{\mathrm{\Σ}}}(w_m*I_{t,t-1}^m(R,G,B))/\underset{m=1}{\overset{9}{\mathrm{\Σ}}}{w}_m;$

S24: according to described divided group transinformation content I _{T, t-1}, according to the poor dist of frame of described two the adjacent frame of video of following formula calculating _{T, t-1},

dist _t，t-1=1-I _t，t-1。

4. the detection method of key frame according to claim 1, is characterized in that, described step S3 further comprises:

S31: first threshold and Second Threshold that the frame of described two adjacent frame of video is poor with predetermined compare;

S32: if the frame of described two adjacent frame of video is poor greater than described first threshold, described current video frame may continue execution in step S33 for the sudden change key frame,

If the frame of described two adjacent frame of video is poor less than described first threshold and greater than described Second Threshold, described current video frame is the gradual change start frame, continues execution in step S34 and carries out the detection of gradual change end frame;

S33: poor according to annular array storage frames centered by described current video frame, two adjacent video frames each r frame of left and right, wherein, r=3 ~ 5,

Judge the frame extent of all 2r+1 adjacent video frames in the poor and described annular array of frame of described two adjacent frame of video, be maximal value if the frame of described two adjacent frame of video is poor, continue next step judgement, otherwise described current video frame not the sudden change key frame

Judge the frame of all 2r+1 adjacent video frames in the poor and described annular array of frame of described two adjacent frame of video poor in time large frame extent, if during the poor frame than all 2r+1 adjacent video frames in described annular array of the frame of described two adjacent frame of video is poor, time large frame is poor large 3 times, described current video frame is the sudden change key frame, otherwise described current video frame is not the sudden change key frame;

S34: the poor dist of frame that calculates described current video frame and k frame of video thereafter _t,k, k=t+1 wherein, t+2 ...,

Judge the poor dist of frame of described current video frame and k frame of video thereafter _t,kWith the size of described first threshold, if the poor dist of frame of described current video frame and k frame of video thereafter _t,kGreater than described first threshold, described k frame is candidate's end frame,

Calculate the poor dist of frame of described k frame and a frame of video after it _{K+j+1, k+j}, wherein, j=0,1,2 ..., a-1;

Judge the poor dist of frame of described k frame and a frame of video after it _t,kWith the size of described Second Threshold, if the poor dist of frame of described k frame and a frame of video after it _{K+j+1, k+j}Less than described Second Threshold, described k frame is the gradual change end frame.

5. the detection method of key frame according to claim 4, is characterized in that, described step S31 further comprises:

S311: between the previous key frame of the described current video frame of calculating and the adjacent video frames of described current video frame, video sequence length is the poor dist of frame of the frame of video of S _{I, i-1}(i=1 ..., S-1),

S312: calculating described sequence length is the poor average μ of frame of the frame of video of S,

S313: average μ poor according to described frame calculates described first threshold and described Second Threshold, and wherein said first threshold equals 5 times of the poor average μ of described frame, and described Second Threshold equals 3 times of the poor average of described frame; And

S314: the poor and described first threshold of the frame of described two adjacent frame of video and Second Threshold are compared.

6. the detection method of key frame according to claim 4, is characterized in that, described step S4 further comprises:

S41: the initial survey result of obtaining described current video frame;

S42: if the initial survey result of described current video frame continues execution in step S43 for the sudden change key frame,

If the initial survey result of described current video frame is the gradual change key frame, continue execution in step S44 and step S45;

S43: and pixel difference histogram variance poor according to described current video frame calculating blocked histogram,

Definite first change threshold poor according to described blocked histogram, and determine the second change threshold according to described pixel difference histogram variance,

Judge the size of described pixel difference histogram variance and described the first change threshold and the size of poor and described the second change threshold of described blocked histogram,

If described pixel difference histogram variance is poor less than described the second change threshold greater than described the first change threshold or described blocked histogram, described current video frame is not the sudden change key frame,

If described pixel difference histogram variance is less than or equal to described the first change threshold and described blocked histogram is poor more than or equal to described the second change threshold, described current video frame is the sudden change key frame;

S44: according to the initial survey result of described step S34, be that the gradual change start frame carries out the sampling of pixel R, G, B to sequence of frames of video between described gradual change end frame to described current video frame,

Judge whether there is complete black sampled point in described sampled point sequence, if there is described complete black sampled point, described current video frame and described gradual change end frame be the gradual change key frame of being fade-in fade-out, the if there is no described sampled point of entirely deceiving, and the initial survey result is flase drop.

S45: judge two initial survey key frame S _aAnd S _bBetween sequence of frames of video length whether greater than 30 frames, if described sequence of frames of video length continues next step calculating and judgement greater than 30 frames, if described sequence of frames of video length is not more than 30 frames, described two initial survey key frame S _aAnd S _bBetween do not have lysotype gradual change key frame,

Add up described two initial survey key frame S _a, S _bBetween the poor average λ of frame of adjacent video frames,

Judge described two initial survey key frame S _a, S _bBetween sequence of frames of video S _a, S _a+1, S _a+2..., S _b-1, S _bIn whether have certain frame S _k, after it, frame of two adjacent video frames of a frame is poor all greater than described average λ and less than Second Threshold, if exist described S _kBe candidate's start frame of lysotype gradual change, frame sequence S detected this moment _{K+ α+1}, continue next step calculating and judgement, otherwise continue to detect candidate's start frame of lysotype gradual change, a=5 ~ 8 wherein,

Judge described S _{K+ α+1}..., S _b-1, S _bWhether middle existence exists certain frame S _r, after it, frame of two adjacent video frames of ω frame is poor all less than described average λ, if exist described S _rBe the end frame of lysotype gradual change, make k=r+ ω change step S44 over to and continue to detect, as k detection of end, wherein ω=5 ~ 8 during b.

7. the pick-up unit of a key frame, is characterized in that, comprising:

The distribution probability statistical module, be used for the current video frame of input and the adjacent video frames of described current video frame are carried out non-homogeneous piecemeal, and add up histogram distribution probability and the joint histogram distribution probability of described two adjacent frame of video on each piecemeal and each color component;

The poor computing module of frame, be used for according to the divided group transinformation content of described two adjacent frame of video between the histogram distribution probability on each piecemeal and each color component and described two the adjacent frame of video of joint histogram distribution probability calculating, and poor according to the frame of described two the adjacent frame of video of described divided group transinformation content calculating;

First detection module is used for poor according to the frame of described two adjacent frame of video described current video frame being carried out the initial survey result that key frame for the first time detects to obtain described current video frame;

The second detection module is used for according to the initial survey result of described current video frame, described current video frame being carried out the final detection result that key frame for the second time detects to obtain described current video frame.

8. the pick-up unit of key frame according to claim 7, is characterized in that, described distribution probability statistical module further comprises:

Non-homogeneous minute module unit is used for described two adjacent frame of video according to the ratio of 1:3:1 respectively to long and the wide piecemeal that carries out, obtain 9 non-homogeneous piecemeal m (m=1,2 ..., 9), and give weights W according to the position of described non-homogeneous piecemeal, wherein,

$W=\left[\begin{array}{ccc}{w}_1& w_2& w_3\\ w_4& w_5& w_6\\ w_7& w_8& w_9\end{array}\right]=\left[\begin{array}{ccc}1& 1& 1\\ 2& 4& 2\\ 1& 1& 1\end{array}\right];$

Histogram distribution probability statistics unit, being used for adding up described two adjacent frame of video value on m piecemeal, R color component is the number of pixels of i, and with the sum of all pixels order of described number of pixels divided by the m piecemeal, obtains described histogram distribution probability

With

And obtain successively described histogram distribution probability on G, B color component With

With

Wherein t represents described current video frame, and t-1 represents the adjacent video frames of described current video frame;

Joint histogram distribution probability statistic unit, be used for adding up described two adjacent frame of video value on corresponding m piecemeal, R color component and be respectively the pixel of i and j to number, and with described pixel to the sum of all pixels order of number divided by the m piecemeal, obtain described joint histogram distribution probability

And obtain successively described joint histogram distribution probability on G, B color component

9. the pick-up unit of key frame according to claim 8, is characterized in that, the poor computing module of described frame further comprises:

Single color transinformation content computing unit is used for the histogram distribution probability according to described two adjacent frame of video

And described joint histogram distribution probability

Calculate the transinformation content of described two adjacent frame of video on m piecemeal, R color component according to following formula

$I_{t,t-1}^m\left(R\right)=-\underset{i=0}{\overset{N-1}{\mathrm{\Σ}}}\underset{j=0}{\overset{N-1}{\mathrm{\Σ}}}{p}_{t,t-1}^m(R_i,R_j)*{\log }_2\frac{p_{t,t-1}^m(R_i,R_j)}{p_t^m\left(R_i\right)*p_{t-1}^m\left(R_j\right)},$

And calculate successively described two the adjacent frame of video of acquisition at the m piecemeal, the transinformation content on G, B color component $I_{t,t-1}^m\left(G\right),I_{t,t-1}^m\left(B\right);$

Total color transinformation content computing unit is used for according to described two adjacent frame of video at the m piecemeal transinformation content on R, G, B color component

And Calculate total color transinformation content according to following formula

$I_{t,t-1}^m(R,G,B)=\frac{1}{3}(I_{t,t-1}^m\left(R\right)+I_{t,t-1}^m\left(G\right)+I_{t,t-1}^m\left(B\right));$

Divided group transinformation content computing unit is used for according to described weight matrix W and described total color transinformation content

Calculate described divided group transinformation content I according to following formula _{T, t-1},

$I_{t,t-1}=\underset{m=1}{\overset{9}{\mathrm{\Σ}}}(w_m*I_{t,t-1}^m(R,G,B))/\underset{m=1}{\overset{9}{\mathrm{\Σ}}}{w}_m;$

The poor computing unit of frame is used for according to described divided group transinformation content I _{T, t-1}, according to the poor dist of frame of described two the adjacent frame of video of following formula calculating _{T, t-1},

dist _t，t-1=1-I _t，t-1。

10. the pick-up unit of key frame according to claim 7, is characterized in that, described detection module for the first time further comprises:

Comparing unit is used for first threshold and Second Threshold that the frame of described two adjacent frame of video is poor with predetermined and compares;

The first judging unit is connected with described comparing unit, is used for the comparative result according to described comparing unit, the first detection mode of the described current video frame of judgement,

The first sudden change key frame detecting unit, be used for the judged result according to described the first judging unit, if the frame of described two adjacent frame of video is poor greater than described first threshold, described current video frame may be the sudden change key frame, poor according to annular array storage frames centered by described current video frame, two adjacent video frames each r frame of left and right, wherein, r=3 ~ 5

Judge the frame extent of all 2r+1 adjacent video frames in the poor and described annular array of frame of described two adjacent frame of video, be maximal value if the frame of described two adjacent frame of video is poor, continue next step judgement, otherwise described current video frame not the sudden change key frame

Judge the frame of all 2r+1 adjacent video frames in the poor and described annular array of frame of described two adjacent frame of video poor in time large frame extent, if during the poor frame than all 2r+1 adjacent video frames in described annular array of the frame of described two adjacent frame of video is poor, time large frame is poor large 3 times, described current video frame is the sudden change key frame, otherwise described current video frame is not the sudden change key frame;

The first gradual change key frame detecting unit, be used for the judged result according to described the first judging unit, if the frame of described two adjacent frame of video is poor less than described first threshold and greater than described Second Threshold, described current video frame is the gradual change start frame, calculates the poor dist of frame of described current video frame and k frame of video thereafter _t,k, k=t+1 wherein, t+2 ...,

Judge the poor dist of frame of described current video frame and k frame of video thereafter _t,kWith the size of described first threshold, if the poor Dist of frame of described current video frame and k frame of video thereafter _t,kGreater than described first threshold, described k frame is candidate's end frame,

Calculate the poor dist of frame of described k frame and a frame of video after it _{K+j+1, k+j}, wherein, j=0,1,2 ..., a-1;

Judge the poor dist of frame of described k frame and a frame of video after it _t,kWith the size of described Second Threshold, if the poor dist of frame of described k frame and a frame of video after it _{K+j+1, k+j}Less than described Second Threshold, described k frame is the gradual change end frame.

11. the pick-up unit of key frame according to claim 10 is characterized in that, described first detection module also comprises:

The threshold value setting unit is the poor dist of frame of the frame of video of S for video sequence length between the adjacent video frames of the previous key frame that calculates described current video frame and described current video frame _{I, i-1}(i=1 ..., S-1),

Calculating described sequence length is the poor average μ of frame of the frame of video of S,

Average μ poor according to described frame calculates described first threshold and described Second Threshold, and wherein, described first threshold equals 5 times of the poor average μ of described frame, and described Second Threshold equals 3 times of the poor average of described frame.

12. the pick-up unit of key frame according to claim 10 is characterized in that, described the second detection module further comprises:

Acquiring unit is for the initial survey result of obtaining described current video frame;

The second judging unit is used for the initial survey result according to described acquiring unit, the second detection mode of the described current video frame of judgement,

If the initial survey result of described current video frame is the sudden change key frame, enter the second sudden change key frame detecting unit and carry out the key frame reinspection,

If the initial survey result of described current video frame is the gradual change key frame, enters respectively the second be fade-in fade-out gradual change key frame detecting unit and the second lysotype gradual change key frame detecting unit and carry out key frame and recheck;

The second sudden change key frame detecting unit is used for poor according to described current video frame calculating blocked histogram and pixel difference histogram variance,

Definite first change threshold poor according to described blocked histogram, and determine the second change threshold according to described pixel difference histogram variance,

Judge the size of described pixel difference histogram variance and described the first change threshold and the size of poor and described the second change threshold of described blocked histogram,

If described pixel difference histogram variance is poor less than described the second change threshold greater than described the first change threshold or described blocked histogram, described current video frame is not the sudden change key frame,

If described pixel difference histogram variance is less than or equal to described the first change threshold and described blocked histogram is poor more than or equal to described the second change threshold, described current video frame is the sudden change key frame;

The second gradual change key frame detecting unit of being fade-in fade-out is that the gradual change start frame carries out the sampling of pixel R, G, B to sequence of frames of video between described gradual change end frame to described current video frame,

Judge whether there is complete black sampled point in described sampled point sequence, if there is described complete black sampled point, described current video frame and described gradual change end frame be the gradual change key frame of being fade-in fade-out, the if there is no described sampled point of entirely deceiving, and the initial survey result is flase drop.

The second lysotype gradual change key frame detecting unit is used for two initial survey key frame S of judgement _aAnd S _bBetween sequence of frames of video length whether greater than 30 frames, if described sequence of frames of video length continues next step calculating and judgement greater than 30 frames, if described sequence of frames of video length is not more than 30 frames, described two initial survey key frame S _aAnd S _bBetween do not have lysotype gradual change key frame,

Add up described two initial survey key frame S _a, S _bBetween the poor average λ of frame of adjacent video frames,

Judge described two initial survey key frame S _a, S _bBetween sequence of frames of video S _a, S _a+1, S _a+2..., S _b-1, S _bIn whether have certain frame S _k, after it, frame of two adjacent video frames of a frame is poor all greater than described average λ and less than Second Threshold, if exist described S _kBe candidate's start frame of lysotype gradual change, frame sequence S detected this moment _{K+ α+1}, continue next step calculating and judgement, otherwise continue to detect candidate's start frame of lysotype gradual change, a=5 ~ 8 wherein,

Judge described S _{K+ α+1}..., S _b-1, S _bWhether middle existence exists certain frame S _r, after it, frame of two adjacent video frames of ω frame is poor all less than described average λ, if exist described S _rBe the end frame of lysotype gradual change, make k=r+ ω change step S44 over to and continue to detect, as k detection of end, wherein ω=5 ~ 8 during b.

Images