KR100468843B1 - Method for automatic detection of scene changes in video and apparatus thereof - Google Patents

Abstract

The method and apparatus for automatically detecting a scene change in a moving picture according to the present invention are a histogram-based detection method that adaptively obtains a threshold value for each frame, compares the distance between adjacent frames and the threshold value, and determines a cut candidate. After dividing the selected frame and the previous frame into subblocks, obtain local variance between subblocks and compare them with the variance threshold to detect true cuts. Obtaining; Obtaining histogram distances between adjacent frames based on the distribution function; Obtaining a threshold for each frame based on the histogram distance; And comparing the histogram distance with the threshold value and a predetermined minimum histogram distance, and selecting the cut when the histogram distance is larger than the threshold value and the minimum histogram distance. Based on automatic binarization, it is possible to reduce false scene detection and errors.

Description

Method for automatic detection of scene changes in video and apparatus Technical}

The present invention relates to a method and apparatus for automatically detecting a scene change in a moving picture, and more particularly, to a method and an apparatus for adaptively detecting the scene change.

Most existing methods for detecting cuts are pixel-based, moving picture experts group (MPEG) -based, motion-based, and histogram, depending on what kind of video features are used as clues to predict cut changes. Can be classified as In general, these methods use a similarity measure of video characteristics between successive frames.

One of the simplest methods is contrast-based pixel differential. However, it is often too sensitive to camera, object movement, and noise, resulting in detection errors. In order to reduce the effects of such drawbacks, contrast-based pixel differentiation is used to compensate for the motion.

MPEG-based methods generally use predefined motion information in encoded MPEG video, whereas most other methods require an encoded MPEG bit stream to be decoded first. They have the advantage of being run in real time, but they are too sensitive to the nature of the MPEG encoder and the encoded bitrate.

Motion-based methods require motion estimation techniques. These algorithms often require too much time to calculate accurate motion information and do not guarantee good performance due to errors in calculating pixel differences when there is a sharp contrast change when the contrasts of consecutive frames are similar.

"Performance Chararcterization of Video-Shot Change Detection Methods" by ULLas Gargi, Rangachar Kasturi, and Susan H. Strayer, IEEE Transanction on circuits and systems for video technology, vol. As can be seen from 10, No. 1, Feb. 2000, histogram-based methods have proven to outperform other methods. These are classified into two items. The first is a global histogram-based method where the histogram is calculated over the entire frame. And if a histogram difference between two consecutive frames exceeds a threshold, a cut is declared. And second, a local histogram-based method, in which case the histogram is calculated for each block of the divided frame. They have the drawback that two consecutive frames in a cut state have a similar histogram distribution but are incorrectly transitioned if they are not visually similar.

The algorithms mentioned above have a problem in that stable performance cannot be guaranteed because they apply fixed thresholds to different videos. In order to solve this problem, various literatures (Hain-Ching Liu and Greg Zick, "Automatic Determination of Scene Changes in MPEG Composed Video," Circuits and Systems, ISCAS '95, IEEE International Symposium on, Volume 1, 1995 (" First Advanced Technology ”, Bilge Gunsel and A. Murat Tekalp,“ Content-Based Video Abstraction, ”Imange Processing, ICP 98, Processing. 1998 International Conference on, 1998. )) Is presented. However, the first leading technique depends on the MPEG encoder and bit rate used, and the second leading technique can guarantee good performance only when the user knows the characteristics of the target video. In addition, there is a problem in that video is generally overlooked as being composed of video clips having various characteristics.

The technical problem to be achieved by the present invention is to obtain a histogram distance between frames by applying the histogram-based method as a whole, and to set a threshold value for each frame based on the frame, and a program for automatically detecting a scene change and a program for executing the method on a computer. To provide a computer-readable recording medium for recording.

Another technical problem to be solved by the present invention is to obtain a histogram distance between frames by applying the histogram-based method as a whole, and to select a cut candidate by setting a threshold value for each frame based on the frame histogram. A computer-readable recording medium that records a local variance and determines a true cut when its value exceeds the variance threshold, automatically detecting scene changes, and a device and a program for executing the method on a computer. To provide.

1 is a flowchart showing steps for automatically detecting a scene change according to the present invention.

FIG. 2 is a flowchart illustrating a flow of a specific embodiment for implementing the steps of FIG. 1.

3 is a block diagram of an apparatus for automatically detecting a scene change according to the present invention.

According to an aspect of the present invention, there is provided a method for adaptively detecting a scene change according to the present invention, the method comprising: obtaining a predetermined distribution function representing a brightness distribution of pixels for each frame in video data; Obtaining histogram distances between adjacent frames based on the distribution function; Obtaining a threshold for each frame based on the histogram distance; And selecting the cut when the histogram distance is greater than the threshold and the minimum histogram distance by comparing the histogram distance with the threshold and the predetermined minimum histogram distance.

An embodiment of the method for automatically detecting a scene change according to the present invention will be described together with an embodiment of another technical problem described below to avoid duplication.

According to another aspect of the present invention, there is provided a method for adaptively detecting a scene change according to the present invention, comprising: obtaining a color histogram representing a brightness distribution of pixels for each frame in video data; Obtaining a histogram distance between adjacent frames based on the color histogram and obtaining a threshold value for each frame based on the value; Selecting as a cut candidate when the histogram distance is greater than the threshold and a predetermined minimum histogram distance; Obtaining a variance between a frame selected as the cut candidate and a frame immediately preceding the cut candidate; And determining, as a cut, a frame whose dispersion exceeds a predetermined variance threshold among the frames belonging to the cut candidates.

According to another aspect of the present invention, there is provided an apparatus for adaptively detecting a scene change according to the present invention, including: a data input unit extracting a DC sequence from video data; A histogram calculator for obtaining a color histogram from the DC sequence; A selecting unit which determines a cut candidate by obtaining a histogram distance between frames by inputting the color histogram value and adaptively obtaining a threshold value for each frame based on the histogram distance; A decoding unit which decodes and outputs a frame determined as the cut candidate and output; And an output unit for dividing the decoded frame and the immediately preceding frame into predetermined subblocks, and then determining a cut among the cut candidates by comparing a variance threshold value obtained by obtaining a variance for each subblock.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart showing a step for automatically detecting a scene change according to the present invention, FIG. 2 is a flowchart showing a flow of a specific embodiment for implementing the step of FIG. 1, and FIG. 3 is a scene change according to the present invention. Is a block diagram of an apparatus for autodetecting a.

After explaining the scene change detection method according to the present invention with reference to FIG. 1, a detailed embodiment will be described with reference to FIGS. 2 and 3.

Basically, the scene change detection method according to the present invention automatically detects a cut change using a global histogram based method and a local variance. First, the data input unit 301 receives a compressed video and extracts a DC sequence. The histogram calculator 303 receives the DC sequence and calculates a color histogram for each frame. The color histogram can be calculated after decoding the compressed video, but if it does, there is a drawback of significantly delaying the processing time while increasing the resolution of the image. Therefore, the compressed video is not decoded and the algorithm of the first prior art is applied to obtain an approximation of the DC value. This is because even in doing so, when the scene change detection method according to the present invention is applied, excellent results can be obtained in terms of both processing speed and performance (step 101). Next, the selector 305 obtains the histogram distance between frames by inputting the color histogram value output from the histogram calculator 303, and selects a cut candidate by applying threshold values for each frame to be described below based on the histogram distance between frames. That is, the selector 305 calculates a threshold for cut detection, which is adaptively calculated based on local statistics of the histogram distance. See below for details.

When Dt is the distance (distance) of the color histogram in the t-th frame, that is, the distance between successive frames in frame t and frame t-1, it is defined as Equation 1 below.

Where H _t is the histogram in frame t calculated by the DC sequence, B is the total number of histogram bins, and T is the histogram normalization factor, which means twice the total number of pixels in each frame.

The threshold in frame t is determined as in Equation 2 below by means of the neighboring interframe histogram distance between frames t-L / 2 and t + L / 2.

Where L is the size of the moving window and K is a constant.

D _t is the minimum boundary D _min or more for determining the Th _t and frame t cut is classified as a candidate for the cut boundary. Where D _min is empirically assigned an appropriate value.

This automatic thresholding technique is a process of finding featuredistance or histogram distance that is more pronounced than neighboring features. As a result, if the ratio of the average of the current feature distance and the neighboring feature distance is greater than K set small enough to detect all the actual cuts, the cut candidate is declared (step 103). The global histogram method using a fixed threshold has two problems during cut detection. One is that when video is dark, black and white or blue tones, there is a tendency to decide that the cut is not a true cut. Because the characteristic distances are smaller than usual. The other tends to generate false alarms because the characteristic distances for video with fast movement or rapid luminance changes are larger than usual. Thus, automatic binarization based on local statistics of the characteristics can reduce false cut detections and alarms caused by the above problems. In order to detect true cuts from the cut candidates generated by the histogram difference threshold, the decoding unit 307 decompresses the frames selected as the cut candidates.

On the other hand, there is a possibility that wrong selection may be made due to a rapid change in luminance in the local area during the detection of the cut candidate. However, in order to rule out this possibility of error, we use a local variance that is computed in each subblock divided into N * N frames. The local variance difference between successive frames in frames t and t-1 is given by Equation 3 below. Accordingly, the output unit 309 divides the decoded cut candidate frame into a predetermined number of subblocks, and then calculates a local variance in a non-overlapping subblock as shown in Equation 3 below to determine a true cut. do.

Where H and V are the horizontal and vertical number of blocks, respectively. Var _t (i, j) is the variance of the video characteristic values in the (i, j) th subblock of frame t.

Finally, the selector 309 determines and outputs a true cut among the candidates only when DV _t is equal to or greater than Th _{var, which} is a preset value (step 105).

A detailed flow of the adaptive screen switching detection method according to the present invention will be described with reference to FIG. 2. However, in order to avoid duplication of explanation, the same description is omitted.

First, as in step 101 described above, an arbitrary t-th DC sequence (hereinafter referred to as F _t ) is obtained (step 201). If it is determined that this F _t is the end of the input sequence, if there is no further work, the process proceeds to the end step, otherwise it proceeds to the normal process step (step 202). Next, a color histogram H _t is obtained for the received sequence (step 203). At this time, it is determined whether t is 1, that is, the current frame is the first frame (step 205). If t = 1, the first frame is declared as a cut since the video starts. Otherwise, since the frame is in progress, the histogram distance D _t is calculated according to Equation 1 (step 207). Then, it is determined whether t is larger than the size of the moving window (L), which is a preset value, that is, the distance between the current frame and the first frame is compared with the size of the moving window (step 208). The process is repeated for the next frame, and if it is large, a value of tL / 2 + 1 is designated as a new t2 value in order to calculate the equations with the frame at the center of the moving window (step 209). One plus here is because the frame starts at one.

Next, Equation 2 is applied to the t2th frame. That is, a threshold value is obtained by calculating an average value of the histogram distance Dt for the frames within the L / 2 range from the left and right centering on the t2th frame (step 210). It is determined whether the histogram distance Dt ₂ obtained in the t2 th frame is larger than the threshold Th _t2 and larger than the preset minimum distance D _min (step 211). In other words, determining whether the histogram distance is larger than the threshold Th _t2 and the minimum distance D _min is a process of checking whether a sudden scene change or cut occurs in the moving window. The reason for considering the minimum distance is also to prevent Dt _{2 from} becoming a cut candidate if all other histogram distance values in the moving window are smaller, even though Dt ₂ is so small that it can be ignored. If it is large, a cut has occurred, so it is selected as a cut candidate. The cut candidate frame, that is, the t2th frame and the frame immediately before that, that is, the t2-1th frame, is decoded to decompress to obtain the original video data, and then divided into subblocks as described above. Since decoding is only required for cut candidates, the scene change detection method according to the present invention can process in real time much faster than other algorithms requiring reprocessing for all frames. Thereafter, DV _t2 is obtained by the equation expressed in Equation 3 above for the local variance between the subblocks of the frames (step 212). If the DV _t2 is larger than the preset dispersion threshold, the t2th frame is declared as a cut, and if the DV _t2 is smaller, it is determined that no scene change occurs (step 214).

In order to verify the performance of the adaptive scene change detection method and the apparatus according to the present invention, an experiment was conducted using five movie clips and eight commercial movies. The five movie clips are two "Gladiator" clips, a "Waterboy" clip, a "Scary Movie" clip, and a "Roman Holiday" clip. Most of the sequences of the eight commercial films have a rapid movement and a sudden change in luminance. The video clips have a sample rate of 29.97 frames per second and a resolution of 352 * 240, which is why the size of the DC sequence for calculating the color histogram is 44 * 30.

The parameters for the experiment are shown in Table 1.

parameter value K 2.68 L 8 Th _var 2.00 D _min 0.08

Table 2 shows a comparison result between the overall color histogram method (CH) having a fixed threshold set to 0.2 and the case (PM) according to the present method.

Video clip Total frames Real cut Correct detection (CH) Correct Detection (PM) False detection (CH) False detection (PM) Gladiator (slow) 3308 18 18 18 0 0 Gladiator (fast and blue tone) 16,500 219 134 204 35 7 Roman Holiday (moderate and black & white) 3501 14 10 14 0 0 Scary Movie (fast and dark) 4905 60 49 59 5 6 Water Boy (fast and rapid luminance change) 4800 64 63 64 51 4

Both methods detected all cuts in the slow motion "Gladiaor". However, in the video having fast motion, dark parts, black and white, and blue tone, the scene change detection method according to the present invention showed better performance than the color histogram method. In particular, the performance was much better when combining fast motion with either blue tones, fast, dark, or fast and rapid brightness changes, as well as monotone video.

Through the results of Table 2, the recall and precision of the conventional method and the detection method according to the present invention can be calculated as in Equations 4 and 5 below.

Here, Detects is the number of known cuts, MissedDetects is the number of undetected cuts, and FalseAlarms is the number of cuts that are not cuts.

Table 3 shows the result of calculating this recovery rate and accuracy. It can be seen that the detection method according to the present invention is excellent also in the recovery rate and accuracy.

Recall Precision CH PM CH PM 72.6% 95.9% 72.6% 96.3%

The invention can also be embodied as computer readable code on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like, which are also implemented in the form of a carrier wave (for example, transmission over the Internet). It also includes. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

As described above, according to the method and apparatus for adaptively detecting a scene change according to the present invention, false scene change detection and error can be reduced by automatic binarization based on local statistics of video characteristics. .

According to Equations 1 and 2 described above, thresholds can be adaptively determined frame by frame, so that they can be simultaneously applied to various types of video regardless of their properties.

In addition, by applying local variance in determining the cut, Equation 3 above can significantly reduce false detection due to rapid movement and rapid luminance change.

Since the scene change detection method and apparatus according to the present invention can be applied to DC images of MPEG compressed video, it is more efficient than other detection algorithms that require fully decoded images in terms of time cost for executing the algorithm. In fact, it is desirable that decoded images be adopted to obtain good performance. However, this method shows good performance in real time because the number of cut candidates by the detection method according to the present invention is much smaller than the total number of frames.

Claims (18)

(a) obtaining a predetermined distribution function representing the brightness distribution of the pixels for each frame in the video data; (b) obtaining histogram distances between adjacent frames based on the distribution function; (c) setting a predetermined moving window range and obtaining a threshold Th having a normalized value for each frame within the range as shown in Equation 6 below; Where K is a predetermined constant and L is the size of the moving window (d) If the histogram distance is larger than the threshold and the minimum histogram distance by comparing the histogram distance with the threshold and the predetermined minimum histogram distance, the candidate is selected as a cut candidate, and immediately before the frame selected as the cut candidate and immediately before the histogram distance. After dividing the frame into a predetermined number of subblocks, a local variance (DV _t ) between the subblocks is obtained as in Equation 7 below. , Where H and V are the number of horizontal and vertical directions of the subblock, respectively, and Var _t (i, j) is the variance in the (i, j) th subblock of frame t) And comparing the local variance with a predetermined variance threshold to determine a true cut if the variance threshold is exceeded. The scene change detection method according to claim 1, wherein the distribution function is a color histogram. The method of claim 1, wherein step (b) Calculating a difference value between the histogram of an arbitrary second frame and the histogram of the immediately preceding first frame by the number of histogram bins to a normalized value. Scene change detection method, characterized in that. delete The method of claim 3, wherein the histogram distance (D _t ) is Where H _t is the histogram in the t th frame (t is a positive integer), B is the total number of histogram bins, and T is twice the total number of pixels in the frame. Scene transition detection method characterized in that obtained by. delete delete delete delete The method of claim 1, wherein step (d) And if the video data is compressed, obtaining the local variance after decoding the frame selected as the cut and the frame immediately before the cut. delete delete delete delete delete delete A data input unit extracting a DC sequence from the video data; A histogram calculator for obtaining a color histogram from the DC sequence; The histogram distance between frames is obtained by inputting the color histogram value, and a predetermined moving window range is set to adaptively set a threshold having a normalized value for each frame based on the histogram distance within the range. A selection unit for determining and determining a cut candidate; A decoding unit which decodes and outputs a frame determined as the cut candidate and output; When the frame selected and decoded as the cut candidate and the frame immediately before the frame are divided into a predetermined number of subblocks, a local variance is obtained between the subblocks, and the true threshold is exceeded when the variance threshold is exceeded. Scene output detection apparatus comprising a; output unit for determining. A computer-readable recording medium having recorded thereon a program for executing the steps of claim 1 on a computer.