KR101358664B1 - A method for detecting shot bounddary using brightness comparation between asymmetrical correspondence regions - Google Patents

Abstract

The present invention relates to a shot switch detecting method between two successive frames of a video image, which includes the following steps of: setting the size of a first small area of a current frame; setting a second small area which is larger than the first small area in a position which corresponds to the first small area with respect to a reference frame which is next to the current frame; and determining whether a shot is switched or not by comparing a brightness value which is obtained from the first small area and a brightness value which is obtained from the second small area. The present invention is able to efficiently provide services including a video index, a search, and an event detection because a shot switch is accurately detected even in case a motion of an object which exists between a current frame and a reference frame exists by sensing the shot switch through a brightness comparison between asymmetric areas.

Description

A method for detecting shot transitions by comparing brightness against asymmetric counterparts {A METHOD FOR DETECTING SHOT BOUNDDARY USING BRIGHTNESS COMPARATION BETWEEN ASYMMETRICAL CORRESPONDENCE REGIONS}

As a technique of a method for detecting a shot change in a video image of the present invention, in detail, a corresponding comparison area between two consecutive frames is asymmetrically set in consideration of movement of an object that may occur between two frames, The present invention relates to a technique for detecting shot transitions more accurately by comparing brightness of asymmetrically set corresponding areas.

The development of digital technology has increased the use of computing resources along with the rapid generation of video data, and related services such as storage, retrieval, and classification are required for efficient management of video data. A video is composed by continuously editing frames taken using one or more cameras, and shots of a series of scenes taken by one camera in a video, and switching between two consecutively connected shots. The part to be cut is called a cut. One shot is the smallest unit of consistency in the content of the video due to the camera's parameters (motion, magnification, reduction, focus) and spatial and temporal continuity. Shot boundary detection is the basis for efficiently providing services such as indexing, browsing, and event detection of large-capacity video data.

Conventional shot transition detection methods include a method of comparing pixels between frames, a method using a histogram, a method using an edge, and the like, and these methods are characterized by a difference in feature values extracted from all or part of two consecutive frames (current and reference frames). If the threshold is greater than or equal to a preset threshold, a shot transition between two consecutive frames is detected.

As a specific example, a technique of using a mean difference between frames after a difference in brightness values between pixels corresponding to each other between two consecutive frames for switching detection is known.

Wherein MAFD _n (mean absolute frame differences) is n to the average value of the absolute differences of the pixel gray level between the first frame and the (n + 1) th frame, the number M, N is the total pixels in the vertical direction and the horizontal direction of the frame, f _n (i, j) represents the brightness value of the (i, j) -th pixel in the n- th frame.

Here, the method using the difference of gray-level histogram calculated in two consecutive frames for the shot detection, the method using the chromatic histogram obtained from the pixel value of the frame, and the color calculated in two consecutive frames Histogram.

The method using the color histogram may use Equation 2 below.

The CHD _i (color histogram difference) is i is the absolute difference in color histogram between the second frame and the (i + 1) th _{frame, P i (r, g,} b) is the frame having the W pixel color (r, g, b), and B is the number of colors that the pixel can represent.

Conventional techniques for detecting shot transition using the brightness value of the pixel as described above 1) compare the brightness value of the pixel of the current frame and the brightness value of the pixel at the same position in the reference frame. 2) the brightness value of one subregion of the current frame and the corresponding subregion in the reference frame, or the brightness value of the corresponding region in consideration of motion information; or 3) the brightness value of the entire region of the current frame and the overall brightness of the reference frame. It takes a way to detect shot transitions by comparing the values.

However, these conventional techniques incorrectly detect that a shot transition occurred when the object existing in the current frame or the small region under examination of the current frame leaves the corresponding small region in the reference frame or the reference frame due to the movement, and the frame is incorrectly detected. Even when there is a flicker phenomenon in which the brightness of the liver changes, there is a problem of incorrectly detecting that a shot transition has occurred.

As such, a pixel comparison method and a histogram method for comparing corresponding pixels or regions between two consecutive frames are commonly used due to the simplicity of calculation, but are vulnerable to changes in object movement or pixel brightness.

The present invention has been proposed to solve the above-mentioned conventional problems, and an object of the present invention is to prevent false detection due to movement of an object occurring between two consecutive frames through asymmetrical area comparison.

Another object of the present invention is to provide a detection technique that is more robust to the change in brightness between frames by solving the conventional problem that is vulnerable to the change in brightness occurring between frames.

According to an aspect of the present invention for achieving the above object, a method for detecting a shot transition between two consecutive frames of a video image, the step of setting the size of the first small region of the current frame, to the current frame Setting a second small region having a size larger than the first small region at a position corresponding to the first small region with respect to successive reference frames, and a brightness value obtained from the first small region and the second small region; There is provided a shot change detection method by comparing brightness for an asymmetric corresponding area, comprising the step of determining whether to switch shots by comparing the brightness values obtained from the above.

Here, the size of the second small region is preferably determined as the size of the first small region plus the maximum movable distance of the object generated between two frames.

The step of comparing the brightness value obtained from the first small region with the brightness value obtained from the second small region to determine whether to switch shots includes obtaining a pixel brightness histogram obtained from the first small region from the second small region. If the histogram obtained from the first subregion is larger than the preset threshold, the two consecutive frames belong to the same shot. You can judge.

In addition, it is more preferable to generate an expansion histogram by applying a gray level expansion operation to the histogram obtained from the second subregion of the reference frame, and compare the histogram and the expansion histogram of the first subregion to determine whether to switch shots. Do.

According to the present invention, by detecting the shot transition by comparing the brightness between the asymmetric region, even if the movement of the object existing between the current frame and the reference frame can be detected accurately whether the shot transition, video index, search, event detection, etc. There is an effect that can provide a more efficiently service.

1 is a conceptual diagram illustrating an asymmetric correspondence region setting relationship between a current frame and a reference frame.
FIG. 2 illustrates the principle of setting the size of the first small region of the current frame and the second small region of the reference frame.
3 is a photograph showing an example of a small region in an actual video.
FIG. 4 illustrates a brightness histogram of small regions of a current frame and a reference frame set in FIG. 3.

Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

In the shot change detection method according to the present invention, the size of the first small region of the current frame is set, and a reference frame continuous to the current frame is located at a position corresponding to the first small region than the first small region. A second small region having a large size may be set, and a brightness value obtained from the first small region may be compared with a brightness value obtained from the second small region to determine whether to switch a shot. In FIG. 2, a method of setting a small region in the current frame and a reference frame will be described. In FIGS. 3 and 4, a method of detecting a shot change by comparing brightness values obtained from the small region of the current frame and the reference frame will be described.

FIG. 1 is a conceptual diagram illustrating an asymmetric correspondence region setting relationship between a current frame and a reference frame, and FIG. 2 illustrates a principle of setting a size of a first small region of a current frame and a second small region of a reference frame.

FIG. 1 illustrates a method of partitioning regions in consideration of movement of an object. Shots taken by one camera consist of several video frames acquired at regular intervals, and due to these temporal differences, the movement of an object existing between the current frame and the reference frame is incorrect in shot transition detection. Is one cause. As shown in FIG. 1, the current frame and the reference frame are divided into a plurality of first subregions 10 and a second subregion 20, respectively. To compensate for the movement of the object, the size of the second subregion 20 of the reference frame is equal to the size of the first subregion 10 of the current frame plus the distance d that the object can move between two consecutive frames. The size is determined.

Therefore, the current frame secures the unset area 30 by the distance d inward so that the small area within the reference frame corresponding to the small area located at the corner of the current frame can be set, and the center area excluding the unset area. Is divided into a plurality of small regions.

The size of the first small region 10 of the current frame is expressed by Equation 3 below.

Here, B _w and B _h are the horizontal and vertical sizes of the first small region 10, f _w and f _h are the horizontal and vertical sizes of the frame, and d is the distance that the object can move between two consecutive frames.

The size of the second small region 20 of the reference frame is (B _w + 2d) ㅧ (B _h + 2d), and the center point of the second small region 20 of the reference frame is the first small region 10 of the current frame. Coincides with the center point.

As described above, even when the movement of the object occurs between successive frames by setting the size of the second small region 20 to the size of the first small region 10 plus the maximum inter-object maximum moving distance. By allowing all of the moving components of the object to be included in the second small region 20, it is possible to prevent the shot change erroneous detection due to the object movement.

3 is a photograph showing an example of a small region in an actual video, and FIG. 4 illustrates a brightness histogram of small regions of a current frame and a reference frame set in FIG. 3.

In FIG. 3, (a) shows the setting of the first small region of the current frame, and (b) shows that the second small region having a size larger than the first small region is asymmetrically set at a position corresponding to the first small region. have.

  4A illustrates a brightness histogram of the first small region of the current frame of FIG. 3, and FIG. 4B illustrates a brightness histogram of the second small region of the reference frame.

In FIG. 4, the histogram (d) obtained from the second small region 20 of the reference frame includes more information than the histogram (c) obtained from the first small region 10 of the current frame. The histogram obtained from the second subregion 20 of the reference frame has information included in the histogram obtained from the first subregion 10 of the current frame, and has motion information of an object between two consecutive frames. Therefore, if two consecutive frames belong to the same shot, the histogram obtained from the first subregion 10 of the current frame should be included in the histogram obtained from the second subregion 20 of the reference frame. As follows.

Here, B _{P, J} is the j-th small region of the p-th frame, and B _{P + 1} , j is the j-th small region of the (p + 1) th frame.

A comparison between the j-th small region histogram of the current frame and the j-th second small region histogram of the reference frame is expressed by Equation 5.

Where n is the number of bins in the histogram, and H _{j, r} (k) and H _{j, c} (k) represent the number of pixels whose brightness is k in the jth small region of the reference frame and the current frame.

If the value of B _j is larger than the set threshold t1, it is assumed that the second subregion of the reference frame to be compared includes all the information of the first subregion of the current frame (Equation 6).

Finally, if the second subregion of the reference frame that contains all the information of the first subregion of the current frame is larger than the threshold t ₂ set by the number, two consecutive frames are in the same shot or a shot transition has occurred. do.

Here, n is the number of divided small regions of each frame, and d _i is the number of reference frames including all the information of the small region of the current frame.

On the other hand, as described above, the pixel comparison method or the method using the histogram has a problem in that the change in the brightness of the pixel is vulnerable. In order to solve this problem, the present invention solves the problem of weakness in brightness change when comparing interframe histograms for shot detection by applying a grey-level expansion operation of morphology to the histogram of the second subregion of the reference frame.

Here, H _r (j) represents the j th bin of the histogram of the second small region, S represents a one-dimensional morphology mask for morphology calculation, and x represents the size of the mask.

 If component S is 7 and is in the j th bin of the histogram, Hr (j) is Hr (j-3), Hr (j-2), Hr (j-1), Hr (j), Hr (j The maximum value is taken from +1), Hr (j + 2) and Hr (j + 3).

When the gray-level expansion operation is applied to the histogram shown in FIG. 4 (d), the expanded histogram of the form in which the histogram is expanded left and right as shown in FIG. 4 (e) can be obtained.

The expanded histogram of (e) has the information included in the histogram obtained from the first subregion 10 of the current frame, and also includes information on the compensation of the change in brightness in addition to the motion information of the object between two consecutive frames. Detection is possible.

 In order to evaluate the performance of the shot transition detection method proposed in the present invention, the following six digitized motion picture films (four color motion picture films and two black and white motion picture films) were used in the digitized motion picture film. The film used in the test was shot between the 1960s and 1970s, using a 'telecine' device to digitize an analog film's movie (720 x 480 MPEG2 format video), containing only radical shot transitions. . The number of frames and shot transitions of the digitized motion picture film used is shown in Table 1.

Digitized color video was first converted to black and white video, and gray-level histograms were used to compare two adjacent frames. The moving distance of the object between two consecutive frames is 20 pixels, and a one-dimensional morphology mask of length 10 is used to compensate for the allowable change in brightness.

The criteria for evaluating the performance of the proposed shot detection algorithm are recall and precision.

Here, the reproducibility may be expressed as the number of detected shots / (detected shots + number of undetected shots), and the accuracy may be expressed as the number of detected shots / (detected shots + number of false shots).

As a result of the experiment, in case of video data such as a movie or news, the moving distance of the object between successive frames was within the threshold range determined by the experiment, and the shot transition detection result was excellent.

Although the present invention has been described in connection with the above-mentioned preferred embodiments, it is possible to make various modifications and variations without departing from the spirit and scope of the invention. Accordingly, the appended claims are intended to cover such modifications or changes as fall within the scope of the invention.

10: first small area
20: second small area
30: unset area

Claims (5)

A method for detecting shot transitions between two consecutive frames of a video image,
Setting a size of a first small region of the current frame;
Setting a second subregion larger in size than the first subregion at a position corresponding to the first subregion with respect to the reference frame subsequent to the current frame;
And determining whether to switch shots by comparing the brightness value obtained from the first small region with the brightness value obtained from the second small region.
The method of claim 1,
The size of the second small region is determined as the size of the first small region plus the maximum movable distance of the object generated between the two frames, the shot switching detection method by comparing the brightness of the asymmetric corresponding region, characterized in that .
The method of claim 1,
The step of determining whether to switch the shot by comparing the brightness value obtained from the first subregion and the brightness value obtained from the second subregion
Verify that the pixel brightness histogram obtained from the first small region is included in the pixel brightness histogram obtained from the second small region, and that the histogram obtained from the first small region is included in the histogram obtained from the second small region. When the number is greater than the predetermined threshold value, the method of detecting a shot change by comparing the brightness of the asymmetric corresponding region, characterized in that it is determined that two consecutive frames belong to the same shot.
The method of claim 3, wherein
An asymmetry characterized by generating an expansion histogram by applying a gray level expansion operation to the histogram obtained from the second subregion of the reference frame, and comparing the histogram of the first subregion with the expansion histogram to determine whether to switch shots Shot transition detection method by comparing the brightness of the corresponding area.
5. The method of claim 4,
The gray level expansion operation is a shot conversion detection method by comparing the brightness of the asymmetric corresponding region, characterized in that by the following equation.

Where Hr (j) is the j th bin of the histogram of the second subregion, S is the one-dimensional morphology mask for morphology calculation, and x is the size of the mask.

Images