JP2007134986A - Shot boundary detection device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a shot boundary detection device which is capable of accurately extracting the shot boundary of a low-luminance image, a monochrome image, or even a moving image that contains the unchanged part of an image in the side region of a picture screen. <P>SOLUTION: An image partially decoded through a variable-length decoding unit 1 is subjected to trimming wherein the upper and lower region or the left and right region of the image divided in a horizontal direction and/or a vertical direction by a center region extraction unit 2 are discarded so as to extract the center region of the image. The image of the center region is inputted into an interframe luminance difference value operation unit 3, a color difference histogram correlation value operation unit 4, and a darkness judging unit 5. The darkness judging unit 5 acquires the average luminance value of the frames. When it is found that the average luminance value of the frames is smaller than a prescribed threshold; the image of the center region is judged to be a low-luminance image or a monochrome image, and a shot boundary judging unit 6 is informed of the judgement result. The shot boundary judging unit 6 performs shot boundary judgement processing by multiplying an interframe luminance difference value D<SB>n</SB>by γ(>1) when the image is judged to be a low-luminance image or a monochrome image. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a shot boundary detection apparatus, and more particularly to a shot boundary detection apparatus that can accurately detect a shot boundary from a moving image including a dark image or an image with subtitles.

  An example of a conventional moving image cut point detection device is disclosed in Patent Document 1 below. The patent document 1 shows what is shown in FIG. 5 as an outline of the moving image cut point detection processing.

  When an image is input in step S21, an encoding parameter is extracted from the image in step S22, and it is determined in step S23 whether or not it is an instantaneous cut point using the extracted encoding parameter. When this determination is negative, the process proceeds to step S24 to perform dissolve detection as a special cut screen, and further proceeds to step S26 to perform wipe detection. On the other hand, if it is determined in step S23 that the point is an instantaneous cut point, it is determined in step S29 whether or not flash detection is performed. If it is determined that flash detection is not performed, an instantaneous cut is determined, and instantaneous cut registration is performed in step S30. The In steps S25 and S27, dissolve registration and wipe registration are performed, respectively.

In Patent Document 2 below, as one method of the instantaneous cut point detection, an instantaneous change in frames n−1, n, and n + 1 using inter-frame luminance difference values, histogram correlation values, and color difference histogram correlation values is described. And identifying whether the screen is a cut screen or a non-cut screen.
JP 11-252509 A Japanese Patent Laid-Open No. 7-236153

  However, in the instantaneous cut point detection method disclosed in Patent Document 2, in a dark image, a black-and-white image, or an image in which an invariant portion of an image such as subtitles or market information enters at the top or bottom of the screen, a shot boundary There has been a problem that the detection accuracy is low.

  The object of the present invention is to solve the above-described problems of the prior art and to accurately detect shot boundaries even in a low-brightness image, a black-and-white image, or a moving image including an image having an invariant portion in an area close to a side piece of the screen. An object of the present invention is to provide a shot boundary detection device that can be well extracted.

  In order to achieve the above object, the present invention comprises means for partially decoding an encoded image, means for removing an area where an invariant part of an image such as a caption enters from the partially decoded image, and the removal. The first feature is that it includes means for detecting a shot boundary using an image of a region that has not been present.

  Means for partially decoding the encoded image; and means for dividing the partially decoded image into m (m ≧ 3) in the horizontal or vertical direction and removing at least one of the upper and lower or left and right peripheral divided regions. There is a second feature in that a means for detecting a shot boundary using an image of the area that has not been removed is provided.

Further, in the present invention, the means for detecting the shot boundary is a means for obtaining an inter-frame luminance difference value D _n , an average luminance value of the image is detected, and the average luminance value is smaller than a predetermined threshold value. A third feature is that it comprises means for multiplying the inter-frame luminance difference value D _n by γ (γ> 1), and means for making a shot boundary candidate if γD _n is greater than a predetermined threshold Th. There is.

  According to the present invention, since the shot boundary can be detected using an image area that does not include an invariant part such as a caption, the detection accuracy of the shot boundary can be improved.

  Further, according to the present invention, the shot boundary is detected using an image obtained by dividing an image into m (m ≧ 3) in the horizontal or vertical direction and removing at least one of the upper and lower or left and right peripheral divided regions. Therefore, even if invariant information such as subtitles and market information is placed on the upper and lower or left and right peripheral areas of the image, there is a high possibility that these will be cut. For this reason, the shot boundary is detected only from the image information, and the detection accuracy of the shot boundary can be improved.

In the case of such a low brightness image or a monochrome image, since the correction to be substantially larger luminance difference value between frames by multiplying the value greater than one γ on the luminance difference value D _n between the frames, the low-luminance image It is possible to improve the accuracy of shot boundary detection in the case of a black-and-white image or the like.

  Hereinafter, the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of a shot boundary detection apparatus according to the present invention. This shot boundary detection corresponds to the processing in step S23 of FIG.

  The shot boundary detection according to the present invention is detected from partially decoded (or partially decoded) image data. That is, for example, an encoded image stream a encoded by MPEG or the like is input to the variable length decoding unit (VLD) 1 and partially decoded. The partially decoded image data b is then input to the central area extraction unit 2 and an area in which an invariant part of an image such as a caption enters is removed. As a specific example, the central region extraction unit 2 extracts a central region of an image divided into m (m ≧ 3) in the horizontal direction. For example, as shown in FIG. 2, the image is divided into four parts, two central divided areas (shaded areas in the figure) are extracted, and one area above and below is removed. This is because invariant information such as subtitles and market information may be displayed in each of the upper and lower areas of one frame of image data b. Note that only one of the upper and lower regions may be removed.

The inter-frame luminance difference value calculation unit 3 obtains the inter-frame luminance difference value D _n by the calculation of the following equation (1).

Here, M and N are the total number per one frame in the vertical and horizontal directions of a block composed of 8 &times; 8 pixels, respectively. Y _n (i, j) is the average luminance block value of the block (i, j) of the nth frame.

Further, the color difference histogram correlation value calculation unit 4 obtains the color difference histogram correlation value ρ _n by the following equation (2).

Here, H _{n, k, l} (k, l = 0, 1, 2,..., Hc−1) is obtained by classifying the DC color difference signals Cb and Cr data in one frame into the hc class. The histogram is described in detail in paragraphs [0029] to [0032] of Patent Document 2, for example.

Darkness decision unit 5 calculates the average luminance value AveY _n of one frame (3) below, is smaller than the threshold Th_d that the average luminance value is predetermined, it is determined that the low brightness or black and white images, This is notified to the shot boundary determination unit 6.

Next, the shot boundary determination unit 6 uses the inter-frame luminance difference value D _n and the chrominance histogram correlation value ρ _n obtained by the inter-frame luminance difference value calculation unit 3 and the chrominance histogram correlation value calculation unit 4 to perform a shot. Determine the boundary.

Next, the operation of the present embodiment will be described with reference to the flowchart of FIG. In step S1, the encoded image stream a is input to the variable length decoding unit 1. In step S2, the central area extraction unit 2 divides the image into a plurality of parts in the horizontal direction, and the central area is selected. In step S3, the inter-frame luminance value difference calculation unit 3 obtains the inter-frame luminance difference value D _{n according} to the equation (1). In step S4, the darkness determination unit 5 obtains the average luminance value aveY _n per frame by the above equation (3). In step S5, it is determined whether or not the average luminance value aveY _n is smaller than the threshold value Th_d or a monochrome image. If this determination is affirmative, it is determined that the image is dark (low luminance image) or a black and white image, and the process proceeds to step S6. In step S6, the constant in the inter-frame luminance difference value _{D n} gamma (although, γ> 1.0) is multiplied by. Then, the process proceeds to step S7 to determine whether or not D _n > Th_pre is satisfied. In the low brightness image or a monochrome image, generally interframe luminance difference value D _n is reduced, because the shot boundary is less likely to be detected. When the determination in step S5 is negative, that is, when the image is bright, step S6 is skipped. Incidentally, the darkness determined by the average luminance value AveY _n determined in multiple stages in the step S5, may be set to different values in accordance with the constant γ in step S6 to the degree of the dark is.

If the determination in step S7 is affirmative, the determination in step S8 is further made. That is, it is determined whether αD _n > D _n−1 , D _{n + 1} and ρ _n > ρ _n−1 , ρ _{n + 1} are satisfied. Here, α is a weighting factor and is a value larger than 1. If the determination in step S7 is affirmative, it is determined that it is a shot boundary, and if it is negative, the determination in step S9 is performed.

When the shot boundary exists in a scene having a large motion, the difference between frames becomes very large, and the shot boundary cannot be determined by the equation of step S8. Therefore, the determination in step S9 is performed. That is, it is determined whether or not ρ _n > Th_ac is satisfied. Here, Th_ac is a threshold for determining the peak value in the [rho _n. If the determination in step S9 is affirmative, it is determined as a shot boundary, and if negative, the process proceeds to step S10.

In two consecutive shots, if the camera angles are simply different, the color difference histograms are similar. For this reason, it is difficult to detect a shot boundary in the determinations in steps S8 and S9. However, since the pixel difference is very large at these shot boundaries, peak detection of the luminance difference value is effective. Therefore, when the determination in step S10, that is, βD _n > D _n−1 , D _{n + 1} or D _n −Th_ad> D _n−1 , D _{n + 1} is satisfied, it is determined that the shot boundary. Here, β and Th_ad are a weight factor and a threshold for detecting the peak value of D _n , respectively.

  When the inventor does not use the process of the present invention, that is, the process of steps S2 and S6 (conventional method), the process of step S6 is used for the “TRECVID2005” test data. When the shot boundary is detected in the case where both the steps S6 and S2 are used, the detection rate is as shown in FIG. From this, it was confirmed that when the processing of the present invention is introduced, the detection accuracy of the shot boundary is improved.

  In the embodiment described above, the central area extraction unit 2 is an example in which the screen is divided into m in the horizontal direction and the central area is extracted. However, the screen is divided into m in the vertical direction and the area including the left and right side pieces is extracted. The central region may be extracted by removal. Alternatively, only the image portion of the central area may be extracted by removing peripheral areas in both the horizontal and vertical directions of the screen.

It is a block diagram which shows the schematic structure of one Embodiment of this invention. It is operation | movement explanatory drawing of the center area | region extraction part of FIG. It is a flowchart explaining operation | movement of this embodiment. This embodiment is an example of shot boundary detection rate. It is a flowchart which shows the outline | summary of a moving image cut point detection process.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Variable length decoding part, 2 ... Center area extraction part, 3 ... Inter-frame brightness | luminance difference value calculation part, 4 ... Color difference histogram correlation value calculation part, 5 ... Darkness determination part, 6: Shot boundary determination unit.

Claims (3)

Means for partially decoding the encoded image;
Means for removing a peripheral region where an invariant part of an image such as a caption enters from the partially decoded image;
A shot boundary detection apparatus comprising: means for detecting a shot boundary using an image of the area that has not been removed. Means for partially decoding the encoded image;
Means for dividing the partially decoded image into m (m ≧ 3) in the horizontal or vertical direction and removing at least one of the upper and lower or left and right peripheral divided regions;
A shot boundary detection apparatus comprising: means for detecting a shot boundary using an image of the area that has not been removed. In the shot boundary detection device according to claim 1 or 2,
The means for detecting the shot boundary is a means for obtaining an inter-frame luminance difference value D _n and an average luminance value of the image, and when the average luminance value is smaller than a predetermined threshold, the inter-frame luminance difference A shot boundary detection device comprising: means for multiplying the value D _n by γ (γ>1); and means for setting a shot boundary candidate if γD _n is larger than a predetermined threshold Th.

Images