JP3390308B2 - Special cut point detector for moving images - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image cut point detecting device, and more particularly, to high-speed switching of moving images consisting of unencoded original moving image information or encoded moving image information. The present invention also relates to a moving image cut point image detection device that can detect with high accuracy.

[0002]

2. Description of the Related Art Indexing of moving images (indexing)
Is one of the important technologies for efficient searching and editing in electronic libraries and video databases. The cut point indicates a boundary point of a scene, which is a basic unit of an image, and as a basic indexing tool, research on its detection method has been actively conducted in recent years.

In particular, recently, a detection method has been reported for a special cut screen such as a dissolve screen in which a scene changes using an editing effect, as well as a normal cut point where the scene changes instantaneously.

The dissolve image is basically an image of a process in which two different scenes are spatially combined and transition from one scene to another scene. For example, "images" by Murata, Nakamura, Ota et al. Automatic Change Detection of Cuts ", IPSJ 50th National Convention, 6D-
7 (1995), a method of detecting a dissolve by modeling a change in luminance with respect to the dissolve has been proposed. As another conventional technique, “dissolve detection focusing on monotonicity of luminance change” by Nagasaka, Miyatake, Taniguchi et al., IEICE Spring Conference, D-615 (199).
As in 6), there is also proposed a method of performing dissolve detection by paying attention to the scene composition process in dissolve.

[0005]

However, in the former case, the maximum point is determined to be the dissolve screen because the temporal change in the ratio of the pixels in which the ratio of the difference between the luminance components between frames is positive to the entire screen is obtained. Even if a scene moves greatly from a dark place to a bright place, it may be erroneously detected as a dissolve screen.

Further, in the latter case, paying attention to the monotonous increase / decrease of the luminance change of the pixel in the dissolve section, for each pixel in the frame, the difference value from the immediately preceding frame is obtained, and if the variance is less than a certain value, the dissolve Although it is determined to be a screen, the dispersion may be small even if the subject has a slight movement, for example, and may be erroneously detected as a dissolve screen.

Therefore, in the present invention, in-screen activities having different luminance change characteristics are used for a moving image and a dissolve image, and compared with before and after the dissolve image.
Using the characteristic that the in-screen activity is reduced in the scene composition part, strong detection against movement is made possible, and in addition to normal cut points, it is possible to detect cut points using special effects such as dissolve, and overall detection It is an object of the present invention to provide a high-speed and high-precision cut point detection device with improved efficiency.

[0008]

In order to achieve the above-mentioned object, the present invention provides an in-screen activity calculating means for obtaining the activity of the luminance component of each frame from the input moving image screen data, and the in-screen activity. From the in-screen activity of each frame extracted by the calculating means, a moving average calculating means for obtaining a moving average of the in-screen activity of each frame, and a moving average from the moving average data of each frame extracted by the moving average calculating means Moving average difference calculation means for obtaining the inter-frame difference value, and the screen of interest and the histogram of the color difference signals of a plurality of frames previous in time from the screen of interest are respectively obtained from the screen data, and the inter-frame color difference signal histogram correlation value is obtained. Color difference histogram correlation calculation means, the moving average difference data, and the color It is characterized in that and a special cut detection processing means for performing special cut detection process using a histogram correlation.

According to the present invention, since the special cut point detection processing is performed using the inter-frame difference value of the moving average of the in-screen activity and the color difference histogram correlation, a special effect such as dissolve is used. The cut point can be detected accurately.

[0010]

BEST MODE FOR CARRYING OUT THE INVENTION The present invention will be described in detail below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of an embodiment of the cut point detection device of the present invention. This embodiment is based on MPEG1 which is an international standard method for moving picture coding.
The special cut point frame is detected from the moving image coded data compressed by (ISO / IEC 11172), but the present invention is not limited to this.

As shown in FIG. 1, compression-coded moving picture coded data a is input to a variable length decoding processing section 1. The variable length decoding processing unit 1 includes a variable length decoding unit 3
1 and an average value component extraction unit 32, and the variable length decoding unit 31 decodes the quantized two-dimensional DCT coefficient of each block from the encoded data a of the moving image, and the two-dimensional DC
The T coefficient b is input to the average value component extraction unit 32.

The average value component extracting method of the average value component extracting unit 32 is, for example, "Detection of cut point from compressed moving image data by simple decoding process" by Ushihara, Nakajima et al., IPSJ 51st National Convention. , 6S-9 (1995),
The method disclosed in Japanese Patent Application No. 7-263681 can be used.

Briefly described, the average value component extraction unit 3
2 shows, for example, as shown in FIG. 2A, from the quantized two-dimensional DCT coefficient of each block of 8 pixels × 8 lines of the frame t, the average value shown in FIG. The (0, 0) component representing the component data is extracted, and the average value component data YDC [i, k] of the luminance signal is sent to the in-screen activity calculation unit 2 and the average value component data Cb [i, k] of the color difference signal Cr [i,
k] are input to the color difference histogram calculation unit 3, respectively.

On the other hand, when the input moving image is not compression-encoded moving image data, the data of each frame is directly input to the average value component extraction unit 32. in this case,
The average value component extraction unit 32 synthesizes each pixel in the small area of each frame to calculate average value component data.

In the in-screen activity calculation section 2,
Using the average value component data YDC [i, k] of the input luminance signal, the in-screen activity (in-screen deviation) Y-VAR ² [t] of the luminance signal of the frame t is calculated according to the following equation, The on-screen activity is input to the moving average calculator 4 and the first memory 5. ^{Y-VAR 2 [t] =} <YDC 2 [i, k]> - (. No) <YDC [i, k]> 2 ... (1) Here, t is a frame number, YDC [i, k] is 8x
The DC component of 8 blocks [i, k], <> indicates the average value within the frame t.

The in-screen activity Y-VAR ² [t] of the luminance component in the dissolve section thus obtained
The shape of is a U-curve that is convex downward, but the shape of the activity may be convex downward due to local fluctuation due to movement even in a place that is not the dissolve section. Therefore, the moving average calculation unit 4 performs the following processing.

In the moving average calculating section 4, for example, as shown in FIG. 3, the in-screen activity of the frame of interest t input from the in-screen activity calculating section 2 and the first memory 5 are stored. The on-screen activity from the frame t to the frame Fv-1 frame before is integrated, and the moving average of Fv frames is calculated according to the following formula.

[0018]

[Equation 1] By performing this processing, it becomes possible to perform filtering that absorbs movements and local fluctuations, and it is possible to prevent erroneous detection of an image that is not a cut point.

However, the moving average shape of the in-screen activity of the luminance component becomes a flat U curve when the dissolve section length is long, and a sharp U curve when the dissolve section length is short.
That is, the shape of the moving average of the in-screen activity of the luminance component changes between the flat U curve and the abrupt U curve depending on the dissolve section length. Therefore, for example, simple peak detection such as minimum value detection may not detect a relatively flat U curve.

In this embodiment, the moving average data is input to the moving average difference calculator 6 and the second memory 7, and the moving average data of one frame before stored in the second memory 7 is also moved. The moving average difference DMV [t] is calculated according to the following equation by transferring to the average difference calculating unit 6, the calculation result is input to the special cut point detection processing unit 8, and the special cut point image is calculated from the moving average difference value. Is determined.

DMV [t] = MV [t] -MV [t-1] (3) Another feature of the dissolve section is that, even if the correlation of the color difference signals between consecutive frames is small, the frame intervals are set to some extent. The correlation between frames taken tends to be large.

Therefore, the color difference histogram calculation unit 3 calculates the color difference histogram of the frame t from the average value component data of the input color difference signals, and inputs the calculation result to the color difference correlation calculation unit 9 and the third memory 10. To do. In the color difference correlation calculation unit 9, the color difference histogram correlation of the input frame t and the color difference histogram of the frame (t−α) (where α is 1 or more) stored in advance in the third memory 10 are used. ρ _{t, t-} α Is calculated and input to the special cut point detection processing unit 8.

The color difference histogram and the color difference correlation may be calculated, for example, by Nakajima et al., "Detection of cut from compressed moving image data by inter-frame luminance difference and color difference correlation", IEICE Autumn Meeting D-501 ( 19
94), Japanese Patent Application No. 5-216895 or Japanese Patent Application 6
The method disclosed in No. 46561 can be used.

The special cut point detection processing unit 8 detects a special cut point using the moving average difference and the color difference histogram correlation of the input luminance signal. The output signal c from the special cut point detection processing unit 8 is sent to the first determination unit 11.

FIG. 4 is a graph showing the change over time of the moving average difference DMV [t] of the luminance signal. The dissolve section can be detected as a portion that changes from a downwardly convex peak to an upwardly convex peak as shown on the left side a in the figure. However, as shown in the right side b of the figure, either the positive peak or the negative peak may occur depending on the fade section, the dissolve while moving, or the shape of the screen. Therefore, the absolute value of DMV is expressed by the following equation. When a certain threshold β is exceeded, a frame when the threshold β is exceeded is determined to be a dissolve candidate screen.

│DMV [t] │> β Next, the color difference histogram correlation ρ _{t, t-} α for the frame which is the dissolve candidate screen by the moving average difference.
Is used to perform the final dissolve screen detection. Color difference histogram correlation ρ _{t, t-} α When the change over time is shown in a graph as shown in FIG. 5, for example, the final dissolve screen is determined according to the following equation.

Ρ _{t, t-} α <Γ a and b in FIG. 5 correspond to a and b in FIG. In addition,
The dissolve section is empirically several frames (for example, 50 to
Since it is known that it will continue for about 70 frames), when the dissolve screen is detected, the above several frames are skipped and the search for the next dissolve screen is restarted. As a result,
It is possible to prevent a plurality of dissolve screens from being detected from one dissolve section.

The first judgment section 11 sends the frame d of the special cut points detected by the special cut point detection processing section 8 to the cut point image holding section 12. On the other hand, the first determination unit 11
For the frame e that has not been determined to be a special cut point by the special cut point detection processing unit 8, the normal cut point detection processing unit 13 further determines whether the frame e is a normal cut point. Send to.

The normal cut point detection processing unit 13 performs normal cut point detection, and the frame f determined to be a normal cut point is input to the cut point image holding unit 12 in the same manner as the special cut point. It On the other hand, the frame g determined to be the non-cut screen is discarded. As a method for detecting the normal cut point, for example, "cut point detection from compressed moving image data by simple decoding process" by Ushihara, Nakajima et al. Mentioned above, IPSJ 51st National Convention, 6S-9.
(1995), the method disclosed in Japanese Patent Application No. 7-263681 can be used.

The cut point image h of the cut point image holding unit 12
By transferring the data to the cut point image display unit 14, the cut point images can be displayed in a list as shown in FIG. 6, for example.

Next, the operation of the controller 15 of FIG. 1 will be described with reference to the flowcharts of FIGS. 7 and 8. In step S1, k and i representing the block number of the frame are set to 0.
And put. In step S2, the encoded data a of the block [i, k] of the moving image input to the variable length decoding processing unit 1 is variable length decoded. In step S3, the average value component extraction unit 32 extracts average value component data of the luminance signal and the color difference signal. In step S4, it is determined whether the block [i, k] is the final block in the frame t. When this determination is negative, the process proceeds to step S5, 1 is added to k or i, and the process proceeds to the next block. Then, the processes of steps S2 and S3 are performed again. The above processing is repeated, and when the determination in step S4 becomes affirmative, the process proceeds to step S6, and the average value component data of the luminance and color difference signals is transferred to the in-screen activity calculation section 2 and the color difference histogram calculation section 3.

In step S7, the in-screen activity Y-VAR ² [t] and the color difference histogram data Cb [i, k], Cr [i, k] obtained by the in-screen activity calculating section 2 and the color difference histogram calculating section 3 are obtained. ] To the moving average calculator 4, the first memory 5, the color difference correlation calculator 9, and the third memory 10, respectively. In step S8,
The moving average data obtained by the moving average calculator 4 is transferred to the moving average difference calculator 6. In step S9, the moving average difference and the color difference correlation data obtained by the moving average difference calculating unit 6 and the color difference correlation calculating unit 9 are transferred to the special cut point detection processing unit 8. In step S10, it is determined whether the frame t is a special cut point, that is, a dissolve screen. When this determination is affirmative, the process proceeds to step S13 in FIG. On the other hand, if not,
It proceeds to step S11.

In step S13, the screen data of the frame t is transferred to the cut point image holding section 12. Then
In step S14, it is determined whether the frame t is the final image. When this determination is negative, the process proceeds to step S17 in FIG. 7 and the frame t is updated by, for example, 60 frames. This 60 frames is for one dissolve section to be about 60 frames, and is for preventing two or more dissolve screens from being detected from one dissolve section. Then, it returns to step S2 and repeats the above-mentioned processing.

On the other hand, when the process proceeds to step S11, the normal cut point detection process is performed for the frame t in step S11. In step S12, it is determined whether or not the frame t is the normal cut point, and when this determination is affirmative, the process proceeds to step S15 in FIG. In step S15, the screen data of frame t is transferred to the cut point image holding unit 12. Then, step S16
Then, it is judged whether the frame t is the final image. When this determination is negative, the process proceeds to step S18 of FIG. 7 and frame t is updated by one frame. Then, it returns to step S2 and repeats the above-mentioned processing. When the determination in step S12 is negative, the process proceeds to step S16.

If the determination at step S14 or S16 is affirmative, the process proceeds to step S19 to determine whether or not the cut image is displayed on the cut point image display unit 14. If this determination is positive, step S20
To the cut point image display unit 1
Transfer to 4. Then, in step S21, it is determined whether or not the process is to be ended, and if this determination is affirmative,
The series of processes described above is completed.

[0036]

As described above, according to the present invention,
Since the special cut point detection is performed using the average value component of the luminance signal and the color difference signal that can be extracted only by variable length decoding, in addition to the normal cut point detection, the cut using the special effect such as dissolve is performed. It becomes possible to detect points while suppressing an increase in the processing amount to a very small amount. It is also possible to improve the overall cut point detection rate.

When the present invention is actually operated,
The following results were obtained. In other words, when cut detection is performed on a bitstream encoded by the MPEG1 system standardized by ISO, with about one hour of material including news, cooking programs, variety programs, advertising programs (CM programs), etc. Comparing the results of detecting the ratio (detection rate) of the detected cut points with respect to the cut points of the method of Ushihara et al. And the method of the present invention, the former detection rate is 92.8%. On the other hand, the latter detection rate is 9
It became 4.5%. This is because the detection rate is improved because it has become possible to detect the cut point using a special effect such as dissolve, in the detection device according to the present invention, compared with the conventional device, comprehensively, It was possible to detect the cut point screen with high accuracy.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a moving image special cut screen detection device according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram of an example of an operation of an average value component extraction unit in FIG.

FIG. 3 is an explanatory diagram of an example of an operation of a moving average calculation unit of FIG.

FIG. 4 is an explanatory diagram showing a moving average difference of luminance signals.

FIG. 5 is an explanatory diagram showing color difference histogram correlation.

FIG. 6 is an explanatory diagram showing a display example of the cut image display unit in FIG. 1.

FIG. 7 is a flowchart for explaining the operation of the control device of FIG.

8 is a flowchart continued from FIG. 7. FIG.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Variable length decoding process part, 2 ... In-screen activity calculation part, 3 ... Color difference histogram calculation part, 4 ... Moving average calculation part, 5 ... First memory, 6 ... Moving average difference calculation part, 7 ...
Second memory, 8 ... Special cut point detection processing unit, 9 ... Color difference calculation unit, 10 ... Third memory, 11 ... First determination unit, 1
2 ... Cut point image holding unit, 13 ... Normal cut point detection processing unit, 14 ... Cut point image display unit, 15 ... Control unit, 31 ...
Variable length decoding unit, 32 ... Average value component extraction unit.

─────────────────────────────────────────────────── ─── Continuation of the front page (56) Reference JP-A-7-236153 (JP, A) JP-A-6-22298 (JP, A) JP-A-6-153155 (JP, A) (58) Field (Int.Cl. ⁷ , DB name) H04N 11/00-11/22 H04N 9/00-9/78

Claims (7)

(57) [Claims] 1. A special cut point detecting device for a moving image, wherein an in-screen activity calculating means for obtaining an activity of a luminance component of each frame from the input moving image screen data, and an in-screen activity calculating means for extracting the activity. The moving average inter-frame difference value is calculated from the moving average calculation means for obtaining the moving average of the in-screen activity of each frame from the in-screen activity of each frame and the moving average data of each frame extracted by the moving average calculation means. A moving average difference calculating means for obtaining, and a color difference histogram correlation calculating means for obtaining a screen of interest from the screen data and a histogram of color difference signals of a plurality of frames previous in time from the screen of interest, and obtaining a color difference signal histogram correlation value between frames. , The moving average difference data and the color difference histogram Special cut detection device of the moving image, characterized by comprising a special cut detection processing means for performing special cut detection process using uncorrelated. 2. The moving image special cut point detection device according to claim 1, wherein the in-screen activity calculating means extracts the in-screen activity from average value component data in a small region of each frame of the input moving image. A special cut point detection device for moving images, characterized by: 3. The moving image special cut point detection device according to claim 1, wherein the in-screen activity calculating means decodes screen data when the input moving image is a compressed moving image. A special cut point detection device for a moving image, wherein activity in a screen is extracted from average value component data of a small area obtained by using a simple decoding process for extracting a part of the decoded data. 4. The moving image special cut point detection device according to claim 1, wherein the in-screen activity calculation means includes luminance component average value component data among the extracted small region average value component data. An apparatus for detecting special cut points in moving images, characterized in that the deviation within the screen is obtained by using. 5. The moving image special cut point detection device according to claim 1, wherein the color difference histogram correlation calculation means uses the average value component data of the color difference components among the average value component data of the small regions of the frame to perform the color difference. A special cut point detection device for moving images, characterized by obtaining a histogram. 6. The moving image special cut point detection device according to claim 1, wherein the special cut point detection processing means performs special cut by a temporal change of the moving average difference data and a temporal change of the color difference histogram correlation. A special cut point detecting device for a moving image, characterized by judging a screen. 7. The moving image special cut point detecting device according to claim 6, wherein a temporal change in the moving average difference data and a temporal change in the color difference histogram correlation are provided for respective threshold values. A special cut point detection device for a moving image, which is characterized in that it is judged as a special cut screen when it exceeds the limit.