JP2001169287A - Device and method for detecting scene change of compressed moving image and recording medium recording program therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique capable of detecting a scene change even when frame/field structures coexist. SOLUTION: This device is provided with an image structure decision part 1 for deciding the image structure of input compressed moving images, a feature amount extraction part 2 for extracting feature vector, based on data for up- and-down double amount in the image vertical direction for a field structure image if the decision result of the image structure decision part is a frame structure image, a memory 6 for the data for recording the data extracted in the feature vector extraction part, an extracted data comparison part 3 for comparing extracted data and obtaining the change amount of video images, and a scene change decision part 4 for deciding the scene change by using the change amount obtained in the extracted data comparison part.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

TECHNICAL FIELD The present invention relates to a method for converting a compressed video
The present invention relates to a scene change detection device for detecting a scene change and related technology.

[0002]

2. Description of the Related Art In recent years, the spread of digital video cameras,
With the advent of digital broadcasting, MPEG, DV, etc.
Opportunities for handling compressed moving images are increasing. In addition, a large amount of past analog video may be stored as a digital compressed moving image. A technique of editing such a compressed moving image without decoding the encoded moving image without decoding is being put to practical use.

[0003] In such editing, compressed moving images,
That is, a technique for automatically detecting a scene change (the position of a transition between a video and a scene) from a bit stream at a high speed is definitely required. This is because the position information of the head of the detected scene or the representative image of the scene separated by the scene change technique is useful as an index of the video content, and is an important aid for searching and editing the content.

Next, MPEG2 encoding, which is widely used as a format of a compressed moving image, will be described. MPEG2 encoding is based on MPEG1, H.264. Motion vector and DCT (discrete cosine transform: Dis
This is a method using “Crete Cosine Transform”. The data is divided into luminance (Y),
It is separated into color differences (Cb, Cr) and is coded in macroblock units of 16 * 16 pixels.

[0005] When coding each macroblock, motion compensation prediction for performing motion prediction from a reference image is selected, or intra coding for performing coding using only data to be coded is selected. You.

[0006] Among them, the motion compensation prediction is a method in which the coding rate is increased when the temporal correlation between the frames is high, and the macroblock to be coded and the macroblock obtained by motion prediction from a reference image. And compressing information in a time-space manner by obtaining a prediction error signal from a difference between the data and the data. In the motion compensation prediction, this prediction error signal is converted into a spatial frequency domain by DCT in units of 8 * 8 pixels.

On the other hand, the intra coding is a method in which data of a block to be coded is divided into blocks of 8 * 8 pixels, and DCT coding is simply performed in units of these blocks.

The encoding unit will be described below. M
In PEG2, an interlaced image is also targeted, and there are a frame structure and a field structure as units of screen coding.

In the frame structure, encoding is performed on a frame in which two fields of an odd field and an even field are arranged in an interlaced manner. On the other hand, in the field structure, encoding is performed on one field of an odd field or an even field.

[0010] In the present specification, an image encoded with a frame structure is called a "frame structure image", and an image encoded with a field structure is called a "field structure image".

Next, the motion compensation will be described. MPE
G2 has a frame structure and a field structure as described above. The motion compensation prediction of the frame structure image includes frame prediction, field prediction, and dual-prime prediction. The motion compensation prediction of the field structure image includes a field prediction, a 16 * 8 MC prediction, and a dual prime prediction. Further, in prediction other than frame prediction, it is possible to select whether a field to be referred to is an odd field or an even field.

Next, an encoding method will be described with reference to FIG. In a frame structure image, two types of DCT, a frame DCT and a field DCT, can be used as an encoding method.

Of these, the frame DCT decomposes the luminance signal of a macroblock into four blocks as shown in FIG. 15A so that each block is composed of frames. DCT is applied to this.

On the other hand, the field DCT is shown in FIG.
As shown in (1), when a luminance signal of a macroblock is decomposed into four blocks, each block is decomposed so that each block is composed of fields, and DCT is applied to this.

Now, at the time of encoding, these two types of DC
Either T may be used, but it is generally known that the coding efficiency is improved by using the field DCT when the difference between the image data of the odd field and the image data of the even field is large. In particular, when two scenes are mixed in one field, the field DCT increases the compression ratio.

However, in the case of field DCT, it is necessary to decompose the frame structure into two fields.
Processing speed is slower than frame DCT. That is,
Coding efficiency of a frame structure image (interlaced image) can be improved by appropriately using two types of DCT in accordance with such a property. In addition,
Regarding the color difference signal in the 4: 2: 0 format,
Frame DCT is always used. In a field structure image, since a macroblock is composed of only one field signal, field DCT is always performed.

Based on the above description, a conventional scene change technique will be described below. Conventionally, scene change detection uses feature amounts such as (1) a color histogram of an image, (2) a data size of a compressed moving image, and (3) a block data difference at the same position of an image of two frames. . (1) To use a color histogram of an image, a color used in an image of one frame is made into a histogram in one frame or in a region obtained by dividing one frame, and this histogram is used as a feature amount of the frame. The degree of similarity is obtained by comparing with the feature amounts of the frame images before and after that (for example, see Japanese Patent Application Laid-Open No. 7-59108). (2) In order to use the data size of the compressed moving image, in the scene change portion, the data size of adjacent frames is compared by using the property that the compression ratio is poor, and the difference is larger than a predetermined threshold value. It is determined that a scene change has occurred (see, for example, JP-A-7-121555).

However, according to the methods (1) and (2), a scene change can be detected only on a frame basis, so that a scene change is caused between an odd field and an even field (between two fields) in one frame. If it exists, this scene change cannot be detected accurately.

In order to solve this problem, Japanese Patent Application Laid-Open No. Hei 9-3
Japanese Patent No. 22120 proposes a method for detecting a scene change from video data encoded using a field prediction method without performing decoding processing. In this method, in a frame in which prediction is performed, a plurality of similarities between fields are calculated based on a reference field selection signal indicating whether to select an odd field or an even field of a reference frame to perform prediction. The scene change is detected using the result.

[0020]

However, since this method relies on the field prediction method, the video without the field prediction method (inter-frame prediction method), the video of the field prediction method and other prediction methods are used. This method cannot be applied to video in which video of different formats is mixed.

(3) DC data at the same position as data
If the difference of only the DC component at T is used, there is a case where the correspondence of the position cannot be obtained. This is because, in a frame structure image, the frame DCT and the field D
Because two types of DCT called CT can be used,
When the block data is compared without solving the DCT, when one of the data to be compared is encoded by the frame DCT and the other data is encoded by the field DCT, data of 8 * 8 pixels in the image is obtained. And 8 * of only odd or even fields in 8 * 16 pixel data
This is because data of eight pixels is compared.

To improve this, the frame structure image 1
One frame has to be compared with one frame of the field structure image (odd field image and even field image). However, in this case, the comparison can be performed only when the data of the two field structure images are prepared, and there is a problem that the processing is complicated and the processing speed is reduced.

Accordingly, it is a first object of the present invention to provide a technique capable of detecting a scene change even when field / frame structures are mixed.

It is a second object of the present invention to provide a technique capable of detecting a scene change existing between fields having a frame structure.

It is a third object of the present invention to provide a technique capable of quickly detecting a target scene change when an interval between a start point and an end point is known in advance.

[0026]

According to the present invention, a compressed moving image in which a field structure image and a frame structure image are mixed is input, and a scene change in the input compressed moving image is detected. According to the first aspect of the present invention, when the compressed moving image is a frame structure image, the compressed moving image is doubled in the vertical direction corresponding to the field structure image in the first, second, sixth, eighth, twelfth, and thirteenth embodiments. Extract the data of

[0027] The present invention provides, for the second object, claim 3,
9 and 14, a field DCT encoded block number counting unit is provided.

The present invention provides, for a third object, claim 4,
In steps 10 and 15, a set of scene changes corresponding to a designated interval is detected from a plurality of detected scene changes, and the scene changes are output as a result.

[0029]

BEST MODE FOR CARRYING OUT THE INVENTION The scene change detection in the present invention does not depend on a prediction method. Therefore, a scene change can be detected regardless of the prediction method or regardless of the presence or absence of the prediction method. Hereinafter, in all the embodiments of the present invention, the input compressed moving image
A description will be given assuming that G2 is used. However, the same effect can be obtained if the compression method uses DCT and can mix the respective structures of fields / frames. Included in the subject of the present invention. In the present invention, a compressed moving image in which a field structure image and a frame structure image are mixed is input, and a scene change in the input compressed moving image is detected.

(First Embodiment) An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of a scene change detection device according to the first embodiment of the present invention.

As shown in FIG. 1, a bit stream encoded according to MPEG2 is input to the scene change detection device as an input compressed moving image. Also,
From the scene change detection device, information indicating the position of the scene change detected in the bit stream is output as a detection result. Here, as described in the section of the prior art, various coding schemes, coding units (frame / field structure), DCT, and the like of the input compressed moving image can be considered. But not
A plurality of types may be mixed on the time axis.

In particular, a case where the field structure image and the frame structure image alternately exist on the time axis may be used. The above points also apply to other embodiments described later.

As shown in FIG. 1, an input compressed moving image is first input to an image structure determining unit 1. Image structure determination unit 1
Determines whether the currently input image is a field-structured image or a frame-structured image by referring to information on a specific area of the bit stream. Then, the result of this determination and the contents of the bit stream are output to the feature extraction unit 2 at the next stage.

Next, the characteristic amount, change amount and characteristic amount extracting unit 2
Will be described with reference to FIGS. 5 to 6 as an example. In FIG. 5, the left side of the figure shows an old image on the time axis, the right side shows a new image, and the t-th image is described as an image t. In this example, the images t-2 to t + 1, t + 4 to
t + 5 is a field structure image, and images t + 2 to t +
3 is a frame structure image. That is, images t + 1 to t +
At 2, t + 3 to t + 4, there is a structural change.

In a field structure image as shown in FIGS. 6A (top field) and FIG. 6B (bottom field), block data D for one block is used.
Using t, the block data Dt is used as it is as the feature amount dt (dt = Dt).

In a frame structure image as shown in FIG. 6C, two blocks of upper and lower blocks (block data Dtu and block data Dtb) are used, and the average value is used as a feature value dt (dt). = (Dtu +
Dtb) / 2).

Here, the block data Dt, Dtu, D
Although tb must be the same type of data, various data representing the average luminance value in the block and other images in the block can be used.

As shown in FIGS. 6A, 6B and 6C, the frame structure image is twice as high as the field structure image. And the frame structure image, in the field structure image, the block data Dt of the block coordinates (x, y), the block data Dtu of the block coordinates (x, 2 * y) and the block coordinates (x, 2 *) The pair with the block data Dtb of (y + 1) is used for comparison.

Next, the feature amounts of the block coordinates (x, y) in the image t and the image s (typically, s = t + 1) at different positions on the time axis are respectively represented by dt (x, y).
y) and ds (x, y), the first change amount R (t, s) between the image t and the image s is defined as follows. This first change amount R (t, s) is obtained by calculating the image t and the image s
And close to a simple difference.

[0040]

(Equation 1)

Similarly, the second variation Q (t, s)
Is defined as follows. The second variation Q (t, s) is
It is close to the differential value between image t and image s.

[0042]

(Equation 2)

Here, the first change amount R (t, s) and a first threshold value which is set empirically in advance are represented by
By comparing, a position where a large change appears in the image can be extracted, and this position can be detected as a scene change. However, depending on the image, although the scene is the same, such as a scene with rapid movement, a continuous change occurs, and a scene change may be detected excessively with only the first change amount R (t, s). . In such a case, the comparison between the second change amount Q (t, s) and a second threshold value set in advance is used in combination, so that
In addition, only a position where there is a large change can be detected as a scene change, and the reliability of the detected scene change can be improved.

It should be noted that the first variation R
In some cases, only one of (t, s) and the second change amount Q (t, s) is sufficient, and the first change amount R (t, s) and the second change amount Q (t, s). An appropriate weighting may be applied to s) to consider both. Of course, the above two expressions are merely examples, and other expressions may be used as long as a scene change can be determined, as well as equivalent expressions.

The first variation R (t, s) and the second
In order to obtain the change amount Q (t, s), as shown in FIG. 3, after obtaining all the extracted data for one input image, it is compared with the comparison target image to calculate the change amount. However, as shown in FIG. 4, one block (or a plurality of blocks)
It is preferable that when the data is extracted, it is compared with the data corresponding to the same position of the comparison target image at the same time. This is because the processing speed increases as shown in FIG.

In this way, a scene change can be uniformly detected on the time axis from a series of videos in which the field structure image and the frame structure image are mixed.

Now, as shown in FIG.
Are stored in the first data memory 6 in association with the extracted feature amount dt and the image information such as the frame number from which the feature amount dt is extracted. This first data memory 6
This is equivalent to a storage area, and may be replaced by another storage means such as a hard disk device.

The extracted data comparing unit 3 at the next stage of the characteristic amount extracting unit 2 extracts the characteristic amount dt from the characteristic amount extracting unit 2 and stores the new characteristic amount dt in the first data memory 6.
Is recorded, the new feature value dt and the feature values recorded before (typically immediately before) in the first data memory 6 are referred to, and the change amount between these feature values is referred to. Ask for. This change amount may be only one of the first change amount R (t, s) and the second change amount Q (t, s), but preferably both are obtained.

Then, the extracted data comparing section 3 records the obtained change amount as comparison result information in the second data memory 7 and, at the same time, compares the image information such as the frame number where the change amount occurs with the comparison result. The information is recorded in the second data memory 7 in association with the information.

The scene change determination data input unit 5 holds a threshold value used for scene change determination.
This threshold value is output to the scene change determination unit 4. The threshold value includes a first threshold value for the first change amount R (t, s) and a second threshold value for the second change amount Q (t, s).

When the extracted data comparing section 3 makes a comparison, the scene change determining section 4 refers to the comparison result information recorded in the second data memory 7 and receives the information from the scene change determining data input section 5. If the threshold is exceeded,
It is determined that a scene change has occurred at this position, and image information such as a frame number (that is, a position in the bit stream) related to this comparison result information is output as a detection result. Otherwise, the scene change determination unit 4
May output no detection result, or may output a detection result indicating that no detection was performed.

FIG. 2 is a flowchart of the scene change detecting device according to the first embodiment of the present invention. Next, the operation of the scene change detection device according to the present embodiment will be described with reference to FIG. First, when the input compressed moving image reaches the image structure determining unit 1, the determining unit 1 determines whether the current image is a frame structure image or a field structure image (step 1). If it is a frame structure image,
The feature amount extraction unit 2 extracts a value using data of two blocks in the vertical direction of the image as a feature amount and records the value in the first data memory 6 (step 2). If it is a field structure image, the feature amount extraction unit 2 extracts a value using data of one block as a feature amount and records it in the first data memory 6 (step 3). That is, the frame structure image uses twice the data in the vertical direction as the field structure image.

Then, in step 4, the extracted data comparison unit 3 compares the current feature value with the previous feature value and records the comparison result information in the second data memory 7. Next, in step 5, the scene change determination unit 4
Compares the comparison result information with a threshold value input from the scene change determination data input unit 5, and if it is determined that the scene change has occurred, outputs the position where the scene change has occurred as a detection result.

In the present embodiment, the block data Dt for one block is used for the field structure image as the feature amount dt, and the average value of the block data Dtu and Dtb for two blocks in the vertical direction is used for the frame structure image. Although used, other methods may be used as long as the comparison levels of the data of the field structure image and the data of the frame structure image become equal.
For example, a feature is that the block data of one block of the field structure image is doubled (dt = 2 * Dt) and the sum of the block data of the upper and lower two blocks in the vertical direction of the frame structure image (dt = Dtu + Dtb). As the amount dt,
These may be compared.

In this embodiment, the data extracted from the field structure image is set in units of one block,
Although the data extracted from the frame structure image is in units of two blocks, the data extracted from the frame structure image is twice as long as the data extracted at the position corresponding to the data extracted from the field structure image. If so, a scene change can be similarly detected. That is,
When the data of the N blocks of the field structure image is extracted as the feature amount, the data from the 2 * N block, which is twice as long as the corresponding position in the vertical direction, may be extracted as the feature amount from the frame structure image. .

Thus, a scene change can be uniformly detected from a compressed moving image in which a frame structure image and a field structure image are mixed. In addition, whether a frame DCT or a field DCT is used in a frame structure image, a scene change can be detected without special consideration. In addition, since this method does not depend on the prediction method, it does not matter what the prediction method is, and a desired scene change can be detected regardless of the presence or absence of the prediction method.

(Second Embodiment) FIG. 7 is a block diagram of a scene change detecting device according to a second embodiment of the present invention, and FIG. 8 is a scene change detecting device according to the second embodiment of the present invention. It is a flowchart of an apparatus.

In FIG. 7, the positions of the feature quantity extraction unit 2 and the image structure determination unit 1 are replaced with those of FIG. 1 according to the first embodiment. Further, the feature amount extraction unit 2 of the present embodiment extracts the feature amount from the block data of each block for the entire screen, whether the input compressed moving image is a frame structure image or a field structure image. (Step 10).

Then, as shown in FIG. 8, the image structure judging unit 1 at the preceding stage of the extracted data comparing unit 3 examines the structure of the image inputted this time, and in the case of a frame structure image, the extracted data comparing unit 3 Then, the data is compared using the data of the upper and lower blocks in the vertical direction of the image (step 12). On the other hand, if it is a field structure image, the extracted data comparison unit 3
Compares data using data for one block (step 13). The comparison here is the same as in the first embodiment.

Now, in the example of FIG. 5, the image t and the image t
When comparing with +1, since both images are field structure images, the data of the blocks at the same position on the screen are compared.

For comparison between the image t + 1 and the image t + 2,
Since the image t + 2 is a frame structure image, the image t + 1
The data obtained from the block coordinates (x, y) and the data of the image t + 2 are represented by the block coordinates (x, 2 *).
y) and data obtained from the block coordinates (x, 2 * y + 1) are compared.

For example, the data obtained from the block coordinates (x, y) in the image t + 1 is A, the block coordinates (x, 2 * y) and the block coordinates (x, 2 * y + 1)
Are respectively B and C, the absolute value of the difference between the data A and the average value of the data B and C is defined as the amount of change. By calculating this change amount for the entire image,
The amount of change between the two images is obtained.

Here, the average value of two blocks in the vertical direction is used as the data of the frame structure image. However, this may be a simple sum. In this case, the data of the corresponding field structure image is doubled. For example, the data may be normalized. Specifically, it is preferable to use the absolute value of the difference between the value twice the data A and the sum of the data B and C as the amount of change.

The other points are the same as in the first embodiment.

(Third Embodiment) FIG. 9 is a block diagram of a scene change detection device according to a third embodiment of the present invention, and FIG. 10 is a scene change detection device according to the third embodiment of the present invention. It is a flowchart of an apparatus.

In this embodiment, as apparent from a comparison between FIG. 1 and FIG. 9, a field DCT coded block number counting section 8 and a third data memory 9 are added.
This field DCT coding block number counting section 8
When the result of the determination by the image structure determination unit 1 is a frame structure image, it is determined whether the encoding uses the frame DCT or the field DCT, and the field DCT encoding in one field is performed. Is counted (or the number of blocks may be used). Then, the number of field DCT encoded blocks counted by the field DCT encoded block number counting section 8 is recorded in the third data memory 9.

Therefore, as shown in FIG.
When the input compressed moving image reaches the image structure determining unit 1, the determining unit 1 determines whether the current image is a frame structure image or a field structure image (Step 2).
0). If it is a frame structure image, the feature amount extraction unit 2
A value using block data of two blocks in the vertical direction of the image is extracted as a feature amount, recorded in the first data memory 6 (step 21), and the field DCT coding block number counting unit 8 outputs the field DCT code. The number of coded blocks is counted, and the number of blocks is recorded in the third data memory 9 (step 23). Step 2
The order of 1 and 23 can be changed.

On the other hand, if the image is a field structure image, the characteristic amount extracting unit 2 extracts a value using the block data of one block as a characteristic amount and records it in the first data memory 6 (step 22). . That is, the frame structure image uses twice the data in the vertical direction as the field structure image.

Then, in step 24, the extracted data comparing section 3 compares the current feature value with the preceding feature value, and records the comparison result information in the second data memory 7. Next, in step 25, the scene change determination unit 4
Compares the comparison result information with a threshold value input from the scene change determination data input unit 5 to determine whether a scene change can be determined. Further, in step 26, the scene change determining unit 4 compares the number of blocks recorded in the third data memory 9 with a threshold value to determine whether a scene change can be determined. Then, the scene change judging unit 4 determines in step 25 or step 26
If a scene change is determined in any of the above, the position where the scene change has occurred is output as a detection result.

As shown in FIG. 11, when two scenes are mixed in the field, the amount of change between the input image and the image immediately before and after may be small, causing detection omission. Was causing it. However, as described in the section of the related art, when field DCT is performed on such an input image, the compression ratio increases.

Therefore, the field DCT in the frame
Are frequently used, it can be considered that the correlation between the odd field and the even field in the frame is low. For this reason, the feature quantity extraction unit 2 uses the field DCT
The number of encodings is counted, and this is added to the comparison target as the third variation in the frame.

The extracted data comparison unit 3 is the same as that of the first and second embodiments, but the scene change judgment unit 4 adds the third and third embodiments in addition to the judgment in the first and second embodiments. When comparing the amount of change with a third threshold value preset for the third amount of change, when the third amount of change is greater than the third threshold value, a scene change between the fields of the frame is performed. Judge that there is a change.

As described above, the field DCT
By counting the number of times that is used, it is possible to detect a scene change (illustrated in FIG. 11) existing between two fields of a frame structure image, which has been very difficult to detect conventionally.

Further, in the present embodiment, the following changes are made regarding the threshold value output from the scene change determination data input section 5 to the scene change determination section 4. That is, with the maximum change amount of the image as a reference (100%), the threshold is defined as a predetermined percentage of the reference.

For example, in comparison images A and B, if the minimum value of the data of one block to be compared is 0 and the maximum value is 255, the maximum change amount of one block is 255. If the number of blocks used for comparing images is 1320, the maximum change amount of the entire image is
255 * 1320 = 336600. This change amount is set as a reference (100%). Also, the threshold here is preferably, for example, about 3% (10098) to 10% (33660).

Of course, if the number of data to be used or the maximum change amount of the data changes, the threshold value changes accordingly, but the ratio between the threshold value and the reference is fixed.

As a result, even if the image size (vertical × horizontal) changes or the type of data used for determination changes, it is possible to suppress a variation in detection and obtain a substantially uniform detection result. it can.

(Fourth Embodiment) FIG. 12 is a block diagram of a scene change detecting device according to a fourth embodiment of the present invention, and FIG. 13 is a scene change detecting device according to the fourth embodiment of the present invention. It is a flowchart of an apparatus.

In the present embodiment, as apparent from a comparison between FIG. 1 and FIG. 12, the scene change determining unit 4 does not directly output the detection result, but determines that the scene change is a scene change. The scene change position information is temporarily stored in the fourth data memory 11.

In the scene change determination data input section 5, an interval on the time axis from a reference scene change to a target scene is set. Then, a scene change interval search unit 10 is added. The scene change interval search unit 10 obtains an interval on the time axis between pieces of scene change position information recorded in the fourth data memory 11, and is provided from the obtained interval and the scene change determination data input unit 5. Compare with interval.

Therefore, as shown in FIG. 13, the scene change judging section 4 looks for a scene change as in the previous embodiment (step 30). When the scene change determining section 4 finds a scene change, the scene change position information is stored in the fourth data memory 11 (step 31).

The scene change interval search unit 10
Accesses the fourth data memory 11 and checks the interval between scene changes (step 32). When a set of scene changes matching the interval given from the scene change determination data input unit 5 is found, The position information of the set consisting of the beginning and end of the found scene change is output as a detection result (step 33).

For example, when it is desired to detect a 30-second scene from a 5-minute video, a scene change is detected from the entire video, and the result is recorded in the fourth data memory 11. After that, a set of scene changes at exactly 30-second intervals is searched for from the recorded data, and is output as a detection result.

Referring to FIG. 14 as an example, scene changes 1 to
When scene change 4 is detected, if scene change 1 and scene change 4 are exactly 30 seconds apart,
A set of scene change 1 and scene change 4 is output as a detection result.

As a result, for example, it is possible to extract only a CM portion from a large amount of video obtained from a television broadcast or the like, or to extract news or a program with a fixed broadcast time length.

Further, when a certain scene change is found, it is possible to omit the scene change determination up to a given interval, to reduce unnecessary detection operation as much as possible, and to shorten the processing time.

For example, from a long video, a 15-second CM
If you want to detect only
Give seconds. The scene change interval search unit 10 determines whether a scene change is detected in a frame 15 seconds later from a frame detected as a scene change by calculation, and only when a scene change is detected 15 seconds later, The frame and the frame 15 seconds later are output as the requested scene change. The 15 seconds are detected as the target scene. To continuously search for a target scene, it is determined whether a scene change exists 15 seconds after the last detected scene change, and if no scene change is detected, the scene change determination is continued from the next frame. If detected, that becomes the target scene.

In the example shown in FIG. 14, if a scene change 1 is detected and a scene change 4 occurs exactly 15 seconds after that, a determination is made between the scene change 2 and the scene change 3 existing during the 15 seconds (wastefulness). Determination) can be skipped.

As described above, in this case, it is determined whether or not a frame after the first scene change is found is a scene change by a time given as a target interval. Therefore, processing for determining a scene change in the detected target scene can be omitted, and processing time can be reduced.

Here, the “recording medium in which the program for detecting a scene change of a compressed moving image is recorded in a computer-readable manner” as used in the present specification includes a case where the program is distributed and distributed to a plurality of recording media. Also, regardless of whether this program is part of the operating system or not, various processes or threads (DLL, O
When a part of the function is taken over by CX, ActiveX, or the like (including a trademark of Microsoft Corporation), the case where the part relating to the substituted function is not stored in the recording medium is also included.

FIG. 1, FIG. 7, FIG. 9 and FIG.
Etc.), a stand-alone system is illustrated, but a server / client system may be used.
In other words, in addition to the case where only one terminal includes all the elements appearing in this specification, one terminal is a client and is connected to a server or network to which it can be connected, in whole or in part. It does not matter if the element exists.

Further, most of the elements shown in FIG. 1 and the like may be provided on the server side, and the client side may be, for example, only a WWW browser. In this case, various kinds of information are usually on a server and are basically distributed to clients via a network. However, when necessary information is on a server, the storage device of the server is stored in the When it is on the client,
The recording device of the client is the “recording medium” here.

Further, the "program for detecting a scene change of a compressed moving image" includes, in addition to an application that has been compiled into a machine language, a program that actually exists as an intermediate code interpreted by the above-described process or thread, And the source code are stored on the “recording medium”, and the compiler and linker that can generate the machine language application from these are stored on the “recording medium”, or at least the resources and source code are stored on the “recording medium” This includes the case where an interpreter capable of generating an application of the intermediate code from these is present in the “recording medium”.

[0094]

According to the present invention, a compressed moving image in which a field structure image and a frame structure image are mixed is input, and a scene change in the input compressed moving image is detected. Therefore, a scene change can be detected without worrying about the image structure, and an important index can be obtained when a compressed moving image is searched. According to the first, second, seventh, eighth, twelfth, and thirteenth aspects, a scene change can be detected uniformly even when a frame structure image and a field structure image are mixed on a time axis. In addition, whether a frame DCT is used or a field DCT is used in a frame structure image, a scene change can be detected without any special consideration.

According to the third, ninth and fourteenth aspects, by counting the number of field-coded blocks and comparing the counted number with a threshold value, it is extremely difficult to detect the number of field-coded blocks. Existing scene changes can also be detected.

According to the structure of the fourth, tenth and fifteenth aspects, some targets can be easily extracted from a long input moving image, and unnecessary scene change determination can be omitted as much as possible. As a result, the processing time can be reduced. .

According to the fifth, eleventh, and sixteenth aspects, even if the size of the image or the type of data changes, the detection variation is small, and a uniform detection result can be obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram of a scene change detection device according to a first embodiment of the present invention.

FIG. 2 is a flowchart of a scene change detection device according to the first embodiment of the present invention.

FIG. 3 is a detailed flowchart of the same.

FIG. 4 is a detailed flowchart of the same.

FIG. 5 is a model diagram of an input compressed moving image according to the first embodiment of the present invention.

6A is a view showing an example of a field structure image according to the first embodiment of the present invention (top field); FIG. 6B is a view showing an example of the same field structure image (bottom field);
(C) Illustrative view of the same frame structure image

FIG. 7 is a block diagram of a scene change detection device according to a second embodiment of the present invention.

FIG. 8 is a flowchart of a scene change detection device according to a second embodiment of the present invention.

FIG. 9 is a block diagram of a scene change detection device according to a third embodiment of the present invention.

FIG. 10 is a flowchart of a scene change detection device according to a third embodiment of the present invention.

FIG. 11 is a view showing an example of a scene change existing between fields according to the third embodiment of the present invention.

FIG. 12 is a block diagram of a scene change detection device according to a fourth embodiment of the present invention.

FIG. 13 is a flowchart of a scene change detection device according to a fourth embodiment of the present invention.

FIG. 14 is a model diagram of a scene change according to the fourth embodiment of the present invention.

15A is an explanatory diagram of a macroblock structure in a conventional frame DCT. FIG. 15B is an explanatory diagram of a macroblock structure in a conventional field DCT.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Image structure judgment part 2 Feature amount extraction part 3 Extracted data comparison part 4 Scene change judgment part 5 Scene change judgment data input part 6 First data memory 7 Second data memory 8 Field DCT coding block count Unit 9 Third data memory 10 Scene change interval search unit 11 Fourth data memory

 ────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Jun Ikeda 1006 Kazuma Kadoma, Kazuma-shi, Osaka Matsushita Electric Industrial Co., Ltd. F-term (reference) 5C059 KK36 MA00 MA23 NN28 NN43 PP05 PP06 PP07 TA64 TB07 TB08 TC14 TC43 TD12 UA38 UA39 5J064 AA01 BA16 BB03 BC01 BD02

Claims (16)

[Claims] An image structure determining unit for determining an image structure of an input compressed moving image; and, if the determination result of the image structure determining unit is a frame structure image, the image structure is vertically doubled with respect to the field structure image. A feature amount extraction unit that extracts a feature amount based on the minute block data, a storage area for recording the block data extracted by the feature amount extraction unit, and a video change amount by comparing the extracted block data. An apparatus for detecting a scene change of a compressed moving image, comprising: an extracted data comparison unit to be obtained; and a scene change determination unit that determines a scene change using the amount of change obtained by the extraction data comparison unit. 2. The method according to claim 1, wherein one of the input compressed moving images has a structure
A feature amount extraction unit for extracting a feature amount based on the block data of the block, a storage area for recording the block data extracted by the feature amount extraction unit, and an image from which the feature amount is extracted is a frame structure image. In this case, an extracted data comparison unit for comparing a feature amount with a field structure image using twice the block data in the image vertical direction to obtain a change amount, and a scene using the change amount obtained in the extracted data comparison unit. A scene change detection device for a compressed moving image, comprising a scene change determination unit for determining a change. 3. When the image is a frame structure image, a field DCT coded block number counting unit for counting the number of blocks subjected to field DCT coding is compared with a threshold value, and the number of blocks is determined between fields. And a scene change determining unit for determining a scene change to be performed. 4. A scene change judging section for judging a scene change, and a scene for searching for a scene change existing at a start point and an end point of a specified specific interval among the scene changes detected by the scene change judging section. A scene change detection apparatus for a compressed moving image, comprising: a change interval search unit. 5. A threshold value used as a criterion for determining a scene change by the scene change determining unit includes a threshold value determined based on a maximum change amount of an image. Scene change detection device for compressed moving images. 6. A method for detecting a scene change in a compressed moving image, comprising: inputting a compressed moving image in which a field structure image and a frame structure image are mixed; and detecting a scene change in the input compressed moving image. 7. An image structure judging step for judging the image structure of an input compressed moving image, and when the judgment result of the image structure judging step is a frame structure image, the image structure is vertically doubled with respect to the field structure image. A feature amount extraction step of extracting a feature amount based on the block data of the minute, a storage area for recording the block data extracted in the feature amount extraction step, and a change amount of the video by comparing the extracted block data. A method of detecting a scene change of a compressed moving image, comprising: a step of comparing extracted data to be obtained; and a step of determining a scene change using a change amount obtained in the step of comparing extracted data. 8. The method according to claim 1, irrespective of the structure image of the input compressed moving image.
A feature amount extracting step of extracting a feature amount based on the block data of the block, a storage area for recording the block data extracted in the feature amount extracting step, and an image from which the feature amount is extracted is a frame structure image. In the case, an extracted data comparison step of comparing the feature amount with the field structure image using twice the block data in the image vertical direction to obtain a change amount, and a scene using the change amount obtained in the extracted data comparison step A scene change detecting method for a compressed moving image, comprising a scene change determining step of determining a change. 9. A field DCT encoded block number counting step for counting the number of field DCT encoded blocks when the image is a frame structure image;
A scene change determining step of comparing the number of blocks with a threshold to determine a scene change existing between fields. 10. A scene change judging step for judging a scene change, and a scene for searching for a scene change existing at a start point and an end point of a specified specific interval from the scene changes detected in the scene change judging step. A change interval search step. 11. In the scene change determining step, a threshold value used as a criterion for determining a scene change includes:
11. The method for detecting a scene change of a compressed moving image according to claim 7, wherein a threshold determined based on a maximum change amount of the image is included. 12. An image structure judging step for judging the image structure of an input compressed moving image, and when the judgment result of the image structure judging step is a frame structure image, the image structure is vertically doubled with respect to the field structure image. A feature amount extraction step of extracting a feature amount based on the block data of the minute, a storage area for recording the block data extracted in the feature amount extraction step, and a change amount of the video by comparing the extracted block data. A computer-readable program for detecting a scene change of a compressed moving image, comprising: a step of determining a scene change using the amount of change obtained in the step of extracting extracted data; The recording medium on which it was recorded. 13. An image processing apparatus according to claim 1, wherein:
A feature amount extracting step of extracting a feature amount based on one block of block data, a storage area for recording the block data extracted in the feature amount extracting step, and an image from which the feature amount is extracted is a frame structure image. In some cases, an extracted data comparison step of comparing a feature amount with a field structure image using twice the block data in the image vertical direction to obtain a change amount, and using the change amount obtained in the extracted data comparison step A recording medium having a computer-readable recording of a program for detecting a scene change of a compressed moving image, comprising a scene change determining step of determining a scene change. 14. When the image is a frame structure image, a field DCT coded block number counting step for counting the number of field DCT coded blocks, and comparing the number of blocks with a threshold value to determine the number of blocks between fields. Recording a scene change detection program for a compressed moving image in a computer-readable manner. 15. A scene change judging step for judging a scene change, and a scene for retrieving a scene change existing at a start point and an end point of a specified specific interval from the scene changes detected in the scene change judging step. A computer-readable recording medium for storing a program for detecting a scene change of a compressed moving image, the program comprising: a change interval search step. 16. In the scene change determining step, a threshold value used as a criterion for determining a scene change includes:
16. The recording medium according to claim 12, further comprising a computer-readable recording program for detecting a scene change of a compressed moving image, wherein the program includes a threshold determined based on a maximum change amount of the image.