JPH07121555A - Method and device for extracting scene changing frame in dynamic image - Google Patents

Abstract

PURPOSE:To make it unnecessary to extract a scene changing frame based upon a restored digital image by extracting a frame corresponding to a condition that a difference is more than a prescribed threshold as a scene changing frame. CONSTITUTION:When a digital image for one frame is compressed by a data compression processing part 1, a compressed data amount counting part 2 counts the amount of compressed data and a difference calculating part 4 calculates a difference between the counted value and the compressed data amount of a frame obtained immediately before storing the amount in a compressed data amount storing part 3. When a discriminating part 5 discriminates that the difference is more than the prescribed threshold, a scene changing frame extracting part 6 extracts the frame concerned as a scene changing frame and executes necessary processing, e.g. data transmission processing to a host processor. Since a scene changing frame is extracted based upon the amount of compressed data obtained by applying data compression to a digital image in each frame, the quantity of processing for the extraction of each scene changing frame can be reduced.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for extracting a scene change frame of a moving image and an apparatus therefor, and more specifically to converting each frame of a moving image obtained by a video camera or the like into digital data and further compressing the data. The present invention relates to a method and apparatus for extracting a scene change frame of a moving image suitable for a moving image processing system for applying and accumulating.

[0002]

2. Description of the Related Art Conventionally, a still image is converted into digital data, and data compression is performed to reduce the amount of data.
An image processing system for storing in a memory or the like has been proposed and put into practical use. And, with the recent progress in semiconductor manufacturing technology, it has become possible to significantly increase the number of gates of integrated circuit chips for processing image data, so that not only still images, It has become possible to convert each frame of a moving image into digital data at a video rate, compress the data, and store it in a memory or the like. Therefore, a moving image processing system for converting a moving image into digital data in real time and compressing and storing the data can be made compact.

[0003]

However, when the compressed digital image data is read from the memory or the like and is subjected to processing such as editing, the scene change frame in the moving image must be extracted. Then, in order to extract the scene change frame, for example, the original digital image is restored by performing a restoration process on the compressed digital image data, and the restored digital images are temporally adjacent to each other. The ratio of the area where a difference equal to or greater than a predetermined threshold has occurred, based on the matching areas (pixel change area)
When the pixel change area increases rapidly, the method of detecting the frame that is temporally later in both frames as a scene change frame {“Movie cut detection”, Seita Otsuji.
Yoshinobu Tonomura, Yuji Ohba, Technical Report of IEICE,
IE 91-113-116 (Imaging Engineering), 1992 2
21st of the month} will be adopted.

Therefore, in order to extract a scene change frame, first, a process for obtaining a digital image for each original frame by subjecting the compressed data of each frame to a decompression process, and for each decompressed frame It is essential to obtain the pixel change area based on the temporally adjacent digital images. As a result, there is an inconvenience that the processing amount for extracting the scene change frame is remarkably increased, and the processing time is remarkably lengthened.

Although the above description is based on the conventional example of calculating the pixel change area, it may be the case of using another method for calculating the degree of correlation between digital images that are temporally adjacent to each other. The same inconvenience will occur.

[0006]

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and when converting each frame of a moving image into digital data and compressing and storing the data, a scene change frame is extracted, and a scene change frame is extracted. For this reason, it is possible to provide a novel method for extracting a scene change frame of a moving image and an apparatus therefor, which makes it unnecessary to decompress the compressed data and extract the scene change frame based on the decompressed digital image. Has an aim.

[0007]

In order to achieve the above object, a method for extracting a scene change frame of a moving image according to claim 1 is a method of converting each frame of a moving image into digital data and further compressing the data. , The amount of compressed data for each frame is obtained, the difference from the reference amount determined based on the amount of compressed data of the preceding frame is obtained, and the difference is greater than or equal to a predetermined threshold, the corresponding frame Is a scene change frame. Here, the amount of compressed data of the preceding frame may be the amount of compressed data of the immediately preceding frame,
It may be an average value of the amount of compressed data of the preceding frames within a predetermined number.

According to a second aspect of the present invention, there is provided a moving image scene change frame extracting apparatus which converts each frame of a moving image into digital data and further compresses the data to obtain a compressed data amount of each frame. Acquisition means,
The compressed data amount obtained by the compressed data amount acquisition means,
A difference calculation unit that calculates a difference from a reference amount that is determined based on the data amount of the compressed data of the preceding frame, and the difference frame that is calculated by the difference calculation unit is a predetermined threshold value or more Frame extraction means for extracting as a scene change frame.

[0009]

According to the method of extracting a scene change frame of a moving image of claim 1, when each frame of the moving image is converted into digital data and further compressed and stored, the data amount of the compressed data for each frame is calculated. Obtain, obtain a difference with the reference amount determined based on the data amount of the compressed data of the preceding frame, because the corresponding frame is extracted as a scene change frame on condition that the difference is equal to or more than a predetermined threshold, Compared with the case where the correlation between digital images for each frame is obtained, the processing amount can be significantly reduced and the processing time can be significantly reduced. Therefore, when the compressed data is restored and the editing process is performed, it is not necessary to extract the scene change frame based on the restored digital image, and the processing amount of the editing process can be significantly reduced. Can be significantly shortened.

According to another aspect of the present invention, there is provided the moving image scene change frame extraction apparatus, wherein when each frame of a moving image is converted into digital data and further compressed and stored, the data amount of the compressed data for each frame is calculated. The difference calculation unit calculates a difference between the compressed data amount obtained by the compressed data amount acquisition unit and obtained by the compressed data amount acquisition unit and a reference amount determined based on the data amount of the compressed data of the preceding frame. Then, on condition that the difference calculated by the difference calculating means is equal to or more than a predetermined threshold value, the frame extracting means extracts the corresponding frame as a scene change frame. Therefore, as compared with the case where the correlation between digital images for each frame is obtained, the processing amount can be remarkably reduced and the processing time can be remarkably shortened. Therefore, when the compressed data is decompressed and the editing process is performed, it is not necessary to extract the scene change frame based on the decompressed digital image, and the amount of the editing process can be significantly reduced.
The required time can be significantly shortened.

[0011]

Embodiments will be described in detail below with reference to the accompanying drawings showing embodiments. FIG. 1 is a flow chart for explaining an embodiment of the method for extracting a scene change frame of a moving image according to the present invention. When a moving image consisting of digital images is given, it is converted into a digital image for one frame in time order at step SP1. In step SP2, the amount of compressed data of the digital image of the corresponding frame is counted, and in step SP3, the amount of compressed data counted in step SP2 and the digital image of the immediately preceding frame are calculated. A difference from the amount of compressed data is calculated, and in step SP4, it is determined whether the calculated difference is larger than a predetermined threshold value. If it is determined that the calculated difference is larger than the predetermined threshold value, Is step SP
In 5 the corresponding frame is extracted as a scene change frame. If it is determined that the difference calculated in step SP4 is less than or equal to the predetermined threshold value, or if the process of step SP5 is performed, the amount of compressed data counted in step SP2 is calculated in step SP6. The amount of compressed data of the digital image of the immediately preceding frame is held, and it is determined in step SP7 whether or not all the frames have been processed. If it is determined that there is a frame that has not been processed, the process of step SP1 is repeated. On the contrary, if it is determined in step SP7 that the processing has been performed for all the frames, the series of processing is terminated.

Regarding the frame extracted as the scene change frame in step SP5, information indicating that it is a scene change frame may be supplied to the host computer of the moving image processing system and held therein. It is preferable to store the image data in a memory or the like in association with the compressed data for each frame. Further, as the method for data compression in step SP1, for example, JPEG which is the standard of CCITT / ISO is preferably adopted, but it goes without saying that other data compression methods may be adopted. However, it is necessary that the parameters for data compression be kept constant throughout a series of moving image processing. The data compressed based on JPEG is S. O. I. (Startof
image), frame header, scan header, quantization table, Huffman table, DCT (discrete cosine transform) image and E.I. O. I. (End of
image), and the DCT image will vary from digital image to digital image. The amount of data compressed based on JPEG is about 15 Kbytes as a whole in the resolution of a normal TV image, whereas the amount of compressed data is about several hundred to 1 Kbytes between frames in which a scene changes. Since there is a change, several hundred bytes, for example, may be adopted as the threshold value.

As is apparent from the above description, according to this embodiment, the scene change frame is extracted based on the compressed data amount when the data compression is applied to the digital image of each frame. The amount of processing for extracting the scene change frame can be significantly reduced. Therefore, when the data-compressed moving image is read from the memory or the like and subjected to the edit processing, the processing can be easily performed based on the information indicating the scene change frame that has already been extracted. Since it is not necessary to extract a scene change frame as a process, it is possible to significantly reduce the amount of processing such as the editing process, and it is possible to greatly reduce the required time. In addition, even when considering the case of decompressing compressed data to make a visual judgment in order to confirm whether the scene change frame extracted by this method is correct, only a small number of frames including the extracted scene are decompressed. Since processing etc. should be performed,
The processing time required for confirmation can be shortened.

[0014]

[Embodiment 2] FIG. 2 is a flow chart for explaining another embodiment of the method for extracting a scene change frame of a moving image according to the present invention. The difference from the flow chart of FIG.
Instead of the process of P3, step SP3 is adopted in which the difference between the amount of compressed data counted in step SP2 and the average value of the amount of compressed data of digital images of a frame within a preceding predetermined number is calculated. Between the steps SP5 and SP6, the step SP5 of resetting the average value
a, and a point obtained by inserting step SP5b that adopts the amount of compressed data obtained in step SP2 as a new average value, and when it is determined that the difference calculated in step SP4 is less than or equal to a predetermined threshold value, Step SP5c of calculating an average value of the amount of compressed data of a predetermined number of frames including the corresponding frame and adopting it as a new average value
And the point where step SP6 is omitted.

Therefore, in the case of this embodiment, the fluctuation of the amount of compressed data due to the movement of the object when the scene has not changed can be greatly reduced by the averaging process, and the scene change frame The extraction accuracy can be improved.

[0016]

[Embodiment 3] FIG. 3 is a block diagram showing an embodiment of a scene change frame extracting apparatus for a moving image according to the present invention. A data compression processing unit for performing data compression processing on a digital image for one frame in time order. 1, a compressed data amount counting unit 2 for counting the amount of compressed data of a digital image for one frame, and compression for temporarily holding the compressed data amount of the immediately preceding frame counted by the compressed data amount counting unit 2. The difference between the amount of compressed data counted by the data amount holding unit 3 and the compressed data amount counting unit 2 and the amount of compressed data of the digital image of the immediately preceding frame held in the compressed data amount holding unit 3 is calculated. Difference calculation unit 4, a determination unit 5 that determines whether the difference calculated by the difference calculation unit 4 is greater than a predetermined threshold value, and a determination unit that indicates that the calculated difference is greater than a predetermined threshold value. Determination and scene change frame extracting unit 6 for extracting the frames corresponding response as a scene change frame in the results, based on the amount of compressed data count in the compressed data amount counter 2 compressed data amount holding section 3
And a compressed data amount updating unit 7 for updating the contents of The compressed data for each frame and the data indicating the scene change frame are sequentially stored in, for example, a memory (not shown), but the configuration of this portion is not shown.

The operation of each component is the same as the process of the corresponding step of the flowchart of FIG. 1, and therefore detailed description thereof will be omitted. Therefore, in the case of this embodiment, when one frame of digital image is compressed by the data compression processing unit 1, the compressed data amount counting unit 2 counts the compressed data amount and the compressed data amount holding unit 3 holds it. The difference calculation unit 4 calculates the difference from the compressed data amount of the immediately preceding frame. Then, the discriminating unit 5 discriminates whether or not the difference calculated by the difference calculating unit 4 is larger than a predetermined threshold value.
When the discriminating unit 5 discriminates that the difference is larger than a predetermined threshold value, the scene change frame extracting unit 6 extracts the corresponding frame as a scene change frame and performs necessary processing, for example, data transmission processing to the host processor. And so on. Of course, when the discriminating unit 5 discriminates that the difference is less than or equal to the predetermined threshold value, the corresponding frame is treated as a non-scene change frame. afterwards,
Regardless of the determination result by the determination unit 5, the compressed data amount updating unit 7 updates the content of the compressed data amount holding unit 3 based on the amount of compressed data counted by the compressed data amount counting unit 2.

The scene change frames can be sequentially extracted from the moving image by repeating the above series of processing. As is clear from the above description, according to this embodiment, the scene change frame is extracted based on the compressed data amount when the digital image of each frame is subjected to data compression. The amount of processing for extraction can be significantly reduced. Therefore, when the data-compressed moving image is read from the memory or the like and subjected to the edit processing, the processing can be easily performed based on the information indicating the scene change frame that has already been extracted. Since it is not necessary to extract a scene change frame as a process, it is possible to significantly reduce the amount of processing such as the editing process, and it is possible to greatly reduce the required time.

[0019]

[Fourth Embodiment] FIG. 4 is a block diagram showing another embodiment of the scene change frame extracting apparatus for a moving image according to the present invention. The difference from the block diagram of FIG. 3 is that the compressed data amount holding unit 3 is used. The point that an average value holding unit 3a that holds the average value of the compressed data amount of a predetermined number or less of the preceding frames is adopted. Instead of the difference calculation unit 4, the amount of compressed data counted by the compressed data amount counting unit 2 And a difference calculation unit 4a that calculates the difference between the average value held in the average value holding unit 3a and the difference value calculated by the difference calculation unit 4a instead of the determination unit 5. The determination unit 5a that determines whether or not the average value held in 3a is larger than the average value held in 3a is used, and the average value is held in response to the relevant frame being extracted as the scene change frame by the scene change frame extraction unit 6. Reset the contents of part 3a In place of the compressed data amount updating unit 7, and in response to the reset of the contents of the average value holding unit 3a by the reset unit 3b, the compressed data amount counting unit 2 In response to the average value setting unit 7a that sets the content of the average value holding unit 3a based on the count value by the determination unit 5a and the determination unit 5a that the difference is less than or equal to the average value, the compressed data amount counting unit 2 Only the average value updating unit 7b that updates the contents of the average value holding unit 3a by calculating the average value of the compressed data amount of a predetermined number of frames including the count value obtained by

The operation of each component is the same as the process of the corresponding step of the flow chart of FIG. 2, so detailed description will be omitted. Therefore, in the case of this embodiment, the fluctuation of the amount of compressed data due to the movement of the object when the scene has not changed can be significantly reduced by the averaging process, and the extraction accuracy of the scene change frame can be improved. Can be increased. Immediately after the scene change frame is extracted, the average value is calculated based on the number of frames smaller than the predetermined number, but this does not cause any inconvenience to the extraction function.

[0021]

As described above, the invention of claim 1 can significantly reduce the processing amount as compared with the case of obtaining the correlation between digital images for each frame.
The processing time can be significantly shortened, and when the compressed data is restored and the edit processing is performed, it is not necessary to extract the scene change frame based on the restored digital image. There is a unique effect that the amount can be remarkably reduced and the required time can be remarkably shortened.

According to the invention of claim 2, the processing amount can be remarkably reduced and the processing time can be remarkably shortened as compared with the case where the correlation between the digital images for each frame is obtained. , When the compressed data is restored and the editing process is performed, it is not necessary to extract the scene change frame based on the restored digital image, the processing amount of the editing process can be significantly reduced, and the required time can be reduced. It has a unique effect that it can be remarkably shortened.

[Brief description of drawings]

FIG. 1 is a flowchart illustrating an embodiment of a method for extracting a scene change frame of a moving image according to the present invention.

FIG. 2 is a flowchart illustrating another embodiment of the method for extracting a scene change frame of a moving image according to the present invention.

FIG. 3 is a block diagram showing an embodiment of a scene change frame extraction device for a moving image according to the present invention.

FIG. 4 is a block diagram showing another embodiment of the scene change frame extraction device for a moving image according to the present invention.

[Explanation of symbols]

 2 Compressed data amount counting unit 4, 4a Difference calculation unit 5, 5a Discrimination unit 6 Scene change frame extraction unit

Claims (2)

[Claims] 1. A moving image processing method for converting each frame of a moving image into digital data, further compressing and storing the data, and obtaining a data amount of the compressed data for each frame to obtain a data amount of the compressed data of the preceding frame. A method for extracting a scene change frame of a moving image, characterized in that a corresponding frame is extracted as a scene change frame on condition that the difference is equal to or larger than a predetermined threshold value, and the difference is larger than a reference amount. 2. A compressed data amount acquisition means (2) for obtaining a data amount of compressed data for each frame in a moving image processing apparatus for converting each frame of a moving image into digital data and further compressing and storing the data. A difference calculating means (4) (4a) for calculating a difference between the compressed data amount obtained by the compressed data amount acquiring means (2) and a reference amount determined based on the data amount of the compressed data of the preceding frame; Frame extraction means (5) (5) that extracts the corresponding frame as a scene change frame on condition that the difference calculated by the calculation means (4) (4a) is equal to or greater than a predetermined threshold value.
a) A scene change frame extraction device for a moving image, including: