JPH05236458A - Moving image processor - Google Patents

Abstract

PURPOSE:To reduce time for moving image analysis and image compression and the cost of a hardware or the like by detecting a scene change point corresponding to the two kinds of differential data for a certain frame and frames with fixed width positioned before and behind that frame. CONSTITUTION:Among transformed data obtained at a DCT part 12, the DC component coefficient of that block is inputted by a DC component arithmetic part 16, after those coefficients are simultaneously added, they are divided with the number of blocks so as to calculate an average luminance value in the entire frame. The calculated value is transmitted to an inter-frame difference part 18 as it is and temporarily stored in an arithmetic result storage part 17a at a storage part 17. The difference part 18 calculates difference between the average luminance value transmitted from the arithmetic part 16 and an average luminance value for preceding one frame read from the storage part 17a. The calculated inter-frame difference is stored in a differential result storage part 17b for each frame number. Since the storage part 17b stores the inter-frame difference totally for the 61 frame of that frame and respectively preceding and following frames 30 while always updating it, the differential result stored in the storage part 17b is read to a detection part 19 and here, the scene change point is detected by any statistical method.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image processing apparatus for detecting a scene change point in moving image data of temporally consecutive frames.

[0002]

2. Description of the Related Art In recent years, with the accumulation of moving images using a computer, the development of display technology, and the realization of multimedia information, moving images have been frequently used.

An image analysis method such as indexing, which is required to grasp and edit the contents of such a moving image,
At present, most of the fast-forward playback, cue function, indexing, etc. are done manually.
Further, some methods of performing image analysis by hardware have been proposed, but all of these are for performing a dedicated circuit, and thus the overall circuit scale becomes large.

[0004]

Since moving images require a huge amount of information, storage techniques such as image compression will be indispensable in the future in order to sufficiently handle moving images on a computer. .. Further, work such as moving image editing requires an image analyzing work of grasping the contents in a short time and extracting a desired portion from a huge amount of moving image data.

However, when the image compression and the image analysis are individually performed, dedicated hardware is required individually, and there is a problem that the circuit scale becomes large and the cost becomes high.

The present invention has been made in view of the above circumstances, and an object of the present invention is to reduce the circuit scale by using at least a part of a circuit required for image compression and image analysis, thereby reducing the cost. An object of the present invention is to provide a moving image processing device capable of reducing the above.

[0007]

That is, according to the present invention, there is provided an orthogonal transform unit for compressing spatial information of moving image data by orthogonal transform, and an image obtained by compressing a part of the transform result obtained by the orthogonal transform unit. A first storage unit that stores the feature amount, a calculation unit that calculates a difference between adjacent frames of the image feature amount stored in the first storage unit, and a difference data obtained by the calculation unit are stored. A second storage unit, and two types of differences for the frame and a frame of a certain number of widths located in front of the frame and a frame of a certain number of widths of the frame and the frame stored thereafter in the second storage unit It is equipped with a detector that detects scene change points from data, and uses part of the information obtained on the assumption of image compression to detect scene change points as image analysis and create index information. As a result, not only can the time required for the above work and the cost of hardware, etc. be significantly reduced, but also changes from a scene with a lot of movement to a scene with little movement, or from a scene with little movement to a lot of movement It is possible to detect the scene change in an appropriate manner, and to reduce the false detection of the scene change point that tends to occur especially in a scene with a lot of movement.

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 shows the circuit configuration thereof. Moving image data input in the form of RGB or YUI from a data input unit 11 is sent to a discrete cosine transform unit (hereinafter abbreviated as "DCT unit") 12. .. The DCT unit 12 compresses spatial information of the input same image data by a discrete cosine transform that performs orthogonal transform according to its frequency component, and outputs the transformed data to the quantizing unit 13 and the DC component calculating unit 16.

The quantizing unit 13 quantizes the converted data obtained by the DCT unit 12 by a prescribed number of bits and outputs the quantized data to the encoding unit 14. The encoding unit 14 variable-length-encodes the converted data quantized by the quantization unit 13, further compresses the data amount, and then outputs the data to the compression result output unit 15. Compression result output section
Reference numeral 15 outputs the encoded converted data from the encoding unit 14 to an external recording medium, for example, as a final compression result.

On the other hand, the DC component calculator 16 is a DCT unit.
The average luminance of one frame is calculated by calculating the average value for one frame by the direct current (DC) component value for each block obtained from 12, and is output to the storage unit 17 and the inter-frame difference unit 18.

The storage unit 17 has three areas, a calculation result storage unit 17a, a difference result storage unit 17b, and a detection result storage unit 17c. The calculation result from the DC component calculation unit 16 is stored in the calculation result storage unit.
It is stored in 17a.

The inter-frame difference section 18 is a DC component calculation section 16
1 stored in the operation result storage unit 17a
The difference from the calculation result before the frame is calculated, the difference result is sent to the difference result storage unit 17b of the storage unit 17 and stored therein, and the difference result stored in the difference result storage unit 17b is detected by the detection unit.
Read to 19.

The detection unit 19 detects a scene change point by adopting a statistical method for the difference result stored in the difference result storage unit 17b, and stores the frame number and the number of changes as a detection result. It is stored in the detection result storage unit 17c.

Thus, the detection result output unit 20 reads out the frame number of the scene change point and the number of changes stored in the detection result storage unit 17c, and outputs it as a result of image analysis, for example, by multi-display. The operation of the embodiment having the above configuration will be described below.

The time-continuous image data input from the data input unit 11 is subjected to discrete cosine transform for each frame in the DCT unit 12. This Discrete Cosine Transform is used for spatial information compression. By orthogonally transforming image data in the frequency domain, spatial correlation is removed, and energy is concentrated on some coefficients to reduce the amount of information. It is possible to reduce significantly. Specifically, one frame of target image data is divided into a plurality of blocks of N pixels × N pixels, and two-dimensional discrete cosine transform based on the following equation (1) is performed for each block.
That is,

[0017]

[Equation 1]

In the conversion coefficient F (u, v), F (0,0) is a direct current (DC) component coefficient corresponding to the average luminance value of the block, and the other is an alternating current (AC) component coefficient. Will correspond to.

In this way, the converted data whose information amount is significantly reduced is quantized by the quantizing unit 13 into a predetermined number of bits, and the encoding unit 14 performs hyper-biased encoding. Is output to. These DCT
Since the technique relating to the compression of the image data via the unit 12, the quantizing unit 13 and the encoding unit 14 is already well known, detailed description thereof will be omitted.

On the other hand, of the converted data obtained by the DCT unit 12,
As described above, DC corresponding to the average luminance value of the block
The DC component calculation unit 16 inputs the component coefficient, adds the DC component coefficients of all the blocks of the corresponding frame at once, and then divides by the number of blocks to calculate the average luminance value of the entire frame. The calculated average luminance value is sent to the inter-frame difference section 18 as it is and is temporarily stored in the calculation result storage section 17a of the storage section 17.

The inter-frame difference section 18 is a DC component calculation section 16
The difference between the average brightness value sent from the device and the average brightness value of one frame before read from the calculation result storage unit 17a is calculated. This inter-frame difference generally indicates a value that stands out at a scene change point, and the calculated inter-frame difference is stored in the difference result storage unit 17b for each frame number.

In the difference result storage unit 17b, the difference between the frame and the preceding and following 30 frames, which is 61 frames in total, is constantly updated and stored, and the difference result stored in the difference result storage unit 17b is stored. Is read by the detection unit 19, and the scene change point is detected by a statistical method. Specifically, the detection unit 19 calculates the average value and the variance value of the difference result, and the following equation (2), that is, [threshold value] = [average value] + [variance value] × [block pixel constant] N] (2) is used to calculate the threshold value, and threshold processing is performed on the difference result.

In this case, the detection unit 19, as shown in FIG. 2, has a first threshold value calculated from a total of 31 frames of the frame and the previous 30 frames, and a total of 31 frames of the frame and the subsequent 30 frames. A second threshold value calculated from the above is set separately, and a constant N as a result depending on the factors of the elapsed time as shown in FIG. 3 is set for each threshold value.
Is substituted into the above equation (2) for weighting, and then threshold processing is performed.

The weighting of the constant N shown in FIG. 3 is set in the detecting unit 19 as a characteristic that adaptively changes on the basis of the knowledge that scene changes rarely occur continuously in a short time. To be done. When the detection unit 19 has a difference result that exceeds at least one of the first threshold value and the second threshold value, the frame is determined as a scene change point as indicated by C in FIG. Perform detection processing. The processing result obtained by the detection unit 19 is composed of a scene number, a frame number, a scene length (time), etc. as index information, and these are stored in the detection result storage unit 17c.

By detecting the scene change points by performing the threshold processing in this way, generally, a general moving image in which various scenes such as a scene with a lot of movement and a scene with little movement are continuous in time series. With respect to the image, it is possible to accurately detect the change point of the scene such as a change from a scene with a lot of movement to a scene with a little movement, a change from a scene with little movement to a scene with a lot of movement, and a scene with a lot of movement False detections in can be reduced.

The detection result output unit 20 calls the index information from the detection result storage unit 17c in response to the user's request, and outputs the image of the scene change point and other information by, for example, multi-display.

In the above embodiment, the moving image data is discrete cosine transformed and the scene change point is obtained by the DC component thereof, but the orthogonal cosine transform is not the only possible method, and another orthogonal transform method may be used. , A of discrete cosine transform
It is also possible to utilize the C component. Further, the same result can be obtained by obtaining the average luminance value of the specific portion of the divided frame instead of obtaining the average luminance value of the entire frame.

Further, the number of frames stored in the difference result storage section 17b for use in determining the threshold value can be changed according to the content of the moving image, and further, as described above, the number of frames can be shortened. It is also possible to adaptively change the constant N in the above equation (2) by a characteristic other than the characteristic shown in FIG. 3 by using the knowledge that the scene is unlikely to change continuously. .. In addition, the present invention can be variously modified and implemented without departing from the scope of the invention.

[0029]

As described above in detail, according to the present invention, an orthogonal transform unit for compressing spatial information of moving image data by orthogonal transform, and an image obtained by compressing a part of the transform result obtained by the orthogonal transform unit are provided. A first storage unit that stores the feature amount, a calculation unit that calculates a difference between adjacent frames of the image feature amount stored in the first storage unit, and a difference data obtained by the calculation unit are stored. A second storage unit, and two types of differences for the frame and a frame of a certain number of widths located in front of the frame and a frame of a certain number of widths of the frame and the frame stored thereafter in the second storage unit Since a detection unit that detects a scene change point from data is provided, a part of the information obtained on the assumption of image compression is used to detect the scene change point as image analysis and create index information. But Therefore, not only can the time and hardware cost required for the above work be significantly reduced, but also the change from a scene with a lot of movement to a scene with little movement, or from a scene with little movement to a scene with a lot of movement. It is possible to provide a moving image processing apparatus capable of appropriately detecting a change, and reducing erroneous detection of a scene change point that is likely to occur particularly in a scene in which the movement is rapid.

[Brief description of drawings]

FIG. 1 is a block diagram showing a circuit configuration according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating a method of calculating a threshold value by the detection unit in FIG.

FIG. 3 is a diagram showing a method of calculating a weighting constant N for the threshold value obtained in FIG.

[Explanation of symbols]

11 ... Data input section, 12 ... Discrete cosine transform (DCT)
Unit, 13 ... Quantization unit, 14 ... Encoding unit, 14 ... Encoding unit, 16 ...
DC component calculation unit, 17 ... storage unit, 17a ... calculation result storage unit,
17b ... Difference result storage unit, 17c ... Detection result storage unit, 18 ... Interframe difference unit, 19 ... Detection unit, 20 ... Detection result output unit.

Claims (1)

[Claims] 1. A moving image processing apparatus for detecting a scene change point in moving image data of temporally continuous frames, and an orthogonal transform means for compressing spatial information of the moving image data by sequential orthogonal transform, and the orthogonal transform. First storage means for storing a part of the conversion result obtained by the means as the feature quantity of the compressed image, and the difference between the adjacent frame of the image feature quantity stored in the first storage means is calculated. Calculating means, second storing means for storing difference data obtained by the calculating means, the frame stored in the second storing means, a frame of a certain number of widths located in front of the frame, and the frame. A moving image processing apparatus, comprising: a detection unit that detects a scene change point based on two types of difference data for frames of a certain number of widths that are located after that.