JP2011129979A - Image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reliably detect a scene change from moving images compressed by a compression system using inter-frame or inter-field motion compensation. <P>SOLUTION: A reduced image generation part 130 acquires the reduced images of a plurality of pictures configuring the moving images in encoding order from an MPEG-2 stream. A frequency component acquisition part 150 extracts each of the frequency components of the plurality of reduced images obtained by the reduced image generation part 130. A scene change degree calculation part 160 rearranges the frequency components of the plurality of reduced images in display order, and obtains a difference of the frequency components acquired by the frequency component acquisition part 150 for each two continuous reduced images or each reduced images of the leading I pictures of two continuous GOPs (Groups Of Pictures), as a scene change degree indicating the degree of the possibility of the scene change. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing technique, specifically, a technique for performing scene change detection on a compressed moving image.

  Scene change detection is performed for various purposes to identify a scene switching point for a compressed moving image. For example, in a video recording apparatus, a scene change location is often set as the starting point of video editing. Also, scene change images are often used to search recorded video scenes. Furthermore, there are cases where the digital broadcast stream is recorded with different resolutions and compression ratios, transcoding, etc. In these cases, encoding at the time of recording is performed using the information of the scene change. This can improve the image quality. Note that “transcoding” means recording by changing the compression encoding method. For example, the encoding method is changed from MPEG-2 to H.264. Recording to change to H.264 corresponds to this. For convenience of explanation, hereinafter, compressing by changing the resolution and compression rate, transcoding and the like are referred to as “recompression”.

  When a scene change occurs, there is no correlation between the current picture and the past picture. When performing compression by a compression method using interframe motion compensation, performing interframe motion compensation between uncorrelated frames has a problem that may cause image quality degradation. Therefore, it is necessary to detect a picture in which a scene change occurs at the time of the above-described recompression before encoding the picture.

  Here, some techniques for performing scene change detection on a compressed moving image will be described. In the description, MPEG-2 is taken as an example as an original compression method for moving images. In the description of the present invention, unless otherwise specified, “picture” means a structural unit of a moving image, and includes both “frame” and “field”.

  MPEG-2 employs encoding that combines frame or inter-field motion compensation and orthogonal transform (DCT) for each block. A block is a picture screen divided into 8 pixels × 8 lines. In the commonly used 4: 2: 0 format, encoding is performed in units of macroblocks (MB) each composed of 4 blocks of luminance signal (Y) and 1 block of 2 color difference signals (Cb, Cr). Done. The MB includes an intra MB that is encoded in the screen, a forward prediction MB that creates a motion compensation prediction value by referring to a past encoded picture, and a motion compensation prediction value that refers to a future encoded picture. There are four types of backward prediction MBs to be created and bidirectional prediction MBs to create motion compensation prediction values by referring to past and future coded pictures. In addition, there are three types of pictures: an I picture that can use only an intra MB, a P picture that can use only an intra MB and a forward prediction MB, and a B picture that can use all types of MBs. Since a B picture is encoded after a future I picture or P picture to which it refers, if there is a B picture, the arrangement order and display order of the pictures in the stream are different. A group of pictures starting from an I picture on the stream for about 0.5 to 1 second constitutes a GOP (Group Of Pictures) as a unit of random access or editing.

  Patent Document 1 discloses a technique for determining whether a picture is a scene change by counting the amount of data for each picture of a compressed moving image and comparing the count value with a threshold value. (Claim 7 in Patent Document 1).

  In Patent Document 2, the ratio of the number of intra-coded macroblocks to the number of other predicted macroblocks is calculated for a picture that may be a scene change, and compared with a threshold value. A technique for determining whether or not the picture is a scene change based on the result is disclosed.

  In Patent Document 3, a code amount for each type of picture is obtained, a predetermined calculation is performed on the obtained code amount to calculate a feature amount representing a correlation between two pictures, and a scene change is detected by comparison with a threshold value. Techniques to do this are disclosed.

  Since the scene change picture has no correlation with the immediately preceding picture in the display order, inter-frame prediction is difficult. Therefore, the code of the scene change picture increases or the ratio of the number of predicted MBs changes. Each technique described above uses this point to detect a scene change.

  Here, with reference to FIGS. 7 to 11, scene change detection by the above technique will be described for the frequently used “M = 1” type and “M = 3” type streams. The “M = 1” type stream means a stream having only I and P pictures, and the “M = 3” type stream means that two B pictures are sandwiched between I or P pictures. Means a stream. In each figure, “I”, “B”, and “P” indicate the type (type) of a picture, and the next number of the picture type indicates a number in display order. For the sake of easy understanding, the arrangement order of pictures at the time of display is also shown. In the figure, the shade indicates a picture in which a scene change occurs.

  FIG. 7 shows an example of a stream when the scene change occurs in the P picture following the P picture of the “M = 1” type. As shown in the figure, the stream is composed of an I picture and a P picture, and a scene change occurs at P3 following P2.

  In this case, since the correlation between P2 and P3 is low, P3 in which motion compensation is performed by referring to P2 forward at the time of encoding tends to have a larger intra MB, and a motion vector tends to be larger in the forward prediction MB. The code amount also increases. In such a case, P3 can be detected as a scene change by the technique described above.

  FIG. 8 shows an example of a stream when the scene change occurs in the P picture following the B picture of the “M = 3” type. As shown in the figure, in the stream, two B pictures are sandwiched between I or P pictures, and a scene change occurs at P5 following I2, B0, and B1.

  In this case, intra MB increases in P5 in which motion compensation is performed by referring to I2 forward during encoding. Also, B3 and B4, in which motion compensation is performed by referring forward to I2 and backwardly referring to P5, have fewer backward prediction MBs and bidirectional prediction MBs. Therefore, P5 can be detected as a scene change by the technique described above.

  FIG. 9 shows an example of a stream when the scene change occurs in the B picture following the P picture of the “M = 3” type. As shown in the figure, a scene change occurs in B3 following I2, B0, B1, and P5 in the stream.

  In this case, intra MB increases in P5 in which motion compensation is performed with reference to I2 forward. Also, B3 and B4, which are motion-compensated with forward reference to I2 and backward reference to P5, have fewer forward prediction MBs and bidirectional prediction MBs. Therefore, B3 can be detected as a scene change by the technique described above.

  FIG. 10 shows an example of a stream when the scene change occurs in the B picture following the B picture of the “M = 3” type. As shown in the figure, the stream has a scene change at B4 following I2, B0, B1, P5, and B3.

  Also in this case, intra MB increases in P5 in which motion compensation is performed with reference to I2 forward. Further, B3 in which motion compensation is performed with reference to I2 forward and backward reference to P5 has fewer backward prediction MBs and bi-directional prediction MB, and B4 with motion compensation compensated for forward reference with I2 and backward reference to P5. The forward prediction MB and the bi-directional prediction MB are reduced. Therefore, B4 can be detected as a scene change by the technique described above.

  FIG. 11 shows an example of a stream when the scene change occurs in the I picture following the B picture of the “M = 3” type. As shown in the figure, the stream has a scene change at I8 following I2, B0, B1, P5, B3, and B4.

  In this case, B6 and B7, in which motion compensation is performed with reference to P5 forward and backward reference to I8, have fewer backward prediction MBs and bidirectional prediction MBs. Therefore, I8 can be detected as a scene change by the technique described above.

  Patent Document 4 discloses a feature amount acquisition technique for determining whether still images are similar. This technique generates a reduced image of a still image, performs frequency analysis on the reduced image, and acquires a direct current component and a partial alternating current component as an image feature amount. Also, for moving images, some or all of the frames are extracted from the moving image data to generate respective reduced images, and frequency analysis is performed for each reduced image to obtain a DC component and a partial AC component as frame feature amounts. Are collected, and these frame feature values are collected as feature values of the moving image.

Japanese Patent Laid-Open No. 06-022304 JP 2005-505165 A Japanese Patent Laid-Open No. 10-023421 JP 2000-259832 A

  By the way, with the technique of patent document 1-3, a scene change cannot be detected or it may detect incorrectly.

  For example, as shown in FIG. 12, when a scene change occurs in an I picture (I7 in the figure) in a stream of “M = 1” type, since all macroblocks of I7 are originally intra MBs, The scene change cannot be detected by comparing the encoding type ratio. In addition, since the code amount of I7 and the code amount of the leading I picture of the previous GOP are not necessarily greatly different, there are cases where scene change detection cannot be performed by comparing the code amounts.

  Also, as shown in FIG. 13, when a closed GOP configuration is adopted so as to facilitate editing in an “M = 3” type stream, a B picture (B6) following the first I picture (I8) of the GOP ), There is a restriction that B6 cannot originally refer to the last P picture of the immediately preceding GOP, so that it is not possible to detect a scene change by comparing the ratios of MB coding types. In addition, as shown in FIG. 14, in a stream of “M = 3” type, a scene change occurs in an I picture (I6), and a picture following the I picture is not a B picture but a P picture (P9). Is the same.

  In addition, in a moving image that is difficult to predict by inter-frame motion compensation, such as fade-in, fade-out, cross-fade, and zoom, the number of predicted MBs and the amount of motion vector data vary from normal. For example, the number of intra MBs increases, or the number of MBs that refer to pictures that are distant in time decreases in the B picture. In these cases, there is a possibility that erroneous detection occurs in the technique of Patent Literatures 1-3.

  Further, it is conceivable to use the technique of Patent Document 4 for detecting a scene change of a moving image. For example, a moving image is decoded to obtain a still image (frame), and a DC component obtained by performing frequency analysis on each frame and a part of an AC component are acquired as a frame feature amount, and a feature amount between frames is obtained. A scene change is detected based on the difference between the two.

  However, in the case of the above-described recompression, it is desirable to recompress in the order of pictures in the stream (that is, the encoding order) so as not to change the encoding type of each picture. The images are normally output in the arrangement order in the stream, that is, in the original encoding order, not in the display order. This makes it difficult to detect a scene change in a stream in which a B picture is used. For example, in the stream shown in FIG. 6, a scene change occurs at P5, but when comparing feature quantities in the encoding order, it is expected that the difference between the three feature quantities B1 and P5, P5 and B3, and B4 and I8 will increase. The

  One aspect of the present invention is an image processing apparatus for moving images compressed by a compression method using motion compensation between frames or fields. The image processing apparatus includes a reduced image generation unit, a frequency component acquisition unit, and a scene change degree calculation unit.

  The reduced image generation unit obtains reduced images of a plurality of frames constituting the moving image from the moving image data in the order of encoding.

  The frequency component acquisition unit extracts the frequency components of the plurality of reduced images obtained by the reduced image generation unit.

  The scene change degree calculation unit rearranges the frequency components of the plurality of reduced images in the order of display, and sets the scene change degree indicating the possibility of a scene change for each of two consecutive reduced images or two consecutive reduced images. The difference is obtained for each reduced image of the first I picture of the GOP.

  Note that a representation in which the apparatus of the above aspect is replaced with a method or system, a program that causes a computer to operate as the apparatus, and the like are also effective as an aspect of the present invention.

  According to the technique according to the present invention, a scene change can be reliably detected from a moving image.

It is a figure which shows the video recording apparatus concerning the 1st Embodiment of this invention. It is a figure which shows the scene change detection part in the video recording apparatus shown in FIG. It is a figure for demonstrating the process of the specific example by the scene change detection part shown in FIG. It is a figure which shows the example of the number of various MB calculated | required for every flame | frame with respect to the stream containing a white fade-out and a scene change. It is a figure which shows the difference of the frequency component between the adjacent frames calculated | required with respect to the stream containing a white fade-out and a scene change by the scene change detection part shown in FIG. It is a figure which shows the scene change detection part in the video recording apparatus concerning the 2nd Embodiment of this invention. It is a figure which shows the example of the stream which can detect a scene change by a prior art (the 1). It is a figure which shows the example of the stream which can detect a scene change by a prior art (the 2). It is a figure which shows the example of the stream which can detect a scene change by a prior art (the 3). It is a figure which shows the example of the stream which can detect a scene change by a prior art (the 4). It is a figure which shows the example of the stream which can detect a scene change by a prior art (the 5). It is a figure which shows the example of the stream for which a scene change detection is difficult with the prior art (the 1). It is a figure which shows the example of the stream for which a scene change detection is difficult with the prior art (the 2). It is a figure which shows the example of the stream for which a scene change detection is difficult with the prior art (the 3).

Embodiments of the present invention will be described below with reference to the drawings. For clarity of explanation, the following description and drawings are omitted and simplified as appropriate. Each element described in the drawings as a functional block for performing various processes can be configured by a CPU, a memory, and other circuits in terms of hardware, and a program loaded in the memory in terms of software. Etc.
<First Embodiment>

  FIG. 1 shows a recording apparatus 100 according to a first embodiment of the present invention. The recording apparatus 100 records an MPEG-2 stream of a moving image, here a digital broadcast. Recompress with H.264 method. As shown in FIG. 1, the recording device 100 includes a recording unit 110 and a scene change detection unit 120.

  The recording unit 110 includes an MPEG-2 decoder 112 and an H.264 decoder. H.264 encoder 114 is provided. The MPEG-2 decoder 112 decodes the MPEG-2 stream and decodes the H.264. H.264 encoder 114. H. The H.264 encoder 114 applies H.264 to the moving image output from the MPEG-2 decoder 112. Recompress with H.264 method. H. The H.264 encoder 114 refers to the scene change detection information from the scene change detection unit 120 during recompression.

  The scene change detection unit 120 performs scene change detection for the MPEG-2 stream, and sets the scene change detection information indicating the scene change location (picture) as H.264. H.264 encoder 114.

  FIG. 2 shows the scene change detection unit 120. The scene change detection unit 120 includes a reduced image generation unit 130, a frequency component acquisition unit 150, a scene change degree calculation unit 160, and a scene change determination unit 180.

  The reduced image generation unit 130 partially decodes the MPEG-2 stream, acquires reduced images of a plurality of pictures constituting the moving image represented by the stream, and outputs them to the frequency component acquisition unit 150. The frequency component acquisition unit 150 extracts the frequency components of each reduced image from the reduced image generation unit 130 and outputs them to the scene change degree calculation unit 160. The scene change degree calculation unit 160 rearranges the frequency components of the respective reduced images obtained by the frequency component acquisition unit 150 in the display order, and obtains a difference between the frequency components for every two consecutive images. These differences are output from the scene change degree calculation unit 160 to the scene change determination unit 180 as a scene change degree indicating the possibility of a scene change. The scene change determination unit 180 determines the presence or absence of a scene change by comparing the scene change degree calculated by the scene change degree calculation unit 160 with a predetermined threshold. Specifically, if the degree of scene change is equal to or greater than the above threshold, it is determined that a scene change has occurred, and scene change detection information indicating the corresponding location is stored in H.264. H.264 encoder 114.

The operation of each functional block of the scene change detection unit 120 will be described with reference to the specific example shown in FIG.
The reduced image generation unit 130 includes a partial variable length decoding unit 132, a partial inverse quantization unit 134, an average component restoration unit 136, an adder 138, a selector 140, a prediction pixel creation unit 142, a frame memory 144, a reduction A picture generation unit 146 is provided. The reduced image generation unit 130 operates in the same manner as a normal MPEG-2 decoder except that only the low frequency components of each picture of a moving image represented by the MPEG-2 stream are decoded. explain.

  The partial variable length decoding unit 132 decodes data necessary for decoding low frequency components such as a picture coding type and a quantization matrix from the picture header, and also MB coding type, motion vector, quantization for each MB header. Decode width.

  The partial inverse quantization unit 134 and the average component restoration unit 136 decode the first DCT coefficient for each block. Specifically, in the intra MB, a DC coefficient prediction error that is a frequency (0,0) component is decoded, and in a frequency other than the intra MB, the frequency (0,0) component of the prediction error coefficient is decoded. In addition, in the case of intra MB, a DC coefficient is decoded using a DC coefficient prediction value.

  The adder 138 adds the output of the average component restoration unit 136 and the output of the selector 140 to obtain the average component of the block. The selector 140 outputs an intra prediction value (128 for an 8-bit image) for an intra MB, and outputs a prediction value created by the prediction pixel creation unit 142 for other than the intra MB. That is, the adder 138 adds the DC component from the average component restoration unit 136 and the intra prediction value from the selector 140 to obtain the average component of each block of the intra MB. Except for the intra MB, the frequency (0, 0) component of the prediction error coefficient from the average component restoration unit 136 and the prediction value created by the prediction pixel creation unit 142 from the selector 140 are added to each of the MBs. Get the average component of the block.

  The prediction pixel creation unit 142 performs a motion vector obtained by reducing the motion vector obtained by the partial variable length decoding unit 132 to 1/8 for an MB other than the intra MB, and a forward reference image or a backward reference image stored in the frame memory 144. The predicted value is created using and. The reason why the motion vector is reduced to 1/8 here is to match the block of 8 pixels × 8 lines to 1 pixel consisting of an average component.

  The reduced picture generation unit 146 collects the average components of each block output from the adder 138 and generates a reduced image for each picture. These are reduced images of each picture. Since the block size is 8 pixels × 8 lines, the size of the image created by the reduced picture generation unit 146 is 1/8 × 1/8 when completely decoded. For example, when each picture of the input MPEG-2 stream is 1920 pixels × 1080 lines, the reduced image output from the reduced picture generation unit 146 has a luminance (Y) of 240 pixels × 136 lines, and a color difference (Cb, Cr) is 120 pixels × 68 lines.

  The reduced picture generation unit 146 sequentially outputs the generated reduced images to the frequency component acquisition unit 150. The reduced picture generation unit 146 also outputs reduced images of the I picture and P picture to the frame memory 144. These reduced images are used by the prediction pixel creation unit 142 as a forward reference image or backward reference image of the subsequent P picture or B picture.

  As shown in FIG. 3, the output order of the reduced images from the reduced image generation unit 130 is the same as the arrangement order in the MPEG-2 stream, that is, the encoding order.

  The frequency component acquisition unit 150 includes a blocking unit 152 and a forward DCT conversion unit 154. The frequency component acquisition unit 150 extracts the frequency components of each reduced image from the reduced picture generation unit 146 and supplies the extracted frequency components to the scene change degree calculation unit 160.

  The blocking unit 152 reduces Y, Cb, and Cr of the reduced image into blocks of 8 pixels × 8 pixels, respectively, and outputs them to the forward DCT conversion unit 154. For example, when the input MPEG-2 stream is 1920 pixels × 1080 lines, Y of the reduced image is divided into regions of 30 pixels × 16 lines and the average value of each region is obtained. The average value of each area is obtained by dividing the area into 15 pixels × 8 lines.

  The forward DCT conversion unit 154 performs 8 × 8 DCT conversion, obtains 8 × 8 conversion coefficients (DCT coefficients) for Y, Cb, and Cr for each reduced image, and outputs them to the scene change degree calculation unit 160. . That is, in the present embodiment, frequency component acquisition section 150 acquires a DCT coefficient as a frequency component.

  As shown in FIG. 3, the output order of frequency components from the frequency component acquisition unit 150 is also the encoding order.

  The scene change degree calculation unit 160 includes a buffer 162, a buffer 164, a selector 166, a buffer 168, and a difference calculation unit 170.

  The buffer 162, the buffer 164, and the selector 166 cooperate to rearrange the frequency components output from the frequency component acquisition unit 150 in the display order and output them to the difference calculation unit 170 and the buffer 168. As shown in FIG. 3, each frequency component is output from the selector 166 in the display order.

  The difference calculation unit 170 obtains a difference between the frequency component of the reduced image currently output from the selector 166 and the frequency component of the previous reduced image stored in the buffer 168 and outputs the difference to the scene change determination unit 180. As shown in FIG. 3, the buffer 168 outputs the frequency component of the reduced image immediately before the frequency component of the reduced image currently output by the selector 166.

In the present embodiment, the difference calculation unit 170 weights the frequency component so that the weighting coefficient becomes larger as the frequency is lower, and obtains the difference between the frequency components between the two reduced images. Specifically, the difference is obtained according to the following equation (1). Although there are 64 frequency components, it is possible to reduce the number of frequency components to be stored by setting the weighting coefficient of the high frequency component to 0.
Difference = Σw (m, n) * | Fi (m, n) −Fi (m, n) | (1)
Where Fi (m, n): frequency (m, n) component of the reduced image i
Fj (m, n): Frequency (m, n) component of reduced image j
w (m, n): Weighting factor for frequency w (m, n)

  The scene change determination unit 180 sequentially compares the scene change degree calculated by the scene change degree calculation unit 160 with a threshold value. If the scene change degree is equal to or greater than the threshold value, the scene change determination unit 180 determines that a scene change has occurred and indicates the corresponding part. The scene change detection information is H.264. H.264 encoder 114.

  According to the recording apparatus 100 of the present embodiment, scenes are obtained based on differences in frequency components between two consecutive frames in display order by acquiring frequency components of reduced images generated from moving image data and rearranging them in display order. Because change detection is possible, scene changes can be detected reliably.

  FIG. 4 shows the number of various MBs obtained for each picture with respect to an image stream of a certain 1920 pixels × 1080 lines. In the figure, the horizontal axis indicates picture numbers in display order for easy understanding. In this stream, picture numbers 3, 18, 33, 48, and 51 are “M = 3” type of I picture. Further, in this stream, white fade-out is performed from the 43rd picture to the 53rd picture, and a scene change occurs in the 57th P picture.

  As shown in the figure, the difference in the number of intra MBs, the number of predicted MBs, etc. is unclear between the fade-out part and the part where the scene change has occurred. For example, in the 45th and 54th P-pictures, the number of intra MBs increases to approximately 4000 and 5400, which are the majority, respectively, and may be regarded as a scene change in the prior art. In the 52nd B picture, the number of backward and bidirectional MBs is smaller than usual, and there is a possibility of being regarded as a scene change. In addition, this stream has the forward prediction MB and the bidirectional prediction MB less than usual in the last B picture of GOP as in Nos. 17 and 32, and may be regarded as a scene change.

FIG. 5 shows the frequency component difference obtained by the difference calculation unit 170 as a result of processing the same stream by the recording apparatus 100 of the present embodiment. As can be seen from the figure, the difference in the scene change location has a size that protrudes from the difference calculated for other locations (including the white fade-out location). Therefore, the scene change detection accuracy is high. Also, the white fade-out portion has a larger difference than other portions, can be identified, and can be used as encoded information in recompression.
<Second Embodiment>

  The second embodiment of the present invention is also a recording apparatus. This recording apparatus has the same configuration as the recording apparatus 100 except that the scene change detection unit is different from the scene change detection unit 120 of the recording apparatus 100. Here, only the scene change detection unit 220 in the recording apparatus according to the second embodiment will be described.

  FIG. 6 shows the scene change detection unit 220. In FIG. 6, the same components as those of the scene change detection unit 120 shown in FIG. 2 are assigned the same reference numerals, and description of these components is omitted.

  As shown in FIG. 6, the scene change degree calculation unit 260 of the scene change detection unit 220 is different from the scene change degree calculation unit 160 of the scene change detection unit 120. The scene change degree calculation unit 260 is the same as the scene change degree calculation unit 160 except that a calculator 262 is added between the selector 166 and the buffer 168.

  The calculator 262 receives the output of the selector 166 (frequency component “i” of the current picture) and the output of the buffer 168 (past frequency component “i”), and calculates the average value (frequency component “i”). This is calculated and output to the buffer 168. That is, the calculator 262 and the buffer 168 constitute an infinite impulse response filter in the time direction for each frequency component output from the selector 166. The processing realized by the calculator 262 and the buffer 168 is shown in Expression (2).

Frequency component [i] = frequency component of current picture [i] / 2 + past frequency component [i]) / 2 (2)

  In the recording apparatus 100 according to the first embodiment, the scene change degree calculation unit 160 includes the frequency component (current picture frequency component “i”) of the reduced image currently output from the selector 166 and 1 stored in the buffer 168. The difference from the frequency component of the previous reduced image is obtained. By the way, depending on the stream, the lighting or the color tone may change in a short time due to a change in illumination. In such a stream, the low-frequency component of the frequency component for each picture fluctuates, and there is a possibility that the recording apparatus 100 may erroneously detect a scene change. The recording apparatus according to the second embodiment can cope with such a stream. In the scene change degree calculation unit 260, an infinite impulse in the time direction is applied to each frequency component output by the selector 166. After applying the response filter, the difference is output to the difference calculation unit 170.

  In this way, the difference between the frequency component of the previous picture and the frequency component of the previous picture is obtained instead of the frequency component of the previous picture. The accuracy can be increased.

  The present invention has been described above based on the embodiment. The embodiment is an exemplification, and various modifications and changes may be made to the above-described embodiment without departing from the gist of the present invention. It will be understood by those skilled in the art that modifications in which these changes and increases / decreases are also within the scope of the present invention.

  For example, in a video recording apparatus, a GOP in which a scene change has occurred is detected in order to search for a recorded video scene or to start video editing. Since video editing is often performed in GOP units, it is not always performed in picture units. Therefore, when scene change detection for editing is performed, a reduced image is generated and frequency components are extracted in the same manner as the method of the scene change detection unit 120 or the scene change detection unit 220, and the leading I of two consecutive GOPs is extracted. A GOP in which a scene change has occurred can be detected by obtaining a difference between frequency components between reduced images of a picture and comparing the difference with a threshold value. The reason why the I picture is selected here is that partial decoding is easier than the P picture and B picture.

  The video recording device may decode and reproduce another stream while recording a stream, or may record a plurality of streams. In these cases, all the streams to be recorded are recorded due to performance limitations of the MPEG2 decoder. Sometimes video cannot be decoded, ie not recompressed. In this case, the input stream is recorded once as it is and then recompressed later. Even in such a case, it is convenient if the scene change can be detected in advance for video editing. Therefore, at the time of recording the input stream, scene change detection may be performed in the same manner as the method of the scene change detection unit 120 so that the scene change information is attached to the stream as attached information. The method of the present invention is characterized in that it requires less processing than the MPEG2 decoder and is easy to implement whether it is implemented by hardware or software.

100 Recording Device 110 Recording Unit 112 MPEG-2 Decoder 114 H.264 H.264 encoder 120 Scene change detection unit 130 Reduced image generation unit 132 Partial variable length decoding unit 134 Partial inverse quantization unit 136 Average component restoration unit 138 Adder 140 Selector 142 Predicted pixel creation unit 144 Frame memory 146 Reduced picture generation unit 150 Frequency component Acquisition unit 152 Blocking unit 154 Forward DCT conversion unit 160 Scene change degree calculation unit 162 Buffer 164 Buffer 166 Selector 168 Buffer 170 Difference calculation unit 180 Scene change determination unit 220 Scene change detection unit 260 Scene change degree calculation unit 262 Calculator

Claims (6)

A reduced image generating unit that acquires reduced images of a plurality of pictures constituting the moving image from the moving image data compressed by a compression method using interframe or field motion compensation;
A frequency component acquisition unit that respectively extracts frequency components of the plurality of reduced images obtained by the reduced image generation unit;
The frequency components of the plurality of reduced images are rearranged in the display order, and the scene change degree indicating the magnitude of the possibility of scene change is reduced for each two consecutive reduced images or for the first I picture of two consecutive GOPs. An image processing apparatus comprising: a scene change degree calculation unit that obtains a difference between frequency components acquired by the frequency component acquisition unit for each image.   2. The image according to claim 1, wherein the scene change degree calculation unit obtains the difference after weighting the frequency component acquired by the frequency component acquisition unit so that the weighting coefficient increases as the frequency decreases. Processing equipment.   3. The image processing according to claim 1, wherein the scene chain degree calculation unit obtains the difference after performing infinite impulse response filtering in a time direction on the frequency components rearranged in the display order. apparatus.   4. The image processing according to claim 1, wherein the frequency component acquisition unit performs forward DCT conversion on the plurality of reduced images and acquires a DCT coefficient as a frequency component. 5. apparatus. The moving picture is encoded with the orthogonal transform coefficient of a block in a picture, or the orthogonal transform coefficient of a prediction error block obtained by performing motion compensation between frames or fields with reference to past and / or future pictures. Compressed with the compression method
The image processing apparatus according to claim 4, wherein the reduced image generation unit obtains the reduced image by partially decoding orthogonal transform coefficients of each picture of the moving image.   The scene change determination unit according to any one of claims 1 to 5, further comprising a scene change determination unit that determines that a scene change has occurred when the scene change degree calculated by the scene change degree calculation unit is equal to or greater than a predetermined threshold. The image processing apparatus according to item.

Images