KR100907707B1 - High-speed extraction method of thumbnail images in H.264 / ACC compressed region, apparatus therefor, and recording medium recording the same - Google Patents

Abstract

Disclosed are a high speed extraction method of a thumbnail image in an H.264 / AVC compressed region, an apparatus thereof, and a recording medium recording the same.

The fast extraction method of the thumbnail image in the H.264 / AVC compression region according to the present invention,

Extracting quantized DCT coefficients for each address of a macroblock stored in a bitstream of a video compressed by the H.264 / AVC standard, and generating pixel data in the macroblock using the extracted quantized DCT coefficients Selecting an intra prediction mode for each address of the macro block in which the pixel data is generated according to intra prediction mode information stored in the bitstream of the video, and surrounding macro blocks surrounding the macro block according to the selected intra prediction mode Computing a luminance value of the macro block using the boundary pixel data of, and generating a thumbnail image of a specific video frame from which the DCT coefficient is extracted according to the calculated luminance of the macroblock.

According to the present invention, by calculating a thumbnail image using DCT coefficients and intra prediction information from a video compressed with H.264 / AVC, an operation amount of encoding generated during intra prediction can be reduced, and compression is performed with H.264 / AVC. It is possible to quickly extract the thumbnail image of each frame from the compressed video, and to extract the luminance image using the spatial component information easily without decoding the entire compressed video, and to extract the thumbnail image using the same. It can be used for video analysis and video analysis used for scene change and editing.

Description

Method for high speed extraction of thumbnail images in H.264 / ACC compressed region, apparatus and recording medium recording them {Method and Apparatus for extracting thumbnail video on high speed in H.264 / AVC compressed domain, and Recording Medium using it}

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to video signal processing, and more particularly to a thumbnail of a luminance signal which is information of a frame space region from a bitstream of an H.264 / AVC encoded video in an H.264 compressed region. The present invention relates to a fast extraction method of a thumbnail image in an H.264 / AVC compressed region capable of extracting an image without a decoding process, an apparatus thereof, and a recording medium recording the same.

Currently, thumbnail images are used to provide effective content summaries, storyboards, keyframe generation, program guides, and the like in a personal video recorder (PVR) attached to a digital TV or a video browser used in a video library.

Thumbnail images represent a specific frame of a video smaller than its original size. A thumbnail image summarizes the contents of the recorded video in a video recorder into several thumbnails, creates a keyframe for video indexing, or uses it in a program guide. There are various fields of application such as creating statues.

A method of generating a thumbnail image may be classified into a method of decoding a frame of video encoded by MPEG, and then directly reducing the reconstructed frame, and a method of using a DCT value of an intra (I) picture. In the former case, the entire pixel of the frame must be decoded, which requires a variable length decoder, which is a large hardware burden.

In the latter case, since a thumbnail image is generated using the DCT coefficient of the I picture, a variable decoder is not required. Since the configuration is simple, the latter method is mainly used.

1 is a schematic diagram of extracting a thumbnail image from an H.264 / AVC video bitstream applied to the present invention.

Referring to FIG. 1, first, a real-time feature image is extracted from binary data which is an H.264 / AVC video bitstream. Real-time feature image extracts thumbnail images using DCT coefficients, Intra Prediction Mode, and Macro Block Address obtained from integer conversion, and retrieves images of video data using extracted thumbnail images. Or retrieve video data and analyze the video data.

2 shows the structure of an H.264 / AVC encoder.

Referring to FIG. 2, the H.264 / AVC encoder divides an input video frame into macro blocks (MBs), and each of the divided macro blocks is in an intra mode or an inter mode. ) Is encoded. When compressing an image, each mode is selected and the discrete cosine transform is performed by the discrete cosine transform (DCT) unit 230 and the quantization unit 240. The quantized image is then subjected to an entropy coding unit 290 through the decoder 280 to generate an output image.

Inverse Quantization unit 250 and Inverse Discrete Cosine Transform (IDCT) unit 260 undergoes inverse quantization and inverse discrete cosine transformation to generate reference images. The image of the quantized data is reconstructed and used as a reference image after passing through the loop filter 270.

The motion estimator 210 is used in inter prediction, and outputs a motion vector after passing through the motion compensator 211 comparing the overlap of the reconstructed image and the next input image.

In order to improve the compression efficiency, the image data of the current block is predicted using the neighboring image data. The intra prediction mode selector 220 selects the intra prediction mode, and the intra predictor 221 predicts the intra data. Will go through.

That is, video data is efficiently compressed through motion estimation in inter prediction and motion prediction in intra prediction, and output data is generated through entropy coding.

3 shows Intra Prediction used in H.264 / AVC.

In the H.264 / AVC video compression standard, the intra prediction shown in FIG. 2 can be used to improve the compression efficiency of an intra frame of a video, and the encoding process is also greatly reduced, resulting in superior image quality than the existing video compression standard. It may provide a video that can be restored.

As shown in FIG. 3, intra prediction in the H.264 / AVC compression region may be applied in units of 4 × 4 blocks and 16 × 16 blocks, and 9 × prediction modes are applied to the 4 × 4 blocks and 16 × 16 blocks. Fourteen prediction modes may be applied to the sixteen blocks.

As shown in FIG. 2 and FIG. 3, each coding unit block (4 × 4 block and 16 × 16 block) in the H.264 / AVC encoder has a residual component predicted and remaining with the neighboring pixels of the neighboring block. ) Is encoded by discrete cosine transform.

This fact can make it difficult to obtain luminance information of an image space region using DCT coefficients extracted directly from a video bitstream, as in the previous video compression standard (MPEG-2).

Therefore, conventional H.264 / AVC intra prediction has video compression standards using the prediction modes used for 4x4 blocks and 16x16 blocks, and uses the DCT coefficient extracted directly from the video bitstream. The information cannot be directly obtained, and in decoding H.264 / AVC, since the boundary pixel values of each block must be obtained by decoding the entire video bitstream or performing intra prediction of intra frames, the encoded video There is a problem that it takes time and money to restore information.

Accordingly, a first problem to be solved by the present invention is H.264 / AVC compression, which can generate a thumbnail image of a video frame that is performed for intra prediction while reducing an operation amount of compensation due to intra prediction from an encoded video bitstream. The present invention provides a fast extraction method of thumbnail images in a region.

In addition, a second problem to be solved by the present invention is to provide a fast extraction apparatus of a thumbnail image in the H.264 / AVC compressed region using a fast extraction method of the thumbnail image in the H.264 / AVC compressed region.

In addition, a third problem to be solved by the present invention is to provide a recording medium recorded by a program so that a computer can perform a fast extraction method of a thumbnail image in the H.264 / AVC compression region.

The present invention to solve the first problem,

Extracting quantized DCT coefficients according to addresses of macro blocks stored in a bitstream of a video compressed by the H.264 / AVC standard; Generating pixel data in the macro block using the extracted quantized DCT coefficients; Selecting an intra prediction mode for each address of the macro block in which the pixel data is generated according to the intra prediction mode information stored in the bitstream of the video; Calculating a luminance value of the macro block using boundary pixel data of a neighboring macro block surrounding the macro block according to the selected intra prediction mode; And generating a thumbnail image of a specific video frame from which the DCT coefficients are extracted according to the calculated luminance of the macroblock.

On the other hand, the unit of the macro block is characterized in that the 4 × 4 blocks or 16 × 16 blocks.

The generating of the pixel data in the macro block may include generating specific boundary pixel data in the macro block preset by the user.

Meanwhile, generating the pixel data in the macro block may include performing inverse quantization using the extracted quantized DCT coefficients; And inverse discrete cosine transforming the inverse quantized DCT coefficients to generate boundary pixel data in the macroblock.

The calculating of the luminance value of the macroblock may include: selecting a luminance value equation according to the selected intra prediction mode in a database in which the luminance value equation of the macroblock is stored; And calculating luminance values of the macro blocks by using boundary pixel data of peripheral macro blocks surrounding the macro blocks according to the selected luminance value calculation equation.

The present invention to solve the first problem,

Extracting first quantized DCT coefficients according to addresses of macro blocks stored in a bitstream of a video compressed by the H.264 / AVC standard; Replacing specific boundary pixel data in the macro block preset by the user using the extracted quantized first DCT coefficients with the first DCT coefficients according to the intra prediction mode information stored in the bitstream of the video. Selecting an intra prediction mode for each address of the macro block in which data is generated; Calculating a luminance value of the macro block using boundary pixel data of a neighboring macro block surrounding the macro block according to the selected intra prediction mode; And generating a thumbnail image of a specific video frame from which the first DCT coefficient is extracted according to the calculated macroblock luminance.

In addition, the macro block may be a 4 × 4 block or a 16 × 16 block.

In addition, the first DCT coefficient is a DCT coefficient of the left boundary pixel located at the top of the macro block.

The calculating of the luminance value of the macroblock may include selecting a luminance value equation according to the selected intra prediction mode in a database in which the luminance value equation of the macroblock is stored; And calculating luminance values of the macro blocks by using boundary pixel data of peripheral macro blocks surrounding the macro blocks according to the selected luminance value calculation equation.

The present invention to solve the second problem,

A DCT coefficient extraction unit for extracting quantized DCT coefficients for each address of a macroblock stored in a bitstream of a video compressed by the H.264 / AVC standard; A pixel data generation unit generating pixel data of a pixel previously input from the outside in the macro block using the extracted quantized DCT coefficients; An intra prediction mode selection unit for selecting an intra prediction mode for each address of the macro block in which the pixel data is generated according to intra prediction mode information stored in the bitstream of the video; A calculator configured to calculate a luminance value of the macro block using boundary pixel data of a neighboring macro block surrounding the macro block according to the selected intra prediction mode; And a thumbnail image generator configured to generate a thumbnail image of a specific video frame from which the DCT coefficient is extracted according to the calculated luminance of the macroblock.

The pixel data generation unit may include an inverse quantization unit performing inverse quantization using the quantized DCT coefficients extracted by the DCT coefficient extraction unit; And a boundary pixel data generation unit generating inverse discrete cosine transform of the inverse quantized DCT coefficients to generate boundary pixel data in the macroblock.

The operation unit may include a database module configured to store a luminance value calculation equation of the macro block, and the calculation unit may select a brightness value calculation equation according to the selected intra prediction mode in the database module, and the selected brightness may be determined. The luminance value of the macro block may be calculated using boundary pixel data of a neighboring macro block surrounding the macro block according to a value calculation formula.

In addition, the unit of the macro block is characterized in that the 4 × 4 blocks or 16 × 16 blocks.

In order to solve the third problem, the present invention provides a recording medium recorded by a program to cause a computer to perform a method of extracting thumbnail images in the H.264 / AVC compressed region.

According to the present invention, by calculating a thumbnail image using DCT coefficients and intra prediction information from a video compressed with H.264 / AVC, an operation amount of encoding generated during intra prediction can be reduced, and compression is performed with H.264 / AVC. Quickly extract the thumbnail image of each frame from the compressed video, and extract the thumbnail image using this by easily extracting the luminance component, which is the spatial domain information, without decoding the entire compressed video. There is an effect that can be used for video analysis used for search, transition and editing.

As described above, in intra prediction of H.264 / AVC, each coding unit block (4 × 4 block and 16 × 16 block) is discrete cosine transformed only with residual components predicted and remaining with neighboring pixels of the neighboring block. Since it is difficult to obtain luminance information of an image space region by using DCT coefficients extracted directly from a video bitstream, as in the conventional video compression standard (MPEG-2), since it is difficult to obtain the information, the present invention is the basic data of image analysis and video analysis. Fast extraction method of a thumbnail image extracting the luminance information of the image space region that can be used as an image and extracting the DCT coefficient and the intra prediction mode from the H.264 / AVC video stream to obtain a thumbnail image based on the extracted luminance information, apparatus And a recording medium recording the same.

Equation 1 below illustrates a 4x4 integer transform applied to a residual pixel X on which intra prediction is performed.

는 특정 레시듀얼 픽셀 X의 정 변환 매트릭스(Forward transform matrix)를 의미하고,

는 상기 정 변환 매트릭스

의 트랜스포즈(transpose) 매트릭스이고,

는 스케일링 팩터(Scaling factor)를 의미한다.In Equation 1

Denotes a forward transform matrix of a specific residual pixel X,

Is the positive transformation matrix

Is the transpose matrix of

Denotes a scaling factor.

This Integer Transform has almost the same effect as the Discrete Cosine Transform, but provides an error-free inverse transform by using an integer-based transformation that does not use multiplication.

Meanwhile, as shown in Equation 1, the discrete cosine transformed coefficients are quantized according to Equation 2 below.

는 이산 여현 변환의 계수를 의미하며,

은 양자화 스텝 사이즈를 의미하는데, 상기 양자화 스텝 사이즈는 사용자에 의미 미리 정해지는 상수 값이다.In Equation 2, i and j mean each row and column of the matrix,

Is the coefficient of the discrete cosine transform,

Denotes a quantization step size, and the quantization step size is a predetermined constant value.

In this way, the discrete cosine transformed coefficients of the 4 × 4 unit is expressed by dividing the quantization step. The division process of the fractional unit generated in this process includes a very large amount of computation, and the H.264 / AVC coder quantizes the process as shown in Equation 3 below to reduce the computation.

In Equation 3, the pre-scaling factor (PF) is a scaling constant that stores a matrix component value corresponding to each component in the matrix.

In Equation 3, Pre-Scaling Factor (PF) values are shown in Table 1 below.

은 양자화 스텝 사이즈를 의미하는데, 상기 양자화 스텝 사이즈는 사용자에 의미 미리 정해지는 양자화 상수 값이다.And as mentioned above

Denotes a quantization step size, and the quantization step size is a quantization constant value that is predetermined to a user.

는 하기의 수학식 4로 표현되는 매트릭스를 의미한다.And,

Denotes a matrix represented by Equation 4 below.

Therefore, as described above, the division process of the fractional unit generated in the quantization process includes a very large amount of computation, and the H.264 / AVC coder performs an operation according to Equations 3 and 4 to reduce the amount of computation. do.

That is, scaling constants generated in the DCT transform process are not applied in the transform process, and are included in the quantization process. Therefore, the DCT transform process performs only a simple shift and add operation, so that the transformation can be completed with a very small amount of computation. The rest of the multiplication process is performed in the quantization step of Equations 3 and 4 above.

In addition, in order to further reduce the amount of computation by changing the division process to a shift operation, a process as shown in Equation 5 below is performed.

, where

, where

는 증배율(multiplication factor)을 나타내며, 상기 수학식 5를 이용하여 나눗셈 연산은 미리 정의된

를 이용하여 나눗셈 연산을 2의 멱승 형태로 바꾸어 쉬프트 연산으로 수행 가능하게 바꾸어주며, 이는 매우 적은 연산량을 포함한다.In Equation 5

Denotes a multiplication factor, and the division operation is previously defined using Equation 5.

By converting the division operation into a power of 2, the conversion is made possible by the shift operation, which includes a very small amount of computation.

는 비트스트림으로부터 직접 추출한 DCT 계수를 의미한다. 따라서, 역 양자화된 DCT 계수를 역 이산 여현 변환(Inverse Discrete Cosine Transform) 하기위해서는 상기 수학식 6의

를 연산하여야 한다.In Equation 6

Denotes DCT coefficients extracted directly from the bitstream. Therefore, in order to perform an inverse discrete cosine transform on the inverse quantized DCT coefficients,

Must be calculated.

들은 곱셈 및 나눗셈을 포함하고 있기 때문에, 매우 많은 연산량을 가지고 있으며, 이를 완화하기 위해서 상기 수학식 7의 역변환식내의 프리-스케일링 상수인

들은 상기 수학식 6의 역 양자화 과정에 포함시킬 수 있다.However, pre-scaling constants resulting from inverse transform equations such as

Since they contain multiplication and division, they have a very large amount of computation, and in order to alleviate this, the pre-scaling constant in the inverse

Can be included in the inverse quantization process of Equation 6.

는 역 변환 매트릭스를 의미하고,

는 역 양자화 상수를 의미하며,

는 프리-스케일링 파라미터를 의미하고,

는 정 변환 매트릭스를 의미한다.Here, the equation (7)

Means the inverse transformation matrix,

Means an inverse quantization constant,

Means the pre-scaling parameter,

Denotes a positive transformation matrix.

는 역변환 과정의 프리 스케일링 팩터가 포함된 역 양자화된 DCT 계수들이며, 이러한 식은 여전히 많은 계산량을 포함하고 있다.Meanwhile, in Equation 8 below

Are inverse quantized DCT coefficients with the prescaling factor of the inverse transform process, which still contains a large amount of computation.

In addition, an inverse quantization method finally used by H.264 / AVC in order to further reduce the amount of calculation in Equation 8 is shown in Equation 9 below.

는 역 이산 여현 변환 과정에서 프리-스케일링 상수가 포함된 역양자화 계수를 의미하며,

는 영사의 비트스트림으로부터 직접 추출된 이산 여현 변환 상수를 의미하고,

는 재-스케일링 팩터를 의미하며

는 양자화 파라미터(Quantizer Parameter:QP)를 의미한다.In Equation 9

Denotes the inverse quantization coefficient that includes the pre-scaling constant during the inverse discrete cosine transformation,

Denotes the discrete cosine transform constant extracted directly from the projected bitstream,

Means the re-scaling factor

Denotes a quantization parameter (QP).

의 값은 하기의 표 2와 같다.For the position of each component in the matrix

The value of is shown in Table 2 below.

In general, the H.264 / AVC decoder according to Equation 9 performs inverse quantization using only predetermined constants and QP parameters extracted from the bitstream. In this process, the multiplication operations are expressed as powers of two, so the final multiplication can be solved by the shift operation.

Finally, the H.264 / AVC decoder uses only a core transform in which the pre-scaling constant is excluded in Equation 7 during inverse transformation, and the core DCT is also composed of addition and shift operations. The inverse transform computation amount of the coefficient can be said to be much smaller than the conventional MPEG-2.

4 illustrates pixel data of a unit macro block of a video frame according to the present invention.

Referring to FIG. 4, the present invention extracts a simple thumbnail image from the video compression region without obtaining all the data of each pixel constituting the macroblock as described above.

As such, in order to extract the thumbnail of the video, as illustrated in FIG. 4, eight pixel data per one 4 × 4 coded block should be calculated.

The DCT coefficient represents the average brightness value of the 4x4 block. In the conventional MPEG-2 based technology, it is possible to easily create a thumbnail image in the video compression region using only the DCT coefficient. However, in H.264 / AVC, as shown in FIG. 3, the boundary pixel of the current coding block and neighboring blocks. Because the prediction with, the thumbnail image cannot be extracted using only the DCT coefficients.

Accordingly, in H.264 / AVC, coefficients used for intra prediction, that is, intra prediction, must be calculated as shown in FIG. 4.

, 그리고

,

,

,

,

,

,

의 픽셀에 해당하는 값을 매크로 블록의 경계 픽셀 값이라고 한다.At the location where the DCT value is stored in FIG.

, And

,

,

,

,

,

,

The value corresponding to the pixel of is called the boundary pixel value of the macro block.

5 is a general overview of a thumbnail image extraction method in H.264 / AVC applied to the present invention.

Referring to FIG. 5, first, a quantized DCT coefficient is extracted from a H.264 / AVC video bitstream in zigzag order.

Extract quantized DCT coefficients of 4x4 blocks in zigzag order, extract inverse quantized DCT coefficients, generate IDCT-converted pixel data, and finally compute thumbnail images by calculating luminance of 4x4 macroblocks. Form.

 6 is a flowchart of a thumbnail image extraction method in H.264 / AVC according to an embodiment of the present invention.

Referring to FIG. 6, first, a quantized DCT coefficient is extracted for each address of a macroblock stored in a bitstream of a compressed video according to the H.264 / AVC standard (step 610).

The present invention applies to video compressed according to the H.264 / AVC standard, not to video compressed with existing MPEG codecs, especially MPEG-2.

That is, the bitstream of a video compressed according to the H.264 / AVC standard includes not only the information of the spatial region of the video but also the address of the macroblock, the type of the macroblock, the DCT coefficient, and the intra prediction information of the macroblock.

Accordingly, the quantized DCT coefficients are extracted for each address of the macro block stored in the bitstream of the compressed video according to the H.264 / AVC standard.

Since the intra prediction information according to the H.264 / AVC standard forms a macro block in units of 4 × 4 blocks or 16 × 16 blocks, the macroblock unit according to the present invention consists of 4 × 4 blocks or 16 × 16 blocks. Can lose.

Next, pixel data in the macro block is generated using the extracted quantized DCT coefficients (operation 620).

This may be to generate specific boundary pixel data in the macro block preset by the user.

In the generating of the pixel data in the macro block, inverse quantization is performed using the extracted quantized DCT coefficients, and inverse discrete cosine transforming the inverse quantized DCT coefficients to perform boundary pixel data in the macro block. It may be to include the step of generating.

Since the process 620 is as described above, overlapping descriptions will be avoided.

In the present invention, when generating pixel data in a macro block, not all pixel data in the macro block is calculated, but a value of a specific boundary pixel data is obtained to obtain a luminance value in the macro block by an intra prediction mode.

Therefore, in the present invention, a thumbnail image is extracted by obtaining only specific boundary pixel data from the macroblock shown in FIG.

,

,

,

,

,

,

,

의 픽셀에 해당하는 것 일 수 있다.The particular boundary pixel in the macroblock is located at the location where the DCT value is stored in FIG.

,

,

,

,

,

,

,

It may correspond to the pixel of.

Therefore, instead of obtaining the total pixel value of the macro block as described above, the DCT coefficient value and the eight specific boundary pixel data are obtained, and intra prediction is performed using the same, thereby reducing the amount of computation for calculating the luminance value in the macro block. You can get a thumbnail image.

In operation 630, the intra prediction mode for each address of the macroblock in which the pixel data is generated is selected according to the intra prediction mode information stored in the bitstream of the video.

In the bitstream of a video compressed according to the H.264 / AVC standard according to the present invention, not only the spatial domain information of the video but also the intra prediction mode information for calculating the luminance value of each macro block are stored.

The intra prediction mode information stores a specific mode among the intra prediction modes according to FIG. 3.

That is, in obtaining the luminance value of each macro block, intra prediction should be performed on each macro block. An intra prediction mode for this is stored in a bitstream of a video compressed by the H.264 / AVC standard and transmitted.

Therefore, the luminance value of the macro block is calculated according to the intra prediction mode information stored in the bitstream of the compressed video.

In operation 640, the luminance value of the macro block is calculated using boundary pixel data of a neighboring macro block surrounding the macro block according to the selected intra prediction mode.

In order to obtain the luminance value of the macro block, an expression of a luminance value is selected according to the selected intra prediction mode, and the luminance value of the macro block is calculated according to the expression of the luminance value.

Herein, the expression of the luminance value is databased and stored according to the intra prediction mode. When the intra prediction mode is selected, the expression of the databased luminance value is selected according to the selected intra prediction mode and surrounds the macro block. The luminance value of the macroblock is computed using the boundary pixel data of the neighboring macroblock.

Here, the boundary pixel data of the neighboring macroblock means the specific boundary pixel data value of FIG. 4 described above.

Finally, a thumbnail image of a specific video frame from which the DCT coefficient is extracted is generated according to the luminance of the macroblock (step 650).

As described above, when the luminance value of the macro block is calculated by using the specific boundary pixel data, the intra prediction mode, and the luminance value calculation equation, a thumbnail image of the specific video frame is synthesized, and based on this, the basis of image search and video analysis is performed. Can be used as a resource.

7 is a flowchart of a method of extracting a thumbnail image in H.264 / AVC according to another embodiment of the present invention.

First, a quantized first DCT coefficient is extracted for each address of a macroblock stored in a bitstream of a video compressed by the H.264 / AVC standard (step 710).

According to another embodiment of the present invention, in order to reduce the amount of calculation of the compensation using intra prediction to the maximum, instead of calculating all boundary pixel values of each block, the specific boundary pixel value of the corresponding macroblock is replaced with the first DCT value. It is.

의 값으로 할 수 있다.Meanwhile, referring to FIG. 4, the first DCT value is a DCT coefficient of the left boundary pixel located at the top of the macro block.

Can be set to

Similarly, since the intra prediction information according to the H.264 / AVC standard forms a macroblock in units of 4x4 blocks or 16x16 blocks, the macroblock units in the present invention are 4x4 blocks or 16x16 blocks. Can be made.

Then, using the extracted quantized first DCT coefficients, the specific boundary pixel data in the macro block preset by the user is replaced with the first DCT coefficients (step 720), and the intra stored in the bitstream of the video. In operation 730, an intra prediction mode for each address of the macroblock in which the pixel data is generated is selected according to prediction mode information.

In operation 740, the luminance value of the macroblock is calculated using boundary pixel data of a neighboring macroblock surrounding the macroblock according to the selected intra prediction mode.

In order to obtain the luminance value of the macro block, an expression of a luminance value is selected according to the selected intra prediction mode, and the luminance value of the macro block is calculated according to the expression of the luminance value.

Herein, the expression of the luminance value is databased and stored according to the intra prediction mode. When the intra prediction mode is selected, the expression of the databased luminance value is selected according to the selected intra prediction mode and surrounds the macro block. The luminance value of the macroblock is computed using the boundary pixel data of the neighboring macroblock.

Here, the boundary pixel data of the neighboring macroblock means the specific boundary pixel data value of FIG. 4 described above.

Finally, in operation 750, a thumbnail image of a specific video frame from which the first DCT coefficient is extracted is generated according to the luminance of the macroblock.

8A to 8I illustrate a pixel compensation relation for each pixel according to an intra prediction mode of a macroblock according to the present invention.

Referring to FIG. 8A, the pixel value of each pixel of the macro block of FIG. 8A is replaced with the boundary pixel value of the upper macro block in contact with the macro block.

That is, in the macro block to be obtained, the pixel value of the upper macro block may be replaced in units of columns. The reason why the pixel value of the macroblock can be replaced by the pixel value of the surrounding macroblock is that the difference in pixel values between adjacent macroblocks in the video frame is weak. This can replace the pixel values of the macro block.

Referring to FIG. 8B, the pixel value of each pixel of the macro block of FIG. 8B is replaced with the boundary pixel value of the macro block on the left side which contacts the macro block.

Referring to FIG. 8C, a pixel value of each pixel of the macroblock of FIG. 8C may be replaced with an average value of boundary pixel values of upper and left macroblocks contacting the macroblock.

8D to 8I, pixel values of each pixel of the macroblock of FIGS. 8D to 8I are calculated according to the pixel compensation relation shown in the drawing according to each prediction mode. Since the pixel compensation relation according to the illustrated intra prediction mode is already standardized, the normalized pixel compensation relation is stored, and the pixel value of the macro block is calculated accordingly.

The pixel compensation relation according to FIGS. 8A to 8I is to obtain the entire pixel value of a macro block, particularly a 4x4 block. Replace with to minimize the amount of computation.

FIG. 9A illustrates a process of obtaining a luminance value equation of the present invention to which the pixel compensation relation of FIG. 8D is applied.

Although the DCT value of the boundary pixel component is used in the process of calculating the luminance value of the present macroblock described above, the luminance value of the macroblock using the pixel value of the boundary pixel component rather than the DCT value of the boundary pixel component. Can be calculated, which is the same as the luminance value calculation process of the macro block described above.

For convenience of description, the method of using the DCT value of the boundary pixel component in the process of calculating the luminance value of the macroblock will be described in detail.

,

,

,

는 윗 블록의 경계 픽셀 성분의 DCT 값을 나타내며,

,

,

,

는 윗 우측블록의 경계 픽셀 성분의 DCT 값을 나타낸다.According to FIG. 9A, according to intra prediction mode 3

,

,

,

Denotes the DCT value of the boundary pixel component of the upper block,

,

,

,

Denotes the DCT value of the boundary pixel component of the upper right block.

The luminance value of the macroblock is calculated by applying the DCT value of the boundary pixel and the pixel compensation relation shown in FIG. 8D.

은 1개 존재하고, 이를 픽셀 보상 관계식에 대비하면, 상단 매크로 블록에 존재하는 픽셀에 있어서, 픽셀 보상 관계식의

성분이 1개,

성분이 1개,

성분이 1개,

성분이 1개 존재한다.Looking in detail, in the macro block shown in Figure 9a according to the present invention

Is one, and in contrast to the pixel compensation relation, for pixels present in the upper macro block, the pixel compensation relation

1 ingredient,

1 ingredient,

1 ingredient,

There is one component.

는 본 매크로 블록에서 2개 존재하고, 이를 픽셀 보상 관계식에 대비하면, 상단 매크로 블록에 존재하는 픽셀에 있어서, 픽셀 보상 관계식의

성분이 1개,

성분이 2개,

성분이 1개 존재하여

는 본 매크로 블록에서 2개 존재하므로 이를 곱하여 총 8개의 성분이 존재하게 된다.Likewise,

Are two in this macroblock, and in contrast to the pixel compensation relation, for pixels present in the upper macroblock,

1 ingredient,

2 ingredients,

There is 1 ingredient

Since there are two in this macroblock, there are eight components by multiplying them.

Similarly, the same operation as described above results in 64 component values used for the luminance value calculation of the present macroblock, 25 components of the upper block, and 39 components of the upper right block. When the average value is multiplied by the representative DCT value of the block, the luminance value of the present macro block is generated.

FIG. 9B illustrates a process of obtaining a luminance value equation of the present invention to which the pixel compensation relation of FIG. 8E is applied.

,

,

,

는 윗 블록의 경계 픽셀의 DCT 값을 나타내며,

,

,

,

은 좌측블록의 경계 픽셀의 DC 값을 나타내고,

는 상단 좌측 블록의 경계 픽셀의 DC 값을 나타낸다.According to FIG. 9B, according to intra prediction mode 4

,

,

,

Denotes the DCT value of the boundary pixel of the upper block,

,

,

,

Denotes the DC value of the boundary pixel of the left block,

Denotes the DC value of the boundary pixel of the upper left block.

The luminance value of the macroblock is calculated by applying the DCT value of the boundary pixel and the pixel compensation relation shown in FIG. 8E.

Similarly, the same operation as described above results in 61 component values used in the luminance value calculation of the present macroblock, 25 components in the upper block, 14 components in the upper left block, and left block. Since there are 22 components, the average value is multiplied by the representative DCT value of each macro block to generate the luminance value of the macro block as shown in FIG. 8B.

10 is a block diagram of an apparatus for extracting a thumbnail image in an H.264 / AVC compressed region according to the present invention.

Referring to FIG. 10, the apparatus for extracting a thumbnail image in the H.264 / AVC compressed region according to the present invention includes a DCT coefficient extractor 1010, a pixel data generator 1020, an intra prediction mode selector 1030, and an operation unit. 1040 and the thumbnail image generator 1050.

 In the H.264 / AVC compression region, the thumbnail image extracting apparatus in the DCT coefficient extractor 1010 extracts the quantized DCT coefficients for each macro block address stored in the bitstream of the compressed video according to the H.264 / AVC standard. .

That is, the bitstream of a video compressed according to the H.264 / AVC standard includes not only the information of the spatial region of the video but also the address of the macroblock, the type of the macroblock, the DCT coefficient, and the intra prediction information of the macroblock.

Accordingly, the DCT coefficient extractor 1010 extracts the quantized DCT coefficients according to addresses of macro blocks stored in the bitstream of the compressed video according to the H.264 / AVC standard.

Since the intra prediction information according to the H.264 / AVC standard forms a macroblock in units of 4 × 4 blocks or 16 × 16 blocks, the macroblock units in the present invention consist of 4 × 4 blocks or 16 × 16 blocks. Can be.

The pixel data generator 1020 generates pixel data in the macro block by using the extracted quantized DCT coefficients.

This may be to generate specific boundary pixel data in the macro block preset by the user.

Meanwhile, the pixel data generator 1020 performs an inverse discrete cosine of the inverse quantization unit 1021 that performs inverse quantization using the extracted quantized DCT coefficients, and the inverse quantized DCT coefficients of the inverse quantization unit 1020. It may include a boundary pixel data generation unit 1022 for converting to generate the boundary pixel data in the macro block.

In the present invention, when generating pixel data in a macro block, not all pixel data in the macro block is calculated, but a value of a specific boundary pixel data is obtained to obtain a luminance value in the macro block by an intra prediction mode.

Therefore, in the present invention, a thumbnail image is extracted by obtaining only specific boundary pixel data from the macroblock shown in FIG.

,

,

,

,

,

,

,

의 픽셀에 해당하는 것 일 수 있다.The particular boundary pixel in the macroblock is located at the location where the DCT value is stored in FIG.

,

,

,

,

,

,

,

It may correspond to the pixel of.

Therefore, instead of obtaining the total pixel value of the macro block as described above, the DCT coefficient value and the eight specific boundary pixel data are obtained, and intra prediction is performed using the same, thereby reducing the amount of computation for calculating the luminance value in the macro block. You can get a thumbnail image.

The intra prediction mode for each address of the macroblock in which the pixel data is generated is selected according to the intra prediction mode information stored in the bitstream of the video.

In the bitstream of a video compressed according to the H.264 / AVC standard according to the present invention, not only the spatial domain information of the video but also the intra prediction mode information for calculating the luminance value of each macro block are stored.

The intra prediction mode information stores a specific mode among the intra prediction modes according to FIG. 3.

That is, in obtaining the luminance value of each macro block, intra prediction should be performed on each macro block. An intra prediction mode for this is stored in a bitstream of a video compressed by the H.264 / AVC standard and transmitted.

Therefore, the luminance value of the macro block is calculated according to the intra prediction mode information stored in the bitstream of the compressed video.

An intra prediction mode selector 1030 selects an intra prediction mode for each address of the macroblock in which the pixel data is generated according to the intra prediction mode information stored in the bitstream of the video.

Then, the calculator 1040 calculates a luminance value of the macro block using boundary pixel data of a neighboring macro block surrounding the macro block according to the intra prediction mode selected by the intra prediction mode selector 1030.

In order to obtain the luminance value of the macro block, an expression of a luminance value is selected according to the selected intra prediction mode, and the luminance value of the macro block is calculated according to the expression of the luminance value.

Here, the expression of the luminance value may be databased according to an intra prediction mode and stored in a database module (not shown). When the intra prediction mode is selected, a database module (not shown) according to the selected intra prediction mode is selected. Selects an expression of a luminance value stored in the macroblock, and uses a boundary pixel data of a neighboring macroblock surrounding the macroblock in accordance with an expression of the selected luminance value in a database module (not shown). Can be calculated.

Here, the boundary pixel data of the neighboring macroblock means the specific boundary pixel data value of FIG. 4 described above.

Then, the thumbnail image generator 1050 generates a thumbnail image of a specific video frame from which the DCT coefficient is extracted according to the calculated luminance of the macroblock.

As described above, when the luminance value of the macro block is calculated by using the specific boundary pixel data, the intra prediction mode, and the luminance value calculation equation, a thumbnail image of the specific video frame is synthesized, and based on this, the basis of image search and video analysis is performed. Can be used as a resource.

The invention can be implemented via software. When implemented in software, the constituent means of the present invention are code segments that perform the necessary work. The program or code segments may be stored on a processor readable medium or transmitted by a computer data signal coupled with a carrier on a transmission medium or network.

The computer-readable recording medium includes all kinds of recording devices in which data is stored which can be read by a computer system. Examples of computer-readable recording devices include ROM, RAM, CD-ROM, DVD ± ROM, DVD-RAM, magnetic tape, floppy disks, hard disks, optical data storage devices, and the like. The computer readable recording medium can also be distributed over network coupled computer devices so that the computer readable code is stored and executed in a distributed fashion.

Although the present invention has been described with reference to one embodiment shown in the drawings, this is merely exemplary and will be understood by those of ordinary skill in the art that various modifications and variations can be made therefrom. However, such modifications should be considered to be within the technical protection scope of the present invention. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

1 is a schematic diagram of extracting a thumbnail image from an H.264 / AVC video bitstream applied to the present invention.

2 shows the structure of an H.264 / AVC encoder.

3 shows Intra Prediction used in H.264 / AVC.

4 illustrates pixel data of a unit macro block of a video frame according to the present invention.

5 is a general overview of a thumbnail image extraction method in H.264 / AVC applied to the present invention.

 6 is a flowchart of a fast extraction method of a thumbnail image in H.264 / AVC according to an embodiment of the present invention.

7 is a flowchart illustrating a fast extraction method of a thumbnail image in H.264 / AVC according to another embodiment of the present invention.

8A to 8I illustrate a pixel compensation relation for each pixel according to an intra prediction mode of a macroblock according to the present invention.

FIG. 9A illustrates a process of obtaining a luminance value equation of the present invention to which the pixel compensation relation of FIG. 8D is applied.

FIG. 9B illustrates a process of obtaining a luminance value equation of the present invention to which the pixel compensation relation of FIG. 8E is applied.

10 is a block diagram of a fast extraction apparatus of a thumbnail image in an H.264 / AVC compressed region according to the present invention.

Claims (14)

Extracting quantized DCT coefficients for each address of a macroblock stored in a bitstream of a compressed video according to the H.264 / AVC standard; Generating pixel data in the macro block using the extracted quantized DCT coefficients; Selecting an intra prediction mode for each address of the macro block in which the pixel data is generated according to the intra prediction mode information stored in the bitstream of the video; Calculating a luminance value of the macro block using boundary pixel data of a neighboring macro block surrounding the macro block according to the selected intra prediction mode; And And generating a thumbnail image of a specific video frame from which the DCT coefficient is extracted according to the calculated macroblock luminance. The method of claim 1, The unit of the macro block is 4 × 4 blocks or 16 × 16 blocks, characterized in that the fast extraction method of the thumbnail image in the H.264 / AVC compressed region. The method of claim 1, Generating pixel data in the macro block And generating specific boundary pixel data in the macro block preset by the user. The method of claim 3, wherein Generating pixel data in the macro block may include: Performing inverse quantization using the extracted quantized DCT coefficients; And And inverse discrete cosine transforming the inverse quantized DCT coefficients to generate boundary pixel data in the macro block. The method of claim 1, Computing the luminance value of the macro block Selecting a luminance value equation according to the selected intra prediction mode from a database in which the luminance value equation of the macro block is stored; And And calculating luminance values of the macro blocks using boundary pixel data of surrounding macro blocks surrounding the macro blocks according to the selected luminance value calculation equation. Fast extraction method of thumbnail images. Extracting first quantized DCT coefficients according to addresses of macro blocks stored in a bitstream of a video compressed by the H.264 / AVC standard; Replacing specific boundary pixel data in the macro block preset by a user with the first DCT coefficients using the extracted quantized first DCT coefficients; Selecting an intra prediction mode for each address of the macro block in which the pixel data is generated according to the intra prediction mode information stored in the bitstream of the video; Calculating a luminance value of the macro block using boundary pixel data of a neighboring macro block surrounding the macro block according to the selected intra prediction mode; And And generating a thumbnail image of a specific video frame from which the first DCT coefficient is extracted according to the luminance of the calculated macroblock. The method of claim 6, The unit of the macro block is 4 × 4 blocks or 16 × 16 blocks, characterized in that the fast extraction method of the thumbnail image in the H.264 / AVC compressed region. The method of claim 6, The first DCT coefficient is And a DCT coefficient of a left boundary pixel located at the top of the macro block. The method of claim 6, Computing the luminance value of the macro block Selecting a luminance value equation according to the selected intra prediction mode from a database in which the luminance value equation of the macro block is stored; And And calculating luminance values of the macro blocks using boundary pixel data of surrounding macro blocks surrounding the macro blocks according to the selected luminance value calculation equation. Fast extraction method of thumbnail images. 10. A recording medium recorded with a program for executing the method of any one of claims 1 to 9 on a computer. A DCT coefficient extracting unit for extracting quantized DCT coefficients according to addresses of macro blocks stored in a bitstream of a compressed video according to the H.264 / AVC standard; A pixel data generation unit generating pixel data of a pixel previously input from the outside in the macro block using the extracted quantized DCT coefficients; An intra prediction mode selection unit for selecting an intra prediction mode for each address of the macro block in which the pixel data is generated according to intra prediction mode information stored in the bitstream of the video; A calculator configured to calculate a luminance value of the macro block using boundary pixel data of a neighboring macro block surrounding the macro block according to the selected intra prediction mode; And And a thumbnail image generator for generating a thumbnail image of a specific video frame from which the DCT coefficient is extracted according to the luminance of the calculated macroblock. The method of claim 11, The pixel data generator, An inverse quantizer for performing inverse quantization using the quantized DCT coefficients extracted by the DCT coefficient extractor; And And a boundary pixel data generator configured to inverse discrete cosine transform the inverse quantized DCT coefficients to generate boundary pixel data in the macroblock. The method of claim 11, The calculation unit A database module for storing a luminance value calculation expression of the macro block; The operation unit selects a luminance value calculation expression according to the selected intra prediction mode in the database module, and uses the boundary pixel data of the neighboring macroblock surrounding the macroblock according to the selected luminance value calculation expression. A high speed extraction device of a thumbnail image in a H.264 / AVC compressed region, characterized in that to calculate a luminance value of. The method of claim 11, The unit of the macro block is a 4 × 4 block or 16 × 16 block, the fast image extraction apparatus of the thumbnail image in the H.264 / AVC compressed region.

Images