KR100931269B1 - Real-Time Edge Detection in H.264 / ACC Compression Domain - Google Patents

Abstract

The present invention relates to a real-time edge detection method in an H.264 / AVC compressed region. More particularly, the present invention provides a method for detecting an edge of a spatial region in real time in a compressed state without decoding a video compressed with H.264 / AVC. By detecting, the present invention relates to a real-time edge detection method that can be applied to video analysis such as video search and scene change in a compressed region.

for teeth,

Receiving an image compressed with H.264 / AVC;

Detecting prediction information of an image using intra prediction mode information in the bitstream of the image;

It provides a real-time edge detection method in the H.264 / AVC compressed region comprising the step of analyzing the image using the detected prediction information.

H.264 / AVC, Video Compression, MPEG, Encoding, Decoding, Intra, Movie, Edge, Intra Prediction.

Description

Method for Real-time Edge Detection in H.264 / ACC Compression Domain {Method For Real-time Edge Extraction In H.264 / AVC Compression Domain}

1 is an encoder block diagram of H.264 / AVC,

2 shows possible intra prediction modes for 4 × 4 blocks in H.264 / AVC,

3 shows possible intra prediction modes for 4 × 4 blocks in H.264 / AVC,

4 shows a possible intra prediction mode for 16 × 16 blocks in H.264 / AVC,

5 is a diagram showing the spatial characteristics of intra prediction in H.264 / AVC,

6 is a diagram illustrating an embodiment of extracting an intra prediction mode according to the present invention;

7 is a diagram illustrating an embodiment of filtering edges extracted using DCT residual according to the present invention;

8 is a view showing an embodiment of extracting a representative component from the edge extracted in accordance with the present invention,

9 is a diagram illustrating an embodiment of dividing an entire image into 16 sub-images.

The present invention relates to a real-time edge detection method in an H.264 / AVC compressed region. More particularly, the present invention provides a method for detecting an edge of a spatial region in real time in a compressed state without decoding a video compressed with H.264 / AVC. By detecting, the present invention relates to a real-time edge detection method that can be applied to video analysis such as video search and scene change in a compressed region.

Recently, due to the rapid development of mobile communication and satellite communication, the role of the wireless communication service becomes more important in the information society, and the multimedia wireless communication service capable of wirelessly accessing the Internet or video communication as well as the transmission of conventional voice or text information It is spreading. In particular, in the 4th generation mobile communication using IMT-2000 business and satellite digital multimedia broadcasting (DMB) system, an environment capable of transmitting high quality video in real time is being established.

The above-mentioned technology has been commercialized, among other things, by digitally encoding an analog video signal by quantization or variable-length encoding, and then transmitting it as a digital signal, which is then decoded by the received terminal. This is made possible by the rapid transmission speed and the development of video compression technology capable of transmitting abundant information. In other words, the characteristics of digital broadcasting enable efficient service in a limited transmission path by digitizing and compressing moving picture information. Moving picture compression technology is recognized as an important technology that determines the nature and quality of service.

Various compression technologies have been developed for storing and transmitting huge amounts of information. In particular, since the late 1980s, the development of technology has been accelerated as the demand for encoding and specification of digital video information has been proposed. Accordingly, the International Telecommunication Union (ITU) enacted H.261 and H.263 as standards for video services in wired and wireless network environments, and the International Organization for Standardization (ISO) also adopted MPEG-1, MPEG-2, International standardization discussions have been actively conducted, including the provision of MPEG-4. Since the development of the H.263 + and MPEG-4 standards, wireless communication has proliferated rapidly. Accordingly, there is a need for a video compression technology specification that provides improved compression efficiency and accommodates various communication environments compared to conventional compression methods. It has emerged.

Subsequently, the Joint Video Team (JVT) jointly decided by the International Telecommunication Union (ITU) and the International Organization for Standardization and the International Electrotechnical Commission (ISO / IEC) provided H.264 (MPEG-4 part) with higher compression efficiency than conventional methods. 10, hereinafter referred to as H.264). H.264 is the standard video compression technology of digital broadcasting. There are many advances compared to existing technical standards such as H263 + and MPEG-2 / 4 in terms of flexibility and video coding efficiency that can easily accommodate various network environments. In other words, H.264 adopts Hybrid Motion Compensated Prediction (MCP) model like existing standard technologies, but has 50% compression efficiency compared to H.263 + or MPEG-4 (part2), and continuously high quality video. To ensure transmission. In addition, H.264 has the advantages of excellent packet loss in packet networks and bit error recovery in a wireless network, and easy transmission in different networks through a network application layer.

1 is a block diagram showing the configuration of an H.264 encoder. In the H.264 international standard, video signals are classified into intra coding using spatial similarity and inter coding using video frame similarity according to temporal differences.

Referring to FIG. 1, the intra prediction unit performs prediction by using pixel data of a block adjacent to a block to be predicted in a picture in which decoding has already been performed, and in inter prediction, decoding is already performed and deblocking filtering is performed. Block prediction of the current picture is performed using the reference picture stored in the buffer. The prediction sample obtained through the above process is transformed, quantized and compressed. The entropy coding unit performs encoding on the quantized video data according to a predetermined method to form a bit stream and transmits the bit stream to the NAL.

In the encoding method, encoding is performed on 8x8 blocks in units of macroblocks of 16x16 pixels in H263 + and MPEG-2 / 4, whereas encoding is performed on 16x16 macroblocks and 4x4 subblocks in H.264. do. The intra coding method is a method of increasing efficiency by obtaining and encoding a difference value (SAD: Sum of Absolute Difference) between a prediction value obtained due to directionality of adjacent pixels with respect to 16x16 macroblock and 4x4 blocks. For 4x4 blocks, encoding is performed in nine different modes, and for 16x16 macroblocks, encoding is performed in four modes on a block basis. That is, by using a plurality of prediction modes in the spatial domain, the compression efficiency is improved by minimizing the residual signal that is the actual encoding target.

FIG. 2 illustrates nine prediction modes of intra prediction coding on luminance components of a 4x4 block, FIG. 3 illustrates an intra prediction method for a 4x4 block, and FIG. 4 illustrates an intra prediction method for a 16x16 macroblock. The figure which shows is shown.

Referring to FIG. 2, there are nine types of intra prediction coding modes, from mode 0 to mode 8, in a 4x4 block. Mode 2 is a directivity DC mode, which is not shown in FIG. 2. In more detail with reference to FIG. 3, intra coding of a 4x4 block generates a prediction block using pixels around the target block, and obtains a sum of absolute difference (SAD) between each prediction block and the original block. The mode having the smallest SAD among the branch modes is selected as the optimal prediction mode.

Mode 0 is a vertical prediction mode, in which the upper four pixels are projected in the vertical direction to predict pixel values of the pixels included in the block. Similarly, mode 1 is a horizontal prediction. Mode, Mode 2 is the DC mode predicting the average value because there is no direction, Mode 3 is the prediction mode of the left diagonal direction (diag_down_left), Mode 4 is the prediction mode of the right diagonal direction (diag_down_right), Mode 5 is the right vertical direction (vertical_right) Prediction mode of, mode 6 is the horizontal down direction (horizontal_down) prediction mode, mode 7 is the left vertical direction (vertical_left) prediction mode, mode 8 is the horizontal up direction (horizontal_up) prediction mode, respectively.

Referring to FIG. 4, the intra prediction method for the 16x16 macroblock has four modes, and mode 0 performs intra prediction encoding by projecting the upper 16 pixels in the vertical direction. Similarly, mode 1 is the prediction mode in the horizontal direction. The mode 2 is a DC mode and the mode 3 is a planar mode in which intra prediction encoding is performed on the left 16 pixels and the upper 16 pixels in a predetermined combination.

As described above, H.264 removes spatial redundancy by using a feature that spatial prediction in intra prediction coding has not been found in a conventional video compression technique, and that a current block has a spatial similarity to neighboring blocks in the same image. In other words, the prediction block of the current block is generated from the neighboring pixels of each block, and only the difference with the current block is encoded, thereby reducing the bit amount.

The present invention has been made in view of the above, and extracts the encoded intra prediction mode from the H.264 / AVC compressed bit stream by using the relationship between the intra prediction mode and the spatial information, and then extracts the spatial edge distribution of the shape. By deriving, an object of the present invention is to provide a real-time edge detection method in an H.264 / AVC compressed region that can analyze a compressed video in an undecoded state.

As described above, the real time edge detection method in the H.264 / AVC compressed region includes

(a) receiving an image compressed with H.264 / AVC;

(b) detecting prediction information of an image using intra prediction mode information in the bitstream of the image; And

and (c) analyzing the image using the detected prediction information.

In particular, step (b),

And converting a part of the prediction information of the detected image into DC prediction by using the number of encoded bits of the DCT residual.

에 의해 결정되며, 상기의 P_{i ,j}는 영상 가로 방향으로 i 번째, 그리고 영상 세로 방향으로 j번째 4X4 블록의 예측 모드를 나타내며, dc는 dc 예측을 의미하고, Dth는 스레숄드(threshold)를, Lr은 DCT 나머지(residual)의 비트수를 나타내는 것을 특징으로 한다.In addition, converting a part of the prediction information of the detected detected image to DC prediction,

P _{i , j} denotes the prediction mode of the i-th 4X4 block in the image horizontal direction and the j-th direction in the image vertical direction, dc means dc prediction, and Dth denotes a threshold. Lr is characterized in that it represents the number of bits of the DCT residual.

Further, after step (b) and before step (c),

The whole image is divided into k non-overlapping k sub-images, and the vertical (HertEdge), horizontal (HoriEdge), 45 degree (45 Edge), and sub-images are used for each sub-image using the detected prediction information included in each sub-image. And extracting five edge components of 135 degrees (135 edge) and non-directional (NonEdge).

In addition, extracting the five edge components using the detected prediction information,

edge _k ^v = a * n (vertical _k ) + b * n (vertical_right _k ) + c * n (vertical_left _k )

edge _k ^h = d * n (horizontal _k ) + e * n (horizontal_down _k ) + f * n (horizontal_up _k )

edge _k ⁴⁵ = g * n (diag_down_left _k ) + h * n (vertical_left _k ) + i * n (horizontal_up _k )

edge _k ¹³⁵ = j * n (diag_down_right _k ) + k * n (vertical_right _k ) + l * n (horizontal_down _k )

edge _k ^nondir = LinearCombination (n (vertical _k ), n (horizontal _k ), n (dc _k ), n (diag_down_right _k ), n (diag_down_right _k ), n (vertical_right _k ), n (horizontal_down _k ), n ( vertical_left _k ), n (horizontal_up _k ))

The edge _k ^x Is the x-edge component value of the k-th sub-image, a to l are weighted as any constant, n (Y _k ) is the number of Y component edges in the k-th sub-image, and LinearCombination () is parentheses And a linear combination of eye variables.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. A singular expression includes a plural expression unless the context clearly indicates otherwise. In this application, the terms “comprises” or “having” are intended to indicate that there is a feature, number, step, action, component, part, or combination thereof described in the specification, and one or more other It is to be understood that the present invention does not exclude the possibility of the presence or the addition of features, numbers, steps, operations, components, parts, or a combination thereof.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

5 shows the spatial feature relationship between the intra prediction mode applied to the 4 × 4 unit block and the 4 × 4 unit block. As illustrated in FIG. 5, it can be seen that the direction (edge direction) of the unit block of the intra frame to which the intra prediction mode is applied is similar to the prediction mode applied to the block.

Hereinafter, a method of extracting the encoded intra prediction mode from the H.264 / AVC compressed bit stream using the relationship between the intra prediction mode and the frame space information to derive the spatial edge distribution of the shape will be described.

6 is a diagram illustrating an intra prediction mode extracted from an H.264 / AVC bitstream and a spatial edge distribution relationship.

의 범위를 갖고, 상기의 X는 영상의 가로크기를, Y는 영상 세로크기를 의미한다.Intra prediction mode extracted directly from the compressed bitstream is defined as P _{i , j} , where i denotes an index of 4 × 4 blocks in the horizontal direction of the image, j denotes an index of 4 × 4 blocks in the vertical direction of the image, and thus i, j Is

X is the horizontal size of the image, Y is the vertical length of the image.

P _{i , j} denotes block prediction information, and since 9 prediction modes exist in 4 × 4 blocks in H.264 / AVC, P _ij is vertical, horizontal, dc, diag_down_left, diag_down_right, vertical_right, horizontal_down, vertical_left, It becomes one of horizontal_up.

On the other hand, since the 16X16 prediction mode is performed on a very large portion of the DC component of the image, in the present invention, all 16 4X4 blocks in the corresponding 16X16 block are classified as dc.

The white line of FIG. 6 represents the boundary of the 4 × 4 block, and each number within the boundary represents the length in the bitstream state of the quantized frequency coefficient.

That is, 4x4 blocks of H.264 / AVC are Discrete Cosine Transform (DCT), quantized, and entropy coded (CAVLC). The length in the bitstream state of the quantized frequency coefficient is shown in FIG. 6.

The numbers in the red circle generally have a small value, which shows that the DCT coefficients of the red circle are encoded with a small bit length in the bitstream.

As shown in the red circle of FIG. 6, the intra prediction mode extracted directly from the H.264 / AVC bitstream does not accurately represent edge components of an image. This phenomenon mainly occurs in the low frequency component of the image, and can be easily compensated by using the coded bit length of the DCT residual on the bitstream as shown in Equation 1 below.

In Equation 1, P _{i , j} is prediction information of the (i, j) th 4X4 block, dc means DC prediction mode, Dth is the threshold, Lr is the DCT residual Indicates the number of bits.

That is, when the DCT transformation is performed after the intra prediction, a perfect edge is hardly predicted in the portion where the real edge is strong in the image, so that a large number of non-zero data exists, and thus, a large DCT residual exists after the DCT transformation.

Therefore, the edge components of the image extracted from the H.264 / AVC bitstream are classified into actual edge components and edge components not using the length of the DCT residual.

Through the above process, the result of classifying the edge component is shown in FIG. As can be seen in Figure 7, it can be seen that the edges in the red circle have been removed.

Hereinafter, a method of dividing an entire image from k H.264 / AVC bitstreams into k sub-pictures and classifying extracted prediction information in each sub-picture into five representative edge information will be described with reference to FIG. 8. do.

There are a total of nine extracted edges, which may have overlapping directionality. To remove such redundancy, the entire image is divided into k sub-images, and nine edge components of the 4 × 4 block extracted in each sub-image unit are classified into five representative edge components. The five representative edge components are composed of five vertical (edge _k ^v ), horizontal (edge _k ^h ), 45 degrees (edge _k ⁴⁵ ), 135 degrees (edge _k ¹³⁵ ), and non-directional (edge _k ^nondir ). . This classification is possible using the number of edges for each component in the sub-image. This is shown in Equation 2.

edge _k ^v = a * n (vertical _k ) + b * n (vertical_right _k ) + c * n (vertical_left _k )

edge _k ^h = d * n (horizontal _k ) + e * n (horizontal_down _k ) + f * n (horizontal_up _k )

edge _k ⁴⁵ = g * n (diag_down_left _k ) + h * n (vertical_left _k ) + i * n (horizontal_up _k )

edge _k ¹³⁵ = j * n (diag_down_right _k ) + k * n (vertical_right _k ) + l * n (horizontal_down _k )

edge _k ^nondir = LinearCombination (n (vertical _k ), n (horizontal _k ), n (dc _k ), n (diag_down_right _k ), n (diag_down_right _k ), n (vertical_right _k ), n (horizontal_down _k ), n ( vertical_left _k ), n (horizontal_up _k ))

The edge _k ^x Is the x-edge component of the k-th sub-image, a to l are weighted as any constant, n (Y _k ) is the number of Y-component edges within the k-th sub-image, and LinearCombination () is in parentheses. Represents a linear combination of variables.

Through the above process, each sub-image has five representative edge components, and among the visual descriptors of MPEG-7, the edge histogram descriptor has five representative edges. Since the component is used, it is compatible with the compressed region search technique of the present invention.

8 shows an embodiment in which the entire image is divided into 16 (k = 16) sub-images. Each sub-image includes nine 4 × 4 blocks and has five representative edge components through Equation 2. That is, one sub-image has vertical component a, horizontal component b, 45 degree component c, 135 degree component d, and non-directional component e, wherein a, b, c, d, and e are arbitrary constants.

9 illustrates an embodiment of dividing an entire image into 16 sub-images such that each sub-image includes 20 4 × 4 blocks.

Since H.264 / AVC encoding basically performs macroblock-based encoding, the process of dividing into the sub-picture as described above can be easily performed by those skilled in the art.

While the invention has been shown and described with respect to certain preferred embodiments, the invention is not limited to these embodiments, and those of ordinary skill in the art claim the invention as claimed in the appended claims. It includes all the various forms of embodiments that can be implemented without departing from the spirit.

According to the real-time edge detection method in the H.264 / AVC compressed region of the present invention as described above, the compressed video can be analyzed in real time without decoding, and the image search, video search, scene through the above analysis. As video analysis such as conversion and editing is possible, considerable technical and economic effects are expected.

Claims (5)

(a) receiving an image compressed with H.264 / AVC; (b) detecting optimal intra prediction mode information on the bitstream of the input image as spatial edge distribution information of the input image; (c) correcting the detected spatial edge distribution information using the number of encoded bits of the DCT residual; And and (d) analyzing the image by using the spatial edge distribution information of the detected input image. The method according to claim 1, In step (c), And converting some of the spatial edge distribution information of the detected input image into the DC distribution using the number of encoded bits of the DCT residual in real time in the H.264 / AVC compressed region. Edge detection method. The method according to claim 2, Converting some of the spatial edge distribution information of the detected input image into a DC distribution,

Where P _{i, j} is prediction information of the (i, j) th 4X4 block, dc is DC prediction, Dth is a threshold, and Lr is a bit of DCT residual Real-time edge detection method in the H.264 / AVC compressed region characterized by the number. The method according to any one of claims 1 to 3, After step (c) and before step (d), The entire image is divided into k non-overlapping k sub-images, and each of the sub-images is vertical (VertEdge), horizontal (HoriEdge), and 45 degrees (45Edge) for each sub-image by using prediction information of the detected image included in each sub-image. ), 135 degrees (135 edge), non-edge (NonEdge) five edge components comprising the step of extracting the real-time edge detection method in the compressed region. The method according to claim 4, For each sub-image, extracting the five edge components of vertical (VertEdge), horizontal (HoriEdge), 45 degrees (45Edge), 135 degrees (135Edge), and non-directional (NonEdge), edge _k ^v = a * n (vertical _k ) + b * n (vertical_right _k ) + c * n (vertical_left _k ) edge _k ^h = d * n (horizontal _k ) + e * n (horizontal_down _k ) + f * n (horizontal_up _k ) edge _k ⁴⁵ = g * n (diag_down_left _k ) + h * n (vertical_left _k ) + i * n (horizontal_up _k ) edge _k ¹³⁵ = j * n (diag_down_right _k ) + k * n (vertical_right _k ) + l * n (horizontal_down _k ) edge _k ^nondir = LinearCombination (n (vertical _k ), n (horizontal _k ), n (dc _k ), n (diag_down_right _k ), n (diag_down_right _k ), n (vertical_right _k ), n (horizontal_down _k ), n ( vertical_left _k ), n (horizontal_up _k )) The edge _k ^x Is the x-edge component of the k-th sub-image, a to l are weighted as any constant, n (Y _k ) is the number of Y-component edges within the k-th sub-image, and LinearCombination () is in parentheses. Real time edge detection method in H.264 / AVC compressed region characterized by linear combination of variables.

Images