KR101082581B1 - Error Concealing Device for H.264/AVC Decorder and Method thereof - Google Patents

Abstract

The present invention relates to a region-specific error concealment apparatus and method of an H.264 / AVC decoder, comprising: detecting error-occurring macroblocks in which an error occurs in an input image; Calculating a motion vector average with respect to the vertical and horizontal directions of the error generating macroblocks using the motion vector values of the normal macroblocks adjacent to the error generating macroblocks; Interpolating at least some of the error-occurring macroblocks using the motion vector average. As a result, the blocking phenomenon between the error-occurring macroblocks during error concealment can be effectively reduced, and the PSNR and subjective picture quality can be improved as compared with using only the spatio-temporal error concealment method through JM.

Description

Error concealing device for H.264 / AVC decorder and method

The present invention relates to an error concealment apparatus and method of an H.264 / AVC decoder. The present invention relates to finding macroblocks that have lost information due to an error occurring during channel transmission, and using motion information of correctly transmitted neighboring macroblocks. An error concealment apparatus and method of an H.264 / AVC decoder for performing error concealment for each region.

Recently, the use of multimedia such as UCC or video telephony through the Internet or wireless network is increasing rapidly. In such multimedia, image information is the center, and in order to send image information as it is, huge data transmission is required. To this end, several compression standards have been created to efficiently transmit image information through channels. Among video compression standards organizations, ISO / IEC has designed MPEG-x for the purpose of video storage / transmission, and ITU-T has designed H.26x standards for real-time video telephony / conference. It is H.264 / AVC that these two different standards bodies have standardized together to create standards that can be applied in all areas.

H.264 / AVC has adopted several techniques that have been proposed so far to increase the compression efficiency but have not been used in the actual standards due to their high complexity. This could result in an approximately double compression rate increase at the same bit rate compared to previous standards. But if you use all of the more complex technologies, the burden on both sides of the decoder / decoder is high. To solve this problem, H.264 / AVC provides three profiles of the technologies applied according to the application purpose. Extended-profile for streaming media services, main-profile for television broadcast or video storage media, and baseline-profile for video telephony and conferencing.

Among them, the baseline profile is a field that does not allow the time delay as much as possible for its application purpose, and employs only relatively low complexity techniques, and has no feedback structure for errors that occur during channel transmission, so there is no FMO (Flexible Macroblock Ordering). Error-proofing techniques, such as ASO (Arbitrary Slice Order) and Data Partition.

In addition, the slice type, which is the minimum unit of the encoded and transmitted packet, is also encoded using only I (Intra) and P (Predictive) slices except for B (Bi-Prediction) slices to prevent time delay through complexity reduction. One picture may have a slice structure in which several macroblocks are grouped, or one picture may be packetized as one slice. The packetized bit stream is sent over the channel with slice header information bits for resynchronization and used to reconstruct the image in the decoder.

The better the compression efficiency, the more sensitive it is to errors occurring during data transmission through the channel, and the worsened image quality due to transmission error propagation. H.264 / AVC transmits data in the form of a packet. Basically, after determining whether a packet is damaged through parity check, if an error occurs, the entire packet is discarded. At this time, since the information of the macroblocks transmitted properly is discarded, a lot of image quality degradation occurs. In order to solve this error problem, various studies have been conducted since the previous standards. What should be preceded for an error problem is the detection of the exact error location. Basically, there are syntax-based detection algorithms, and research has been conducted to improve the detection capability at the decoder side by taking a bit of loss such as PSNR reduction or bit increase for better detection rate, and sending hidden information when encoding. .

In this way, after an error is detected in the decoder, the image must be resynchronized, skipping the errored portion, and continuous decoding must be performed. In general, several macroblocks contain a new header bit for each grouped group, and resynchronization can be done by finding this header start bit. Through this process, error detection and resynchronization is performed so that image decoding can be performed to the end even when an error bit string is encountered.

When correct error detection and resynchronization is performed, the error concealment process is performed as a post-processing in the decoder in units of macroblocks or groups in which information is lost. Since the lost information is continuously propagated according to the characteristics of the video, it is necessary to make the interpolated video with the most similar information. In the prior art, in order to effectively conceal errors, various concealment methods have been studied, which use spatially adjacent pixel values and obtain information corresponding to a previous position in time. However, in this method, if the degree of motion of the macroblocks at both edge regions is different, the difference between the interpolated motion vector values varies greatly in the center of the screen, and as a result, a blocking phenomenon, which is a mismatch between macroblocks, occurs.

An object of the present invention is to find macroblocks in which errors occur during compression transmission of an image, and perform concealment using a different concealment method according to a region, thereby preventing PS from blocking and improving PSNR and subjective picture quality. An apparatus and method for error concealment in a region of a .264 / AVC decoder are provided.

The objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

The object of the present invention is to detect error-produced macroblocks in which an error occurs in an input image; Calculating a motion vector average with respect to the vertical and horizontal directions of the error generating macroblocks using the motion vector values of the normal macroblocks adjacent to the error generating macroblocks; And interpolating at least some macroblocks of the error-occurring macroblocks using the motion vector average. The method may be achieved by a region-specific error concealment method of an H.264 / AVC decoder. .

The object may include an error detection unit for detecting error-occurring macroblocks in which an error occurs in an input image; And calculating a motion vector average in a vertical direction and a horizontal direction by using motion vector values of normal macroblocks adjacent to the error-occurring macroblocks, and comparing at least some macroblocks of the error-occurring macroblocks to the motion vector average. It can be achieved by the region-specific error concealment apparatus of the H.264 / AVC decoder, characterized in that it comprises a;

The region-specific error concealment apparatus and method of the H.264 / AVC decoder can effectively reduce the blocking phenomenon between error-occurring macroblocks during error concealment, and can provide PSNR and subjective picture quality more than the case of using the spatiotemporal error concealment method through JM. Can improve.

1 is a configuration diagram of an error concealment apparatus of an H.264 / AVC decoder according to the present invention;
2 is a flowchart of an error detection algorithm at a macroblock level performed by the error detection unit of FIG. 1;
3 is a conceptual diagram illustrating an error concealment process performed for each section of the first and second concealment portions;
4 is a structure diagram of a syntax of a standard video bitstream;
5 is a conceptual diagram illustrating a process in which a JM concealment technique is performed on a P slice by a second concealment unit;
6 is a conceptual diagram illustrating a process in which a JM concealment technique is performed for each macroblock by a second concealment unit;
7 is a flowchart illustrating a process of concealing errors of macroblocks for each region in the error concealment apparatus according to the present invention.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention.

1 is a configuration diagram of an error concealment apparatus of an H.264 / AVC decoder according to the present invention.

The error concealment apparatus of the H.264 / AVC decoder includes an error detector 10, an area partitioner 20, a first concealment portion 30, and a second concealment portion 40.

The error detector 10 receives the current slice decoded by the decoding apparatus and detects a macroblock in which an error occurs among macroblocks included in the current slice. That is, an error occurs during video data transmission to detect a lost block.

In general, video compression schemes such as MPEG-1, MPEG-2, MPEG-4, H.264 / MPEG 4 Advanced Video Coding (AVC) compress the video data with high efficiency, so that part of the bitstream including the video data An error can occur in a large number of macroblocks.

In particular, when an error occurs in a macroblock included in a P slice, an error is propagated to another P slice that refers to the P slice in which an error occurs, thereby causing a serious error in the entire image sequence.

The error detection unit 10 performs a Syntax / Semantic check on bitstreams created according to the baseline profile by the method illustrated in the flowchart of FIG. 2. Accordingly, error detection may be performed at the macroblock level instead of the packet unit. If an error is detected by the error detector 10, all macroblocks from the macroblock in which the error occurs until the next resynchronization bit are encountered are errors. It is determined by the occurrence macroblocks and the information is discarded. Accordingly, in the related art, the information of the entire frame, which is an error packet, is discarded. However, in the present invention, information of normal macroblocks in the error packet may be maintained.

The region dividing unit 20 may include a macroblock in which an error detected by the error detecting unit 10 has occurred, as shown in FIG. 3, edge region macroblocks located at regions A and C, which are edges of the image, and A; It is divided into the remaining area macroblocks located in the remaining area B except for the area C and the area C.

Then, the area partitioner 20 provides the pixel information for the remaining area macroblocks to the first concealment unit 30 and the pixel information for the edge area macroblocks to the second concealment unit 40. .

The first concealment unit 30 performs concealment of the remaining area macroblocks by using a concealment method in consideration of the movement direction of the entire screen.

The bitstream coming into the decoder of H.264 / AVC has a structure as shown in FIG. 4, and FIG. 3 shows a slice in which an error is detected through the error detector 10. In the current_frame on the right side of FIG. 3, white squares represent macroblocks in which the correct transmission is performed, and dark colored squares represent error generating macroblocks in which an error is detected and information is discarded.

The first concealment unit 30 performs a concealment process on the remaining area macroblocks among the error-occurring macroblocks for which information is lost.

The first concealment unit 30 uses the motion vector values of the normal macroblocks immediately adjacent to the remaining region macroblocks in which an error occurs first, and is a motion average of each of the vertical and horizontal directions with respect to the remaining region macroblocks. The vector mean Conceal _ mv ( x ) and Conceal _ mv ( y ) are obtained using Equation 1.

Here, N means the number of macroblocks belonging to the horizontal axis of the image.

The motion vector average values thus obtained become respective motion vectors for all remaining region macroblocks, and the first concealment unit 30 obtains pixel information from the reference frame. The first concealment unit 30 performs a process of interpolating the current macroblocks with pixel information consisting of motion compensation. This error concealment method uses the simple motion vector averaging to move all the error-producing macroblocks in either direction so that the opposite edges of the movement direction are not filled, which causes a phenomenon to propagate to the next image as its own error. Can be. However, since the first concealment unit 30 conceals only the region macroblocks except for the edge of the image using the motion vector average, the conventional edge region macroblocks may not be properly filled.

The second concealment unit 40 performs error concealment on the edge region macroblocks using a spatiotemporal error concealment method using the reference software JM (Joint Model).

Here, the adjacent pixel interpolation method, which is a spatial use method, performs concealment through interpolation by using a distance correlation between pixels that are vertically and horizontally adjacent to a pixel position of an errored current macroblock. The temporal concealment method is mainly performed in a P (Predictive) slice, and as shown in FIG. 5, a motion vector that is a motion vector of a neighboring normal macroblock or an already hidden macroblock with respect to a macroblock in which an error has occurred. Vector) pairs. In general, adjacent macroblocks show a high degree of similarity in motion. Using this property, JM uses the motion vector of normal macroblocks around an errored macroblock as a reference motion vector. Through motion prediction and compensation using reference motion vectors, pixel information of a previous frame correlation position is obtained to interpolate a macroblock in which a current error occurs. The second concealment unit 40 selects a reference motion vector pair that satisfies a boundary matching algorithm (BMA) that makes the smallest sum of differences between adjacent pixels among the reference motion vector pair as an optimal motion vector.

6 shows a process of BMA progressing with respect to a macroblock, and BMA is determined by Equation 2 below.

는 참조 모션 벡터쌍(mv^dir)에 의해 은닉되는 매크로블록의 Y-픽셀값을 의미하고, mv^dir은 주변 참조가능 매크로블록(top, bottom, left, right) 중 dir에 해당되는 매크로블록의 참조 모션 벡터쌍을 의미하고, Y는 은닉될 매크로블록에 인접한 정상 복호화되었거나 보간된 매크로블록의 Y-픽셀값을 의미한다.
Where dir denotes referenceable macroblocks adjacent to four directions of direction top, bottom, left, and right, and d _sm denotes a minimum distortion value.

Is the Y-pixel value of the macroblock concealed by the reference motion vector pair (mv ^dir ), and mv ^dir is the reference of the macroblock corresponding to ^dir of the surrounding reference macroblocks (top, bottom, left, right). Means a motion vector pair, and Y means the Y-pixel value of a normal decoded or interpolated macroblock adjacent to the macroblock to be concealed.

If the optimal motion vector is determined as the motion vector for the hidden macroblock through the BMA, the second concealment unit 40 performs a process of interpolating the current macroblocks with pixel information consisting of motion compensation. At this time, the motion vector of the hidden macroblock becomes a reference motion vector for the next concealment. In general, in order to improve the efficiency of using the reference motion vector for each macroblock in JM, concealment ordering is determined from both edges to the center direction. However, in the second concealment part 40 of the present invention, the edges A are both edges. Only the edge region macroblocks of region C and region C are hidden.

In addition, since the edge of the image is generally located in the background or non-major objects, there is no degree of movement, or similar to the motion vector values of almost adjacent macroblocks, the edge region is determined by the JM used in the second concealment portion 40. It is preferable to conceal using a motion vector of.

A process of concealing an error in the error concealment apparatus having such a configuration will be described below with reference to FIG. 7.

When the decoded image is input (S700), the error detector 10 detects error-occurring macroblocks in the frame (S710). Then, the area partitioner 20 separates the edge area macroblocks and the remaining area macroblocks from the error-occurring macroblocks (S720), and respectively stores the pixel information of the second concealment portion 40 and the first concealment portion ( Provided as 30) (S730). In this case, the first concealment unit 30 is provided with pixel information for the remaining area macroblocks, and the second concealment unit 40 is provided with pixel information for the edge area macroblocks.

The first concealment unit 30 calculates a motion vector average with respect to the vertical and horizontal directions using Equation 1 using the motion vector values of the normal macroblocks (S740), and uses the calculated motion vector average values to calculate the remaining region. Interpolate macroblocks (S750).

The second concealment unit 40 determines an optimal motion vector using Equation 2 (S745), and interpolates the edge region macroblocks using the determined motion vector (S755).

As described above, in the region error concealment method of the decoder, the macroblocks in which the error is generated are separated into both edges and the remaining center region of the image, and the edge region macroblocks are a motion prediction compensation method using a reference motion vector of the JM concealment method. Interpolation is performed and the remaining region macroblocks are concealed by using a motion vector average that emphasizes the directionality of the entire image.

When the spatio-temporal error concealment method through the JM is conventionally used, the difference between the motion vector values interpolated varies greatly in the center of the screen if the motion degrees of the macroblocks at both edge regions are different, and as a result, between the macroblocks. The blocking phenomenon, which is a mismatch phenomenon, can be prevented. However, in the case of the concealment method using the motion vector averaging, since the hidden macroblocks are moved all in one direction, the opposite edges of the movement direction are not filled, which can prevent the propagation to the next image as a self error. have. This can be prevented through the second concealment concealment method through the region compartment.

Accordingly, the blocking phenomenon between the error-occurring macroblocks during error concealment can be effectively reduced, and the PSNR and subjective picture quality of about 1.3 dB on average can be improved compared to when using the spatio-temporal error concealment method through JM.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, Of the right.

Claims (10)

Detecting error generating macroblocks in which an error occurs in the input image;
Calculating a motion vector average with respect to the vertical and horizontal directions of the error generating macroblocks using the motion vector values of the normal macroblocks adjacent to the error generating macroblocks;
Interpolating at least some of the error-occurring macroblocks using the motion vector average; And
Separating the edge region macroblocks positioned at the edge of the image among the error-occurring macroblocks and the remaining region macroblocks which are remaining macroblocks,
The interpolating may include interpolating the edge region macroblocks using a motion vector of surrounding normal macroblocks as a reference motion vector; And interpolating the remaining region macroblocks using the motion vector average,
The interpolating of the edge region macroblocks may be performed by using Equation 1 below to optimize a reference motion vector pair that satisfies a boundary matching algorithm (BMA) that results in the smallest sum of differences between adjacent pixels in the reference motion vector pair. Selecting as the motion vector of the; And
Interpolating the edge region macroblocks using the optimal motion vector.
[Equation 1]

Where dir denotes referenceable macroblocks adjacent to four directions of direction top, bottom, left, and right, and d _sm denotes a minimum distortion value.

Is the Y-pixel value of the macroblock concealed by the reference motion vector pair (mv ^dir ), and mv ^dir is the reference of the macroblock corresponding to ^dir of the surrounding reference macroblocks (top, bottom, left, right). Means a motion vector pair, and Y means the Y-pixel value of a normal decoded or interpolated macroblock adjacent to the macroblock to be concealed. delete delete The method of claim 1,
The method of concealing an error for each region of the H.264 / AVC decoder, wherein the motion vector average is calculated using Equation 2 below.
[Equation 2]

Where N is the number of macroblocks belonging to the horizontal axis of the image. delete An error detector for detecting error-occurring macroblocks in which an error occurs in the input image;
A motion vector average of a vertical direction and a horizontal direction is calculated by using motion vector values of normal macroblocks adjacent to the error generating macroblocks, and at least some macroblocks of the error generating macroblocks are used by the motion vector average. Interpolating the first concealment unit; And
A second concealment unit interpolating the edge region macroblocks using the motion vector of the surrounding normal macroblocks as a reference motion vector,
The second concealment unit selects a reference motion vector pair that satisfies a boundary matching algorithm (BMA) that minimizes the sum of differences between adjacent pixels among the reference motion vector pairs as an optimal motion vector using Equation 3 below. And interpolating the edge region macroblocks using the optimal motion vector.
[Equation 3]

Where dir denotes referenceable macroblocks adjacent to four directions of direction top, bottom, left, and right, and d _sm denotes a minimum distortion value.

Is the Y-pixel value of the macroblock concealed by the reference motion vector pair (mv ^dir ), and mv ^dir is the reference of the macroblock corresponding to ^dir of the surrounding reference macroblocks (top, bottom, left, right). Means a motion vector pair, and Y means the Y-pixel value of a normal decoded or interpolated macroblock adjacent to the macroblock to be concealed. The method according to claim 6,
The region-specific error of the H.264 / AVC decoder, characterized in that it further comprises a region partition for separating the edge region macroblocks located at the edge of the image of the error-occurring macroblocks, and the remaining region macroblocks that are the remaining macroblocks. Concealment device. The method of claim 7, wherein the first hidden portion,
And the remaining region macroblocks are interpolated using the motion vector average calculated using Equation 4 below.
[Equation 4]

Where N is the number of macroblocks belonging to the horizontal axis of the image.
delete delete

Images