KR20070023732A - Device for producing progressive frames from interlaced encoded frames - Google Patents

Abstract

The present invention relates to a method and apparatus for decoding an encoded frame set at a first resolution (SD) to produce an output frame set at a lower resolution (QCIF). The apparatus comprises: a partial decoding unit DECp for generating a first residual error field at a second resolution lower than the first resolution and a second residual error field at a third resolution lower than the first resolution based on the encoding frame ), A first prediction unit PRED1 for generating a first motion compensation field based on the first residual error field, the first reference field Fix1 and the second reference field Fix2, and the first residual error field. A second prediction unit that generates a second motion compensation field based on the first adder, the second residual error field, the first reference field and the second reference field in combination with the motion compensation field to obtain a next first reference field Fiy1 (PRED2), a second adder for combining a second residual error field with the second motion compensation field to obtain a next second reference field (Fiy2) corresponding to the output frame.

Description

Device for generating progressive frames from interlaced frames {DEVICE FOR PRODUCING PROGRESSIVE FRAMES FROM INTERLACED ENCODED FRAMES}

The present invention relates to a method and apparatus for decoding an encoded frame set at a first resolution to produce an output frame set at a lower resolution, wherein the encoded frame is configured to display a first encoding field interlaced with the second encoding field. Include.

The present invention can be used in video decoding applications, and more particularly in applications where compressed video bit-streams in interlaced format must be displayed at a lower level on a progressive display. Typical applications include receiving Digital Video Broadcast-Terrestrial (DVB-T) programs on a phone or mobile device such as a personal digital assistant (PDA).

Low power consumption is an important feature of mobile devices. Mobile devices provide video encoding and decoding capabilities that are known to consume large amounts of energy today. Thus, a so-called low power video algorithm is needed.

In fact, access to external memory, such as SDRAM, is a headache for video devices. This is due to both power consumption issues as the memory is known to be the most power consuming part of the system, and speed limitations due to the interchange bandwidth between the central processing unit (CPU) and the memory.

In a conventional video decoder, the motion compensation module requires a lot of such access, since in the so-called reference frame the motion compensation module constantly instructs blocks of pixels. To overcome this problem, international patent application WO 03/010974 discloses a video decoding apparatus in which embedded resizing is used in combination with external scaling to reduce the computational complexity of the decoding.

Such a video decoding apparatus is shown in FIG. 1 and includes a variable length decoding block (VLD), an inverse scan and inverse quantization block (ISIQ), an 8x8 inverse discrete cosine transform block (IDCT), and a decay block (DECI). It includes a first path consisting of. In operation, the VLD block decodes the incoming video bit-stream at standard resolution (SD) to produce a motion vector (MV) and quantization transform coefficients. Next, the ISIQ block inverse scans and inverse quantizes the quantization transform coefficients received from the VLD block. The IDCT block also performs filtering to remove high frequencies from the transform coefficients. After performing IDCT, the condensation block samples the output of the 8x8 IDCT block at a predetermined rate to reduce the resolution of the decoded video output frame (OF).

As can be further appreciated, the decoder also includes a second path consisting of the VLD block, the downscaling block DS, the motion compensation unit MC and the frame storage MEM. In operation, the downscaling block DS reduces the magnitude of the motion vector MV provided by the VLD block in proportion to the rate of decrease in the first path. Thereby, memory access can be reduced as motion compensation is performed at the reduced resolution to match frames generated in the first path. In addition, as the size of the stored memory frame is reduced, the memory size is also reduced.

However, the sequence of output frames is still interlaced, resulting in unacceptable artifacts when rendered on the progressive display. Of course, a deinterlacing unit can be inserted between the modified decoder and the RGB converter, but at the cost of complexity and memory transfer.

Summary of the Invention

It is an object of the present invention to provide a method and apparatus for decoding an interlaced video sequence to produce a progressive downsized video sequence having an appropriate complexity.

To this end, the decoding device according to the invention:

A partial decoding unit for generating a first residual error field at a second resolution lower than the first resolution and a second residual error field at a third resolution lower than the first resolution based on the encoding frame;

A first prediction unit for generating a first motion compensation field based on the first residual error field, the first reference field, and the second reference field;

A first adder combining the first residual error field with the first motion compensation field to obtain a next first reference field,

A second prediction unit for generating a second motion compensation field based on the second residual error field, the first reference field and the second reference field;

And a second adder that combines a second residual error field with the second motion compensation field to obtain a next second reference field Fiy2 corresponding to the output frame.

Similarly, the decoding method according to the invention is:

Based on the encoding frame, generating a first residual error field at a second resolution lower than the first resolution,

Based on the encoding frame, generating a second residual error field at a third resolution lower than the first resolution,

Generating a first motion compensation field based on the first residual error field, the first reference field and the second reference field,

Combining the first residual error field with the first motion compensation field to obtain a next first reference field,

Generating a second motion compensation field based on the second residual error field, the first reference field and the second reference field,

Combining the second residual error field with the second motion compensation field to obtain a next second reference field corresponding to the output frame.

As will be described later in detail, the decoding solution according to the present invention includes embedded resizing, which is adapted to output a progressive sequence directly so that deinterlacing is implicitly performed by the decoding loop. The cost of this solution in terms of computation, memory size and access is greater than that of a conventional video decoder without deinterlacing, but provides a large amount of good visual quality. The decoding solution according to the invention is also more cost effective and cheaper than conventional video decoding combined with de-interlacing, and is almost the same as the conventional combination in terms of visual quality.

Advantageously, the partial decoding unit is in series an entropy decoding unit for generating a transform coefficient block at a second or third resolution from an encoded data block at a first resolution, at a second or third resolution from a quantized transform coefficient block. An inverse quantization decoding unit for generating a transform coefficient block, and an inverse transform unit for generating a decoded coefficient block at a second or third resolution from the transform coefficient block. As a result, the inverse transform is smaller, thereby lowering the complexity of the decoding solution.

According to an embodiment of the invention, the second resolution is the same as the third resolution. Thanks to such a feature, the decoding solution provides good visual quality.

According to another embodiment of the present invention, the second resolution may vary depending on the resources available to the decoding apparatus. As a result, decoding is sufficiently efficient when full resources such as battery level or CPU are available, and still possible when low resources are available.

The invention also relates to a portable device comprising a decoding device and a screen for displaying an output frame set.

Finally, the invention relates to a computer program product comprising program instructions for implementing a decoding method according to the invention.

These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described below.

The invention will now be described in more detail with reference to the accompanying drawings, for example.

1 shows a block diagram of a decoding device according to the prior art.

2 shows a block diagram of an embodiment of a decoding apparatus according to the invention.

3 illustrates field prediction according to an embodiment of the present invention.

4 illustrates field prediction according to another embodiment of the present invention.

5 shows DCT coefficient extraction from an 8x8 DCT matrix.

6 illustrates picture reconstruction in the pixel domain according to an embodiment of the present invention.

7 illustrates picture reconstruction in the pixel domain according to another embodiment of the present invention.

The present invention relates to a method and apparatus for generating progressive frames from interlaced encoded frames. The method can be applied to any video decoding apparatus in which a frame sequence must be stored in memory. In particular, the present invention is concerned with reducing the size of the reference frame memory while sufficiently maintaining the overall visual quality of the decoded output frame.

The principle of an embodiment of a decoding device according to the invention is shown in FIG. 2.

Such a decoding apparatus comprises a partial decoding unit DECp that generates a first residual error field at a second resolution lower than the first resolution and generates a second residual error field at a third resolution lower than the first resolution. ) And two residual error fields are generated based on the encoded frames. More specifically, the encoded frame is divided into a plurality of blocks of encoded data values. These encoded data values are, for example, DCT coefficients corresponding to the discrete cosine transform of the luminance or chrominance of the pixel. According to an embodiment of the invention, the partial decoding unit DECp is in series:

An entropy decoding unit (VLDp) for generating a transform coefficient block at a second or third resolution from a data block encoded at a first resolution, for example based on variable field decoding,

An inverse quantization decoding unit IQp for generating a block of transform coefficients at a second or third resolution from a block of quantized transform coefficients, and

For example, an inverse transform unit (ITp) for generating a decoding coefficient block at a second or third resolution from the quantized coefficient block.

It will be apparent to those skilled in the art that other embodiments are possible for a partial decoding unit such as the partial decoding unit disclosed in the prior art. This DECp unit is called a partial decoding unit because it performs both decoding and downsizing of an encoded frame.

The decoding device according to the invention also comprises two prediction units PRED1 and PRED2. As shown in FIG. 3, the first prediction unit PRED1 may predict the frame based on the first residual error field, the first reference field Fix1 of the reference frame Ix, and / or the second reference field Fix2. Is adapted to generate a first motion compensation field of (Py). Next, the first adder is adapted to combine the first residual error field with the first motion compensation field to obtain the next first reference field Fiy1.

Similarly, the second prediction unit PRED2 is adapted to generate a second motion compensation field based on the second residual error field, the first reference field Fix or Fiy1 and / or the second reference field Fix2. Next, the second adder is adapted to combine the second residual error field with the second motion compensation field to obtain the next second reference field Fiy2, the next second reference field corresponding to the output frame.

In the present description, it will be apparent to those skilled in the art that the first field is a top field and the second field is a bottom field, but the first field may be a bottom field and the second field may be a top field. something to do. In addition, the encoding frame is here a predictive P frame, but may also be a bidirectional predictive B frame.

By default, two fields of the current encoded frame are decoded with reduced resolution, and only one of them is displayed. Since one field is displayed at a given time, there is no interlace defect. Thus, the visual quality is optimally adapted to the final display. Moreover, the first field is also an undisplayed field and is also decoded since it can be used as a reference for reconstruction of the display field. In the MPEG-2 standard, this is especially the case when "field motion compensation" is applied.

Of course, the second field, which is the display field, is decoded at the display resolution (eg QCIF). With respect to other fields, the most natural solution is to decode at the same resolution as well. This results in memory requirements that are double compared to the embedded resizing solutions of the prior art without deinterlacing in terms of size and transfer. Although this field is not indicated, it can only be decoded at any resolution, since it is here only predicting other fields. It is detailed in the following description.

For the sake of clarity, the following description is based on the MPEG-2 encoded bit-stream, which is the most common compression standard in the broadcast world, but any encoding technique is any block-based technique (e.g. MPEG-2, MPEG). -4, H.264, etc.) will be apparent to those skilled in the art.

According to the first embodiment shown in FIG. 3, the decoding method is adapted to convert an interlaced standard definition SD sequence into a progressive QCIF sequence by decoding two QCIF fields.

Typical input spatial resolution is standard definition SD, which means 720x576x25Hz (PAL) or 720x480x30Hz (NTSC) in interlace format. Currently, most mobile devices are equipped with near-QCIF (progressive 180x144 or 120 pixels) screens. This implies spatial downscaling by a ratio of 4 in both the horizontal and vertical directions. Applicants now describe in detail partial IDCT and motion compensation that cause resizing and deinterlacing.

As discussed above, the low frequency content of the VLD decoded 8x8 DCT matrix is used to simulate anti-aliasing low pass filtering. According to the third approach, the higher AC coefficients are skipped and the reduced IDCT is executed, resulting in MxN pixel output data blocks instead of 8x8.

In the case of the applicant, as shown in Fig. 5, the lower 2x2 or 4x2 coefficient of the DCT matrix is used depending on the dct_type (field DCT or frame DCT) in the macroblock header. More specifically, if a 16x16 pixel interlaced macroblock from an interlaced frame picture is frame DCT coded, each of the four blocks has pixels from two fields. If an interlace macroblock from an interlaced frame picture is field DCT coded, each block consists of pixels from only one of the two fields. In the latter case, each 16x16 macroblock is divided into fields of 16 pixels wide and 8 pixels high by alternating lines of pixels, and then each field contains two 8x8 blocks from one field. And split it into left and right parts, making two from the other fields.

In Applicants' decoding solution, the indicated frame corresponds to one of the original fields, which is already downscaled vertically by a factor 2 compared to the original frame. Next, this field must be further downscaled by two in the vertical direction and four in the horizontal direction to obtain an output progressive frame in QCIF format. If dct_type is set to 1, the field DCT is applied to the encoder, and thus 4x2 IDCT is performed. Conversely, if dct_type is set to 0, the frame DCT is applied to the encoder, so that two 2x2 IDCTs are performed in different phases for each field.

및

이다.More specifically, from four input 8 × 8 DCT matrices of 16 × 16 macroblocks, we derive two output 4 × 4 pixel blocks (for each rescaled field). For this purpose, a modified inverse transform function, hereafter referred to as IDCT_N × M (), is used. The arguments of this function are the 8 × 8 DCT matrix F, the expected dimensions N × M of the output pixel block f (N is vertical, M is horizontal), and the vertical and horizontal phase shifts applied to maintain proper pixel alignment.

And

to be.

The definition of IDCT_N × M is as follows (for y = 0 to N-1 and x = 0 to M-1):

Where f (x, y) represents the output pixel at position (x, y), F (u, v) represents the input DCT coefficient at position (u, v), and C (u) is Is defined as:

In an embodiment of the present invention, and for SD to QCIF rescaling, the following values are selected:

In terms of phase, those values were determined to maintain matching between frame IDCT and field IDCT modes for a given field to maintain proper pixel alignment. The phase shift between two fields is not critical because the fields are never displayed together in the proposed invention. Nevertheless, a phase that ensures the center position of the sub-sampling pixel on the original grid is preferred because its phase prevents the ambient effect. In fact, trimming the DCT coefficients is just the same as the ideal low pass filtering within the current block. Using the values of the previous table, the output pixel P1 corresponding to the first field Fi1 and the output pixel P2 corresponding to the second field Fi2 are spatially positioned as shown in FIG. 6.

Thus, motion compensation MC is obtained. In particular, the motion vector is adjusted to take into account the phase difference between the two fields. Moreover, different approaches may be considered depending on both the motion type (frame or field MC) and the motion vector values.

The strategy is straightforward for field motion compensation because the Boolean value clearly provides the reference field used for prediction.

In field motion compensation, three cases are envisioned according to MV.y (expressed as half a pixel in MPEG-2) of the vertical component of the original motion vector.

Case 1: MV.y modulo 4 = 0

In the frame motion compensation process, the fields are aligned, that is, the line corresponding to the top (each bottom) field remains the original compensation frame block predicted by the top (each bottom) field of the original reference frame. Thus, in rescaled motion compensation, prediction of each field is done using only the corresponding resized reference field. Interpolation can be used to reach sub-pixel accuracy.

Case 2: MV.y modulo 4 = 2

In the motion compensation process, the line corresponding to the top (each bottom) field is predicted as the bottom (each top) field of the original reference frame. Thus, in rescaled motion compensation, prediction of each field is done using only the corresponding resized reference field. Thus, in rescaled motion compensation, prediction of each field is done using only the corresponding resizing reference field. Interpolation can be used to reach sub-pixel accuracy.

Case 3: Other

The prediction is done by half pixel interpolation between two fields of the original reference frame. This is changed to the appropriate sub-pixel interpolation between the appropriate lines of the two resized reference fields.

According to another example shown in FIG. 4, the decoding method is adapted to convert an interlaced standard definition SD sequence into a progressive QCIF sequence by decoding the first field in the QCIF format and the second field in the 1/2 QCIF format. . In this embodiment of the present invention, the non-display field is further downscaled vertically. The parameter values for IDCT_N × M are as follows (when the bottom field is displayed):

Therefore, the output pixel P1 corresponding to the first field Fi1 and the P2 corresponding to the second field Fi2 are spatially positioned as shown in FIG. 7.

Motion compensation is derived according to a new dimension of the new phase and non-display fields.

This embodiment is justified in applications where CPU and memory resources need to be further reduced compared to the solution described in the first example. The visual quality deteriorates slightly because the non-display reference field has a smaller resolution than when displayed, resulting in blurry prediction, but the complexity of decoding is reduced.

It will be apparent to those skilled in the art that the resolution of the non-display field may take half the resolution of the display field. Furthermore, the resolution of the non-display field can be varied depending on the resources available at the decoding device (battery level, CPU, ...). For example, in the case of frame DCT, if N = 4 and M = 2 for the display field, then N × M is given by the following values: 4 × 2, 3 × 2, 2 × 2, 1 × 2, 4 × 1, 3 × 1, 2 × 1 or 1 × 1.

In broadcast conditions, the spatial resolution of the encoded video sequence does not necessarily need to be SD. The original SD sequence is often downsized horizontally before being encoded. This can serve as a preprocessing step to further reduce the final compression bit-rate. In a normal application, such as a set top box connected to a television set, the decoded sequence is upsized horizontally to retrieve the correct aspect ratio before display.

Typical spatial resolutions are: (576 lines for PAL, 480 lines for NTSC)

SD: 720 pixels per line

3/4 SD: 540 pixels per line

2/3 SD: 480 pixels per line

1/2 SD: 360 pixels per line.

In targeted applications, the proposed invention can be applied to all formats. Similarly, the target progressive display may be different from QCIF (CIF or QVGA format is already commercially available). The size of IDCT_N × M should be applied and the phase set accordingly so as to fit as closely as possible to the scaling ratio between the input and output spatial resolutions. If this ratio cannot be represented as an integer value (of form n / 8 of an 8x8 DCT matrix), the preferred solution is to approximate this ratio to the nearest integer value and if the dimension is larger (or smaller) than the display resolution. Cuts (or pads) at render time.

Improvements over the foregoing embodiments can be made using the best vertical component of the DCT matrix to better distinguish the two fields in the case of frame DCT.

For example, in the case of SD (interlace) to QCIF (progressive), two 2x2 pixel blocks are generated (with different phases for each field) using 2x2 low frequency AC coefficients. It causes interference between the two fields as the odd lines are contaminated by the even lines via DCT truncation and vice versa. The way to overcome this issue is to use the last row of coefficients in the DCT matrix. In fact, these frequencies represent the difference between the odd and even lines of the original block.

By applying this better field separation, when two fields are very different from one another (for example, a scene-cut between two fields, a flash that occurs only during one of the two fields, etc.) In certain cases, significant visual enhancements are made. This can be implemented as an additional quality improvement tool if the available resources are very sufficient.

The proposed invention can be applied to any device having a video capability that needs to decode a compressed interlaced video sequence and need to render it in a reduced solution on a progressive display. While the present invention significantly reduces CPU usage, memory requirements, memory bandwidth, latency and power consumption compared to a complete sequential process, interlaced visual defects are reduced compared to conventional simplified techniques. Thus, although the resources are limited, extended battery life and good visual quality are important features and are particularly suitable for DVB-T reception on mobile devices, which differentiates factors.

Some embodiments of the invention have been described above by way of example only, and it will be apparent to those skilled in the art that modifications and variations may be made to the above embodiments without departing from the scope of the invention as defined in the appended claims. Also, in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The term "comprising" shall not exclude the presence of elements or steps other than those described in the claims. The term "a" or "an" does not exclude a plurality. The invention can be implemented by means of hardware comprising several individual components, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means may be embodied as one hardware item. The fact that any means are cited in different independent claims does not mean that a combination of these means cannot be used to advantage.

Claims (7)

A decoding device for decoding an encoding frame set at a first resolution SD to generate an output frame set at a lower resolution QCIF, the encoding frame comprising a first encoding field interlaced with a second encoding field. A partial decoding unit (DECp) for generating a first residual error field at a second resolution lower than the first resolution and a second residual error field at a third resolution lower than the first resolution based on the encoded frame , A first prediction unit PRED1 for generating a first motion compensation field based on the first residual error field, the first reference field Fix1 and the second reference field Fix2, A first adder for combining the first residual error field with the first motion compensation field to obtain a next first reference field Fiy1, A second prediction unit PRED2 which generates a second motion compensation field based on the second residual error field, the first reference field and the second reference field, A second adder that combines the second residual error field with the second motion compensation field to obtain a next second reference field Fiy2 corresponding to an output frame Decoding apparatus comprising a. The method of claim 1, The encoded frame is divided into a plurality of encoded data blocks, The partial decoding unit DECp is in series, An entropy decoding unit (VLDp) for generating a transform coefficient block at a second or third resolution from an encoded data block at a first resolution, An inverse quantization decoding unit IQp for generating a transform coefficient block at a second or third resolution from the quantized transform coefficient block, and An inverse transform unit (ITp) for generating a decoded coefficient block at a second or third resolution from the transform coefficient block. The method of claim 1, And the second resolution is the same as the third resolution. The method of claim 1, And said second resolution varies in accordance with the resources available on said device. A portable device comprising a screen for displaying the device of claim 1 and an output frame set. A decoding method for decoding an encoding frame set at a first resolution SD to generate an output frame set at a lower resolution QCIF, wherein the encoding frame includes a first encoding field interlaced with a second encoding field. Based on the encoded frame, generating a first residual error field at a second resolution lower than the first resolution, Generating a second residual error field at a third resolution lower than the first resolution based on the encoded frame; Generating a first motion compensation field based on the first residual error field, the first reference field Fix1 and the second reference field Fix2, Combining the first residual error field with the first motion compensation field to obtain a next first reference field (Fiy1), Generating a second motion compensation field based on the second residual error field, the first reference field and the second reference field; Combining the second residual error field with the second motion compensation field to obtain a next second reference field (Fiy2) corresponding to an output frame; Decoding method comprising a. A computer program product comprising program instructions for implementing the method of claim 6 when the program is executed by a processor.

Images