JP2647230B2 - Image decoding device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial Application Field] The present invention relates to an image encoding apparatus that can perform a predetermined image quality even when a part of information of an intra-frame encoded image signal is lost. The present invention relates to an image decoding device for obtaining a reproduced image signal of

[Prior Art] FIG. 2 is a schematic diagram of an image packet encoding device including a conventional image encoding device and an image packet decoding device including a conventional image decoding device (Nomura et al., “DCT Of Cell Loss and Its Countermeasure in Interframe Coding Using CDMA, "1988 Image Coding Symposium, pp85
-86).

FIG. 2A shows an encoding device. In FIG. 2A, an image signal s input from an input terminal 21 is shown.
Is divided into blocks of several n × n pixels for each frame, and then divided into discrete cosine transformers (DCT converters) 22.
Is subjected to discrete cosine transform in block units. From the obtained discrete cosine coefficients, a frame memory is
The discrete cosine coefficient of the corresponding block one frame before stored in 24 is subtracted to obtain these difference values d. The difference value d is converted into a code i by the quantizer 25 and output to the scanner 26.

This code i is sent to an inverse quantizer 27 having the inverse characteristic of the quantizer 25, and after being inversely quantized, the adder 20 outputs the discrete value one frame before used to obtain the difference value d. The signal is added to the cosine coefficient, converted to a locally reproduced discrete cosine coefficient, and output to the frame memory 24. This locally reproduced discrete cosine coefficient is used for encoding the next frame.

With such a processing system, a code i obtained by predictively coding the discrete cosine coefficient is obtained.

As shown in FIG. 3, the scanner 26 sequentially scans the input code i while repeatedly reciprocating diagonally in block units. FIG. 3 shows the scanning order of the code i of the block of 8 × 8 pixels. At this time, the scan is divided into the low frequency component iL and the high frequency component iH by two.
The code i is hierarchized by division. The hierarchized code components iL and iH are sent to the corresponding variable-length encoders 28 and 29 and run-length-Huffman-encoded, respectively. They are hierarchized, assembled into packets (cells) of the same type, and sent out to the transmission path.

That is, information indicating priority / non-priority is added to each cell as header information. Introduce a hierarchical structure with priority to the network, against cell discards that occur at the time of network congestion, without discarding priority cells containing low-frequency elements important for image signals,
Non-priority cells are discarded, and image quality deterioration due to discarding is minimized.

FIG. 2 (B) shows the configuration of the decoding device. Second
In FIG. 2B, a cell sent from the encoding device is decomposed by a packet decomposing unit 211 and is divided into encoded image data and parameters required for decoding. The coded image data (data for the low-band code iL and data for the high-band code iH) constituting the priority cell and the non-priority cell are run-length / Huffman-coded by the corresponding variable-length decoding units 212 and 213, respectively. Decrypted. After that, the image code (low-band code iL, high-band code iH) that has been divided into priority / non-priority by the data combining unit 214 is reconstructed, and further, the inverse operation of scanning in the encoding device is performed for each block. Get the playback code is.

The reproduced code is subjected to inverse quantization, which is the inverse processing of the quantizer 25, by the inverse quantizer 215.
Is added by the adder 217 to the reproduced discrete cosine coefficient of the corresponding block one frame before stored in the corresponding block, and is converted into the reproduced discrete cosine coefficient. An inverse DCT transformer 218 performs an inverse transform of the discrete cosine transform on the reproduced discrete cosine coefficient, and a reproduced image signal sS obtained by this is output to an output terminal 219.
And is stored in the frame memory 216 for processing of the next frame.

[Problems to be Solved by the Invention] However, in the above-described image packet encoding / decoding device, a cell is given a priority according to the importance of data in the cell, and a network is also referred to as a priority channel / non-priority channel. It is assumed that a hierarchical circuit is prepared. In this way, the high-priority cells are protected from being discarded to suppress image quality deterioration, but the control configuration and the like for that purpose have been complicated.

In addition, although the priority concept is introduced, the capacity of the priority channel may be insufficient, and the priority cell may be discarded. The effect on the reproduced image quality at this time is very large.

Furthermore, when an error occurs in the transmission path such as cell discard, the contents of the frame memory 24 used for inter-frame coding and the frame memory 216 used for inter-frame decoding become inconsistent and cannot be decoded properly. There is also a problem that this state propagates for a long time.

Therefore, a coding method that does not assume the use of a hierarchical network and that can prevent a large adverse effect from being propagated for a long time even when an error occurs in a transmission system has been separately proposed by the same applicant. ing. That is, the image signal is horizontally divided by the band division filter for each frame,
And by filtering in the vertical direction, it is divided into several band components. Of these bands, the band component having low-frequency components in both the horizontal and vertical directions, which is the most important for forming an image, is adversely affected. Has been proposed in which only one frame is used for intra-frame coding, and for other band components, inter-frame coding is performed.

However, even with such an encoding method, when a part of the encoded image signal arrives at the image decoding device due to cell discarding or the like, it is unavoidable that the image quality deteriorates at that moment. .

The present invention has been made in view of the above points,
It is an object of the present invention to provide an image decoding apparatus which can ensure a high level of quality of a reproduced image even when a coded image signal is lost or the like in a system from an image encoding apparatus to an image decoding apparatus.

Means for Solving the Problems The present invention provides an image decoding apparatus that receives an intra-frame coded low-frequency component image signal and intra-frame decodes the low-frequency component image signal. And an inter-frame decoding unit that receives an inter-frame encoded image signal of another band component and inter-frame decodes the image signal of another band component. Further, the intra-frame decoding unit includes: an intra-frame decoder for intra-frame decoding the received low-frequency component image signal; a frame memory for storing the decoded low-frequency component image signal; A motion compensator for motion-compensating the low-frequency component image signal stored in the memory according to the motion vector received from the encoding device; and an intra-frame decoder when the low-frequency component image signal is normally received. And if the image signal of the low-frequency component cannot be received normally, the output of the motion compensator is selected and output to the subsequent processing unit.

[Operation] The intra-frame decoding unit intra-frame decodes the image signal of the low-frequency component, and the inter-frame decoding unit inter-frame decodes the image signal of another band component. Further, the intra-frame decoding unit stores the decoded low-frequency component image signal in the frame memory, and according to the motion vector received from the encoding device, stores the low-frequency component image signal stored in the frame memory. Is motion compensated. The selecting means of the intra-frame decoding section selects the output of the intra-frame decoder when the image signal of the low-frequency component is normally received, and selects the output when the image signal of the low-frequency component cannot be normally received. The output of the compensating unit is selected and output to the subsequent processing unit.

Embodiment Overall Image Packet Encoding Apparatus Prior to description of an image packet decoding apparatus incorporating an image decoding apparatus according to an embodiment of the present invention, a corresponding image packet encoding apparatus will be described.

FIG. 4 shows the structure of the image packet coding apparatus, and FIGS. 5 and 6 show the detailed structures of the intra-frame coding unit and the inter-frame coding unit, respectively.

In FIG. 4, an image signal input from an input terminal 41 is QMF (Quadrature) as a two-dimensional band division filter.
The Mirror Filter unit 42 divides each frame into two bands of a high frequency component and a low frequency component in each of the horizontal direction and the vertical direction, and divides the entire frame into signal components of four bands.

The image signal components of the four bands obtained by the division are respectively assigned to the corresponding decimation units (thinning units) 43, 44, 4
5 and 46, the image signal components SLL, SLU, SUL, and SUU (subscripts are horizontal, In the vertical order, L indicates a low-frequency component and U indicates a high-frequency component).

An image signal component SLL, which is a low-frequency component in both the horizontal direction and the vertical direction, is provided to an intra-frame encoding unit 47 having a detailed configuration shown in FIG. 5, and other image signal components SLU, SUL,
The SUUs are respectively supplied to the corresponding motion compensation interframe coding units 48, 49 and 410 whose detailed configuration is shown in FIG. Each of the encoding units 47, 48, 49, and 410 encodes a given image signal component, and encodes the encoded value iLL, iLU, iLU, iUL, iUU
To the corresponding variable-length coding units 411, 412, 413, 414.

Each of the variable-length coding units 411, 412, 413, and 414 performs run-length / Huffman coding of the input code values iLL, iLU, iUL, and iUU, and outputs the result to the MUX unit (multiplexing unit) 415.

The MUX unit 415 multiplexes the input code into one series of signals and provides the multiplexed signal to the packet assembling unit 416, which assembles this into cells of a certain length. In this case, the type of information in the cell (such as which band the signal is in) and the parameters (block average value h and motion vectors MVLU, MVUL, MVUU) required for decoding are included as header information. Will be added.

Here, the reason why the intra-frame coding is used only for the image signal component SLL which is a low-frequency component in both the horizontal direction and the vertical direction is because such an image signal component SLL is an important component for an image. , This image signal component S
It is not good that LL is unstable for a long time due to cell discarding, etc., because the influence of cell discarding was kept in that frame by adopting intra-frame coding and it did not affect the next frame .

Details of Intra-Frame Encoding Unit Next, an intra-frame encoding unit that encodes an image signal component SLL that is a low-frequency component in both the horizontal and vertical directions.
47 will be described.

In FIG. 5, an input image signal component SLL is provided to a block dividing section 51. The block division unit 51 divides the image signal into several blocks each including n × n pixels on a frame basis, and supplies an image signal component of each block to the average value calculation unit 52 and the subtractor 53. The average value calculation unit 52 calculates the average value h in the block and supplies it to the subtractor 53. The subtractor 53 supplies a difference value dLL obtained by subtracting the intra-block average value h from the image signal component to an orthogonal transform unit 54 composed of, for example, a discrete cosine transformer. The orthogonal transformer 54 orthogonally transforms the difference value dLL and supplies it to the quantizer 55. The quantizer 55 quantizes the orthogonal transform coefficient to obtain an output code iLL.

The average value h in the block obtained by the average value calculation unit 52
Are transmitted to the image packet decoding device as parameters necessary for the decoding process, as described above.

Next, the image signal components SLU and SLU other than the image signal component SLL
UL, SUU interframe coding units 48, 49, 410
Will be described.

In FIG. 6, the input image signal component SLU, SUL or S
UU is provided to the block dividing unit 61. Block division unit 61
Divides an input image signal component into several blocks each composed of n × n pixels on a frame basis and supplies the block to a subtractor 62. The subtractor 62 subtracts the predicted value of the signal component provided from the motion compensator 63 from the signal component, and calculates a difference value dL.
U, dUL or dUU is given to an orthogonal transform unit 64 composed of, for example, a discrete cosine transform unit. The orthogonal transform unit 64 calculates the difference values dLU, dLU
The UL or dUU is orthogonally transformed for each block and given to a quantizer 65, which quantizes the orthogonal transform coefficient and outputs the obtained code iLU, iUL or iUU.

The code iLU, iUL, or iUU is provided to an inverse quantization unit 66, which performs inverse processing of the quantization unit 65, converts the code into local orthogonal transform coefficients, and performs inverse orthogonal transform 67 Given to. The inverse orthogonal transform unit 67 performs an inverse process of the orthogonal transform unit 64 on the local reproduction orthogonal transform coefficient, obtains a local reproduction difference value, and supplies the difference value to the adder 68. The adder 68 is supplied with the motion-compensated local reproduction orthogonal transform coefficient (prediction value) one frame before output from the motion compensator 63, and the adder 68 calculates the local reproduction orthogonal transform coefficient and the prediction value. The values are added to obtain a local reproduction image signal component, and the local reproduction image signal component is supplied to the leakage unit 69. The leakage part 69
After multiplying the image signal component of the local reproduction by a leak coefficient α (smaller than 1), the image signal component is given to the frame memory 610 and stored.

The image signal components stored in the frame memory 610 are used for processing of the next frame, and
And is given to the subtractor 62 as a predicted value as described above.

The motion compensator 63 is provided with an image signal component from the block divider 61 although signal lines are not shown, and the motion compensator 63 and the image signal component stored in the frame memory 610 one frame before , A motion vector (for example, a pair of horizontal and vertical motion vectors) is detected, and motion compensation is performed based on the motion vector. As described above, this motion vector information
The MVLU, MVUL, and MVUU are transmitted to the image packet decoding device as parameters necessary for the decoding process.

Next, an image packet decoding apparatus incorporating the image decoding apparatus according to an embodiment of the present invention will be described.

FIG. 7 shows the configuration of the image packet decoding apparatus, and FIGS. 1 and 8 show the detailed configurations of the intra-frame decoder and the inter-frame decoder, respectively.

In FIG. 7, cells from the transmission path are packet disassembly units.
It is given to 71 and decomposed. The packet decomposing unit 71 also detects a cell discard position. The data decomposed by the packet decomposing unit 71 is provided to a DEMUX unit (multiplex separation unit) 72.

The DEMUX unit 72 divides this into encoded image information and parameters (such as motion vectors) required for decoding,
The signal components are divided into signal components of each band and supplied to corresponding variable-length decoding units 73, 74, 75, and 76, respectively.

The variable-length decoding units 73, 74, 75, and 76 perform the inverse processing of the above-described variable-length coding units 411, 412, 413, and 414, and perform reproduction codes YLL, YLU, YUL, and YUU (iLL, iLU, and iUL, respectively). ,
corresponding to iUU) and corresponding decoding units 77, 78, 79, 710
Give to. The reproduction code YLL relating to the image signal components in the low band in both the horizontal and vertical directions is given to the intra-frame decoding unit 77 shown in FIG. 1 showing the detailed configuration, and the other reproduction codes YLU, YU
L and YUU are given to inter-frame decoding sections 78, 79 and 710 whose detailed configuration is shown in FIG.

As will be described later, the decoding units 77, 78, 79, and 710 decode the reproduced codes YLL, YLU, YUL, and YUU, and obtain the reproduced image signal components S'LL, S'LU,
S′UL and S′UU are provided to corresponding interpolation units (sample interpolation units) 711, 712, 713, and 714, respectively.

Each of the interpolation units 711, 712, 713, and 714 is provided with a given reproduced image signal component S'LL, S'LU,
S'UL and S'UU are upsampled twice in the horizontal and vertical directions, that is, four times in total, and provided to the QMF section 715. QMF section 7
15 filters up-sampled image signal components, then synthesizes signals of four bands and outputs them.
A reproduced image signal is output from the 716.

Details of Intra-Frame Decoding Unit Next, details of the above-described intra-frame decoding unit 77 that decodes the reproduction code YLL will be described.

In FIG. 1, the reproduced code YLL is given to an inverse quantizer 11. The inverse quantizer 11 performs inverse processing of the above-described quantizer 55 (FIG. 5) on the reproduced code YLL to obtain a reproduced orthogonal transform coefficient, and supplies the coefficient to the inverse orthogonal transform unit 12. The inverse orthogonal transform unit 12 performs inverse processing of the above-described orthogonal transform unit 54 on the reproduced orthogonal transform coefficient, obtains a reproduced difference value d′ LL, and supplies the difference value d′ LL to the adder 13. The in-block average value h extracted by the DEMUX unit 72 is also given to the adder 13.
Adds the reproduced difference value d'LL to the intra-block average value h to obtain a reproduced image signal component. This reproduced image signal component is applied to a switch circuit 14 having a two-input one-output configuration. The switch circuit 14 is supplied with a switching control signal from the DEMUX unit 72, for example, indicating whether or not the block involves cell discarding. When the block is not related to the cell discard, the switch circuit 14 selects the reproduced image signal component from the adder 13 and supplies it to the interpolation unit 711.

The intra-frame decoding unit 77 is provided with a compensation structure for cell discard. The reproduced image signal component output from the adder 13 is also supplied to the frame memory 15. The frame memory 15 stores a reproduced image signal component in case some cells related to the next frame are discarded. The reproduced image signal component delayed by one frame output from the frame memory 15 is supplied to the motion compensator 16. The motion compensator 16 is provided with motion vector information detected by the encoding device for bands other than the image signal component SLL. The motion compensating unit 16 performs motion compensation by shifting the reproduced image signal component of the previous frame from the frame memory 15 according to the motion vector information, and supplies the resultant to the switch circuit 14. The switch circuit 14 selects the reproduced image signal component from the motion compensation unit 16 for the block information lost due to the cell discard, and supplies it to the interpolation unit 711.

In this way, when the cell is discarded, the information of the previous frame stored in the frame memory 15 is subjected to motion compensation by considering the motion vector obtained when performing motion compensation inter-frame coding in another band. To compensate.

Details of Interframe Decoding Unit Next, details of the interframe decoding unit 78, 79, or 710 for the reproduced code YLU, YUL, or YUU will be described.

In FIG. 8, the reproduced code YLU, YUL or YUU is supplied to a switch circuit 81, and is selected by the switch circuit 81 and supplied to an inverse quantization unit 82 except when the cell is discarded. The reproduced code YLU, YUL or YUU is quantized by the inverse quantization
The inverse processing of 65 (FIG. 6) is performed to convert the data into reproduced orthogonal transform coefficients, which are provided to the inverse orthogonal transform unit 83. Inverse orthogonal transform unit 83
Performs an inverse process of the orthogonal transform unit 64 on the reproduced orthogonal transform coefficient to obtain a reproduced difference value d′ LU, d′ UL or d′ UU, and gives the value to the adder 84. Based on the motion vector information MVLU, MVUL, or MVUU output from the motion compensation unit 63 of the encoding device and given to the decoding device, the adder 84 includes one frame before the frame that has been motion-compensated by the motion compensation unit 85. A reproduction orthogonal transform coefficient (predicted value) is given. The adder 84 adds a reproduced difference value d'LU, d'UL or d'UU to the predicted value to obtain a reproduced image signal component S'LU, S'UL or S'UU. The component S′LU, S′UL or S′UU is output to the interpolation section 712, 713 or 714.

Also, a reproduced image signal component S'LU, S'UL or S'UU
Is given to the leakage part 86. The leakage part 86
After multiplying the reproduced image signal component S'LU, S'UL or S'UU by a leak coefficient α equal to the leak coefficient in the encoding device, the multiplied image signal is given to the frame memory 87 and stored.

The signal component stored in the frame memory 87 is used for processing of the next frame, is motion-compensated through the motion compensating unit 85, and is added as a predicted value as described above.
84.

Thus, the normal inter-frame decoding process is performed.

At the time of cell discard detection, the above-described switch circuit 81 is switched to the other input terminal side, and the block information (reproduction code) lost due to cell discard is set to 0. At this time,
Since the difference value given from the inverse orthogonal transformer 83 to the adder 84 is also 0, the reproduced image signal component S 'obtained by directly compensating the reproduced image signal component of the previous frame via the motion compensator 85 as it is.
Output as LU, S'UL or S'UU.

In general, when an error occurs during transmission in inter-frame coding, the frame memory 610 in the encoder and the frame memory 87 in the decoder become asynchronous, and there is a problem that the error propagates with time. . Therefore,
In this embodiment, by adding the above-described leakage units 69 and 86, the frame memories of the encoder and the decoder can be synchronized with time. Note that it is conceivable that the addition of the leakage units 69 and 86 increases the prediction error and lowers the coding efficiency. However, the provision of the leakage unit is not limited to the image signal components SLU, SUL or This is only for SUU and has relatively small power, so there is no practical problem.

Effects of Embodiment Since intra-frame encoding / decoding is applied to a low-frequency component that is an important component on an image, the effect of cell discard can be kept within that frame, and the next frame is affected. Can not be. In addition, since the frame in which the cells have been discarded is compensated by the reproduced image signal component of the previous frame using the frame memory, the deterioration of the image quality can be minimized.

By employing such an encoding / decoding method, it is not necessary to layer each cell, and it is not necessary to perform priority control on a transmission path.

Another embodiment The transmission form from the image encoding device to the image decoding device includes:
It is not limited to packet transmission. For example, a transmission form of recording and reproducing on a recording medium may be used.

Various methods can be applied as a method of transmitting parameters such as a motion vector and a block average value.

The motion vector information used in the intra-frame decoding unit is
The average of a plurality of pieces of motion vector information for image signal components in other bands may be used, or any one piece of motion vector information may be used. Further, a configuration may be provided in which the intra-frame encoding unit detects a motion vector for the component, and the component may be transmitted.

Further, a filter other than QMF may be applied as the band division filter, and a one-dimensional band division filter instead of a two-dimensional filter may be applied.

The present invention has a feature in a compensation method in intra-frame decoding at the time of loss in a transmission system of an encoded image signal, and performs intra-frame encoding on the entire input image signal. The same can be applied.

The intra-frame encoding method may be a method using a line correlation or a method using a correlation with an adjacent pixel, in addition to the method of obtaining the average value described above.

The motion compensation in the intra-frame decoding unit may be performed on the writing side to the frame memory, or may be performed on the reading side from the frame memory.

[Effects of the Invention] As described above, according to the present invention, when a loss or the like occurs in an intra-frame encoded image signal in a system from an image encoding device to an image decoding device, Since the memory is used to compensate for the reproduced image signal component of the previous frame, a high-quality reproduced image can be secured.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an intra-frame decoding unit in an image packet decoding apparatus according to one embodiment of the present invention.
FIG. 1 is a block diagram showing a conventional image packet encoding / decoding device, FIG. 3 is an explanatory diagram showing a method for extracting low-frequency components and high-frequency components in the device, and FIG. 4 is an image according to the above embodiment. FIG. 5 is a block diagram showing an image packet encoder corresponding to the packet decoder, FIG. 5 is a block diagram showing an intra-frame encoder shown in FIG. 4, and FIG. 6 shows an inter-frame encoder shown in FIG. FIG. 7 is a block diagram showing an image packet decoding apparatus according to the embodiment, and FIG.
The figure is a block diagram showing an inter-frame decoding unit. 11 ... inverse quantizer, 12 ... inverse orthogonal transformer, 13 ... adder, 14 ... switch circuit, 15 ... frame memory, 16 ...
... Motion compensator, 77 ... Intra-frame decoder.

 ──────────────────────────────────────────────────続 き Continuation of the front page (72) Inventor Kenichiro Hosoda 1-7-12 Toranomon, Minato-ku, Tokyo Oki Electric Industry Co., Ltd. (56) Reference IEICE Technical Report, IE 88-87, P. 45-52 1988 Image Coding Symposium (PCSJ88) 79-80 SPIE Vol. 707 Visual Communications and Image Processing (1986) p. 51-61

Claims (1)

(57) [Claims] An image decoding apparatus receives an image signal of a low-frequency component encoded in a frame,
An intra-frame decoding unit for intra-frame decoding the low-frequency component image signal; receiving an inter-frame encoded image signal of another band component; and inter-frame decoding the image signal of the other band component. An intra-frame decoding unit, wherein the intra-frame decoding unit is an intra-frame decoder for intra-frame decoding the received low-frequency component image signal,
A frame memory for storing the decoded low-frequency component image signal, and a motion compensation unit for performing motion compensation on the low-frequency component image signal stored in the frame memory according to a motion vector received from an encoding device. When the image signal of the low frequency component is normally received, the output of the intra-frame decoder is selected. When the image signal of the low frequency component cannot be normally received, the output of the motion compensation unit is An image decoding apparatus, comprising: a selection unit that selects an image and outputs the result to a subsequent processing unit.