JP2006203598A - Digital image decoder and decoding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a digital image decoder in which concealment performance can be enhanced for an error block, and to provide a digital image decoding method. <P>SOLUTION: The digital image decoder comprises a means for setting a motion compensation vector depending on an error detection result from an error detection means and a motion vector of each block stored in a motion vector storage means, a motion compensation decoding means outputting prediction reference image data based on the motion compensation vector, a means for outputting decoded image data by adding the prediction reference image data and difference decoding data, and a means for determining the panning of a video image represented by encoded image data. When the panning determination means determines a panned video image and the error detection means detects an error, the motion vector setting means sets a motion compensation vector based on a motion vector stored in the motion vector storage means of each of a plurality of normal blocks in the vicinity of the error block. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a digital image decoding apparatus and method for decoding encoded image data encoded for each block by dividing the image data into a plurality of blocks for each frame.

As a simple method of error concealment (error concealment processing) in MPEG image data, the motion of blocks lost due to an error is not taken into account, and a reference image such as an image of the previous frame is used for blocks at the same position. There is a method of fitting image data. In addition, as a method for taking motion into consideration, in the I picture, a block lost due to an error using a concealed motion vector is compensated by performing motion compensation from a reference image, and in a P or B picture, it is lost due to an error. There is a method of performing concealment by performing motion compensation from a reference image using a motion vector of a block near the block (see Patent Documents 1 and 2).
Japanese Patent No. 3396378 JP 2001-177836 A

  When the concealed motion vector is not encoded in the I picture, the motion vector is not encoded in all the blocks in the I picture, and the concealed block is used by using the motion vector of the neighboring block for the block lost due to the error. Can't do the same. Therefore, the image may be remarkably disturbed in a scene where movement is intense.

  On the other hand, in the P and B pictures, since the motion vector encoded in each block is only determined by the encoder to improve the compression efficiency of the image data, the motion vector value may vary from block to block. In the case where an image is constructed by simply performing motion compensation from a reference image using motion vectors of blocks near an error block, it may feel that a lot of block noise has occurred and the image may be significantly disturbed.

  Also, when an image is constructed by simply selecting the most frequent vector among the motion vectors of the neighboring blocks of the error block and performing motion compensation, for example, only two motion vector values of neighboring blocks are the same value. When disjoint, the selected motion vector value is unlikely to be the same value that should have been encoded in the error block, and it may refer to the image at the wrong position and the image may be significantly disturbed .

  The problems to be solved by the present invention include the above disadvantages as an example, and it is an object of the present invention to provide a digital image decoding apparatus and method capable of improving the performance of concealment for error blocks. .

  A digital image decoding apparatus according to a first aspect of the present invention is a digital image decoding apparatus that decodes encoded image data encoded for each block by dividing the image data into a plurality of blocks for each frame and for each block. Output decoding data indicating the difference between the image data of the current frame and the image data of the previous frame for each block, and outputting a motion vector in the image data of the current block, and an error of the image data Error detection means for detecting presence / absence of each block; motion vector storage means for storing the motion vector for each block; error detection result by the error detection means; and motion vector of each block stored in the motion vector storage means Motion vector setting means for setting a motion compensation motion vector in accordance with the motion vector, and the motion Motion-compensated decoding means for outputting predicted reference image data based on a compensation motion vector, addition means for adding the predicted reference image data and the differential decoded data to output decoded image data, and the encoded image data The motion vector setting means determines that the video has been parsed by the pann determination means and detects an error block with an error by the error detection means. In such a case, the motion compensation motion vector is set based on the motion vector stored in the motion vector storage means of each of a plurality of normal blocks that are not error blocks near the error block.

  A digital image decoding method according to a seventh aspect of the invention is a digital image decoding method in which image data is divided into a plurality of blocks for each frame and encoded image data encoded for each block is decoded for each block. A decoding step for outputting, for each block, differential decoded data indicating a difference between the image data of the current frame and the image data of the previous frame, and outputting a motion vector in the image data of the current block; and an error in the image data An error detection step for detecting presence or absence for each block; a motion vector storage step for storing the motion vector for each block; an error detection result in the error detection step; and a motion vector of each block stored in the motion vector storage step Motion vector to set motion compensation motion vector according to A setting step; a motion compensation decoding step for outputting predicted reference image data based on the motion compensation motion vector; and an adding step for adding the predicted reference image data and the difference decoded data to output decoded image data; A pann determining step for determining a panning of the video indicated by the encoded image data, wherein the motion vector setting step is determined to be a video that has been parsed in the pann determining step and an error is detected in the error detecting step. When a certain error block is detected, the motion compensation motion vector is set based on the motion vector stored in the motion vector storing step of each of a plurality of normal blocks that are not error blocks in the vicinity of the error block. It is characterized by doing.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing the configuration of a digital image decoding apparatus according to the present invention.

  As shown in FIG. 1, the digital image decoding apparatus includes a decoding unit 11, an error detection unit 12, a motion vector storage unit 13, a motion vector setting unit 14, a motion compensation decoding unit 15, an adder 16, a frame memory 17, and a panning determination. A unit 18 and a panic information storage unit 19 are provided.

  The decoding unit 11 performs a decoding process on the input encoded image data. The data encoding method is a method in which one frame is divided into a plurality of blocks and encoded for each block, and motion compensation prediction is performed for each block between temporally neighboring frames (for example, H.261, MPEG1). / 2 and vector quantization). The decoding unit 11 calculates a difference value between the previous frame and the current frame for each block in the decoding process, and outputs it to the adder 16 as difference decoded data. Further, the decoding unit 11 outputs the encoded data for error analysis to the error detection unit 12. The encoded data for error analysis may be only the header data of the encoded image data or the encoded image data itself, as long as the data has information that enables error detection. The decoding unit 11 outputs the motion compensation decoded vector of the currently decoded block to the motion vector storage unit 13.

  The error detection unit 12 detects whether or not there is an error in the block being decoded in the decoding unit 11 based on the encoded data for error analysis from the decoding unit 11. For example, during variable length decoding of the input encoded image data, the error analysis encoded data is analyzed to detect an error. As the analysis, a mismatch with the syntax defined in the standard or a codeword that is not in the defined variable-length codeword is detected. The error detection unit 12 outputs an error flag to the decoding unit 11 and the pann determination unit 18 when an error is detected.

  The motion vector storage unit 13 stores the motion compensation decoded vector output from the decoding unit 11 for each block. The motion vector storage unit 13 has a storage capacity for one frame, for example. When the error flag is supplied to the decoding unit 11 by the error detection unit 12, the motion vector storage unit 13 stores the vector 0 (that is, no motion vector) from the decoding unit 11.

  When receiving an error flag from the error detection unit 12, that is, when an error block is detected during decoding, the decoding unit 11 sets a motion vector for the error block to the vector setting unit 14. The instruction signal is issued as described above, and the difference decoded data is output as 0 to the adder 16.

  When a normal block that is not a normal error block is detected, the motion vector setting unit 14 reads out and outputs a motion vector stored in correspondence with the normal block in the motion vector storage unit 13, and indicates by error detection. When a signal is issued, the motion vector stored in the motion vector storage unit 13 in response to the instruction signal, that is, the motion vector of the block located immediately above the error block is set. This uses the tendency that neighboring blocks generally have a correlation in a moving image.

  If there is no error in the block, the motion compensated decoded vector decoded by the decoding unit 11 is stored in the motion vector storage unit 13 and then output to the vector setting unit 14 as it is and output to the motion compensation decoding unit 15.

  The motion compensation decoding unit 15 performs motion compensation on the image data of the previous frame read from the frame memory 17 that stores the previous frame (reference image) that is temporally adjacent in accordance with the input motion vector, and predicts the reference image. The data is output to the adder 16 as data.

  The adder 16 adds the difference decoded data output from the decoding unit 11 and the prediction reference image data output from the motion compensation decoding unit 15 in units of blocks. The addition result is output to the display device as decoded image data, and is output to the frame memory 17 when used for prediction of the next frame.

  The frame memory 17 stores decoded image data output from the adder 16 for use as a reference image in the next frame. The image data stored in the frame memory 17 is used as a reference image for motion compensation.

  The pan determination unit 18 creates a motion vector distribution of normal blocks in order to determine the entire frame of the video, that is, the pan video in which the video captured by the camera moves in a certain direction (for example, the left direction) If the frequency distribution satisfies a predetermined condition, it is determined that the image is panned. The predetermined condition is that the frequency of the motion vector value having the maximum frequency in the frequency distribution occupies a predetermined ratio (for example, 50%) or more. In addition, the motion vector value having the highest frequency is selected from the motion vectors for which the distribution has been created and stored in the panic information storage unit 19, and information on the pun determination result is also stored in the parse information storage unit 19. When it is determined that one line on the error block is panic, a motion vector on one line stored in the panic information storage unit 19 is set in the motion vector setting unit 14 for the error block.

  In the panning determination by the panning determination unit 18, if the frequency distribution satisfies a predetermined condition, the image is not limited to an image in which the entire frame is moving, but a still image or a part of the frame, for example, in a stationary background. An image in which a small object is moving may also be determined as a pan.

  Next, the operation of the digital image decoding apparatus having such a configuration will be described. Here, in order to simplify the description, a case of P picture encoded image data having only forward prediction will be described. In the P picture, only forward prediction in which motion compensation is performed from a past reference image exists.

  As shown in FIG. 2, the motion vector setting unit 14 reads information on whether or not it has been determined as a PAN stored in the PAN information storage unit 19 in response to the instruction signal from the decoding unit 11, and according to the information. Then, it is determined whether or not it is determined to be pann (step S1). When it is determined by the pan determining unit 18 that one line of the error block is parsed, the motion vector setting unit 14 stores the motion vector on one line stored in the parse information storage unit 19 for the error block. (Motion compensation motion vector) is set (step S2).

  FIG. 3 exemplifies a block sequence in the case where the panning determination is performed using the motion vector of the normal block located one line above the line including the error block. In FIG. 3, the arrows shown in the normal blocks represent the motion vector values, and the motion vectors indicated by the arrows A have the same value. For such normal block line motion vector values, the frequency distribution as shown in the distribution diagram of FIG. When the frequency distribution satisfies the predetermined condition in the frequency distribution, the pan determination unit 18 determines that it is a parse, selects the motion vector having the highest frequency indicated by the symbol B as shown in FIG. For each line in which the distribution is created, it is stored in the pann information storage unit 19.

  FIG. 5 shows an example in which the selected motion vector is applied to the error block when it is determined by the panic determination unit 18 based on the motion vector of each block on one line of the error block. If it is determined that the block on one line of the error block is a parsed image, the motion vector on one line stored in the parse information storage unit 19 is transferred to the motion vector setting unit 14 for the error block. Is set. In FIG. 5, the motion vector indicated by the symbol A among the motion vectors of a plurality of blocks on one line is selected, and the selected motion vector is applied to the error block.

  The motion compensation decoding unit 15 performs concealment processing by performing motion compensation according to the motion vector set by the motion vector setting unit 14 (step S4). That is, the motion compensation decoding unit 15 reads the image data of the previous frame from the frame memory 17 according to the motion vector set in the motion vector setting unit 14, performs motion compensation on the read image data, and adds it as predicted reference image data. 16 is output. In the adder 16, the differential decoded data indicating 0 output from the decoding unit 11 and the prediction reference image data output from the motion compensation decoding unit 15 are added, and the addition result is output to the display device as decoded image data. At the same time, it is output to the frame memory 17 when used for referring to the next frame.

  As described above, the panning is detected using the motion vector of the block on one line of the error block, and when it is determined that the block is a panning, the same motion vector value is applied to each error block. Unlike the above method, it is possible to eliminate the image shift between the blocks and reduce the deterioration of the image.

  On the other hand, if it is determined by the panic determination unit 18 that one line of the error block is not parsed, the motion vector setting unit 14 moves the motion vector stored in the motion vector storage unit 13, that is, directly above the error block. A motion vector (motion compensation motion vector) of the located block is set (step S3). When one line on the error block is not parsed, the concealment process is performed by performing motion compensation in accordance with the motion vector set by the motion vector setting unit 14 as in the case where it is determined to be parsed. Performed (step S4).

  In the above-described embodiment, in the case of encoded image data such as a P picture in MPEG2, there is a block in which a motion vector is not encoded when an image is configured using an image at the same position as the predicted reference image. In such a case, the motion vector is set to “0” and the frequency distribution of FIG. 4 is created.

  In the above-described embodiment, the motion vector frequency distribution is created. However, it is also possible to create a frequency distribution for each value in the X and Y (horizontal and vertical) directions. In this case, it is determined that the Pann is a Pann when a predetermined condition is satisfied for each distribution in the X and Y directions. At this time, the values in the X and Y directions are stored in the panic information storage unit 19. The motion vector for the error block is also applied in each of the X and Y directions.

  Furthermore, in the above-described embodiment, the distribution map is created with the motion vector of the block above the error block, but the frequency distribution is created with the motion vector of the block in the same column as the error block and the motion vector of the block below the error block. You may do it. Also, a motion vector frequency distribution in an arbitrary range in the vicinity of the error block, not in units of lines, may be created.

  Further, in the above-described embodiment, it is determined whether or not panning is detected only in the motion vector frequency distribution, but no error block or motion vector is encoded in the block in the range in which the frequency distribution is created. Blocks may also be included. If there are many such blocks, there is a risk of erroneous detection if the determination of the Pann detection is made only with the frequency distribution. Therefore, when the number of blocks in which motion vectors are encoded in a range for creating a frequency distribution does not satisfy a predetermined condition, it can be determined that the data is not parsed.

In the above embodiment, the case of a P picture in which only forward prediction exists is described. However, in the case of a B picture in which forward prediction and backward prediction exist, motion compensation can be performed as follows. In the case of a B picture, there are forward prediction in which motion compensation is performed from a past reference image and backward prediction in which motion compensation is performed from a future reference image. A motion vector exists in each of the forward prediction and backward prediction, but when only forward prediction is encoded for each block, when only backward prediction is encoded, both forward prediction and backward prediction are encoded. There are four types of cases where both are encoded and not encoded. The creation of the frequency distribution of each motion vector is the same as the method shown in the above embodiment, but a selection means for selecting one of the following (1) and (2) is required.
(1) Using the pann information obtained from the motion vector of the forward prediction, motion compensation is performed from the past reference image, and the error block is concealed.
(2) Using the pann information obtained from the motion vector of backward prediction, motion compensation is performed from a future reference image, and an error block is concealed.

  As a selection method of the selection means, there is a method of selecting one of (1) and (2) which is determined in advance. There is also a method of selecting on the condition that the number of blocks in which forward prediction is encoded and the number of blocks in which backward prediction is encoded. Specifically, if the number of blocks in which forward prediction is encoded is larger, select (1), and if the number of blocks in which backward prediction is encoded is larger (2 ) Is performed. Further, there is a method of selecting pann information from a reference image that is close in time. Specifically, (1) is selected when the current picture is temporally close to a past reference image, and (2) is selected when the current picture is temporally close to a future reference image. Is called.

  In creating the motion vector frequency distribution in the above-described embodiment, the frequency distribution may be created using the motion vector used for motion compensation. In encoded image data such as a B picture in MPEG2, the current block may be configured from a predicted reference image using the motion vector used in the previous block, even if the motion vector is not encoded in the current block. In such a case, a frequency distribution as shown in FIG. 4 can be created using the motion vector used for motion compensation.

  In the above-described embodiment, when the concealed motion vector is not encoded in the I picture, the motion vector value is not encoded in each block, and the motion vector distribution as in the above-described embodiment. Cannot be created. Therefore, the Pern information of the P picture immediately before the I picture is stored, and motion compensation is performed using, for example, a motion vector of the Pane information on the same line as the error block to perform concealment. This makes it possible to perform concealment from the reference image in consideration of the motion even in the case of an I picture in which the motion vector is not encoded, such as in the case of an image that has been panned and moved, thereby reducing image degradation. Is possible.

  As described above, according to the present invention, the motion vector setting means for setting the motion compensation motion vector according to the error detection result by the error detection means and the motion vector of each block stored in the motion vector storage means, Motion compensation decoding means for outputting predicted reference image data based on a compensation motion vector, addition means for adding predicted reference image data and difference decoded data to output decoded image data, and video indicated by the encoded image data When the motion vector setting means determines that the video is parsed by the pann determination means and the error detection means detects an error block with an error, the motion vector setting means Motion vector storage means for each of a plurality of normal blocks that are not error blocks near the error block Motion compensation motion vectors based on the stored motion vectors is set. Therefore, the performance of concealment with respect to an error block can be improved.

It is a block diagram which shows the Example of this invention. 3 is a flowchart showing a procedure of error processing by a motion vector setting unit and a motion compensation decoding unit in the apparatus of FIG. 1. It is a figure which shows the case where a panning determination is performed using the motion vector of the normal block located only one line above the error block. It is a figure which shows frequency distribution of a motion vector value. It is a figure which shows application to the error block of the motion vector selected when it determines with it being a pan.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Decoding part 12 Error detection part 13 Motion vector storage part 14 Motion vector setting part 15 Motion compensation decoding part 18 Pann determination part 19 Pann information storage part

Claims (7)

A digital image decoding apparatus that decodes image data divided into a plurality of blocks for each frame and encoded image data encoded for each block, for each block,
Decoding means for outputting difference decoded data indicating a difference between image data of the current frame and image data of the previous frame for each block and outputting a motion vector in the image data of the current block;
Error detection means for detecting the presence or absence of an error in the image data for each block;
Motion vector storage means for storing the motion vector for each block;
Motion vector setting means for setting a motion compensation motion vector according to the error detection result by the error detection means and the motion vector of each block stored in the motion vector storage means;
Motion compensation decoding means for outputting predicted reference image data based on the motion compensation motion vector;
Adding means for adding the predicted reference image data and the differential decoded data to output decoded image data;
Pann determination means for determining the pann of the video indicated by the encoded image data,
When the motion vector setting means determines that the video is parsed by the panic determination means and an error block with an error is detected by the error detection means, the motion vector setting means is not an error block near the error block. A digital image decoding apparatus, wherein the motion compensation motion vector based on the motion vector stored in the motion vector storage means of each normal block is set. The Pann determination means creates a frequency distribution of motion vectors of normal blocks having no error within a predetermined range in the frame, and determines that the current frame is a parsed video when the frequency distribution satisfies a predetermined condition. A determination unit;
A pann information storage unit that stores a motion vector value of the maximum frequency of the frequency distribution for each of the predetermined ranges when it is determined by the determination unit that the image has been parsed;
The motion vector setting means is a predetermined range located in the vicinity of the error block when it is determined that the video is parsed by the determination section and an error block with an error is detected by the error detection means. 2. The digital image decoding apparatus according to claim 1, wherein a motion vector value having a maximum frequency is read from the Pann information storage unit and set as the motion compensation motion vector.   3. The digital image decoding apparatus according to claim 2, wherein the predetermined condition is that a frequency of a motion vector value having a maximum frequency in the frequency distribution occupies a predetermined ratio or more.   3. The digital image decoding according to claim 2, wherein the predetermined range is one line for each block, and the predetermined range located in the vicinity of the error block is a line one line before the error block. apparatus.   The motion vector setting means sets the motion vector of the block stored in the motion vector storage means as the motion compensation motion vector for the normal block detected as having no error by the error detection means, When an error block is detected as having an error by the error detecting means when it is not determined to be a pan by the pan determining means, a normal block that is not an error block in the vicinity of the error block stored in the motion vector storing means is detected. 2. The digital image decoding apparatus according to claim 1, wherein the motion compensation motion vector is predicted and set based on a motion vector. The motion compensation decoding means includes a frame memory for storing the decoded image data for each block for each frame;
A motion compensation demodulator that creates the predicted reference image data by performing motion compensation on the decoded image data of the current block of the previous frame stored in the frame memory in accordance with the motion compensation motion vector. The digital image decoding apparatus according to claim 1, wherein the digital image decoding apparatus is characterized. A digital image decoding method for decoding, for each block, encoded image data encoded for each block by dividing the image data into a plurality of blocks for each frame,
A decoding step of outputting differential motion data indicating the difference between the image data of the current frame and the image data of the previous frame for each block and outputting a motion vector in the image data of the current block;
An error detection step of detecting the presence or absence of an error in the image data for each block;
A motion vector storing step for storing the motion vector for each block;
A motion vector setting step for setting a motion compensation motion vector according to the error detection result in the error detection step and the motion vector of each block stored in the motion vector storage step;
A motion compensation decoding step of outputting predicted reference image data based on the motion compensation motion vector;
An adding step of adding the predicted reference image data and the differential decoded data to output decoded image data;
A pann determining step for determining a pann of the video indicated by the encoded image data,
In the motion vector setting step, when it is determined that the video is parsed in the panning determination step and an error block with an error is detected in the error detection step, a plurality of error blocks that are not in the vicinity of the error block are included. A digital image decoding method comprising: setting the motion compensation motion vector based on the motion vector stored in the motion vector storing step of each normal block.