JP5701018B2 - Image decoding device - Google Patents

Description

  The present invention relates to an image decoding apparatus having a function of performing concealment processing for concealing a location where an error has occurred when an error occurs during image decoding processing.

As an encoding technique for compressing moving image data with high efficiency, for example, MPEG-2 or H.264, which is an international standard, is used. H.264 / AVC.
These encoding techniques are hybrid encoding schemes in which motion compensation prediction processing, discrete cosine transform processing, variable length encoding processing, and the like are combined.
The type of picture to be encoded is classified according to the method of motion compensation prediction processing.
Specifically, motion-compensated prediction between an intra-coded picture (hereinafter referred to as “I picture”) that is a picture that has been subjected to intra-frame prediction and compressed, and a temporally forward frame. Between a forward prediction coded picture (hereinafter referred to as a “P picture”) that is a compressed picture that has been implemented and is temporally forward, backward, or bidirectional. The pictures are classified into three types: bi-predictive coded pictures (hereinafter referred to as B pictures), which are pictures that have been subjected to compensation prediction and compressed.
In general, encoding is performed by periodically repeating a group of pictures called GOP having an I picture at the head by performing these prediction processes.

The encoded data of the moving image is transmitted from the image encoding device through a transmission path such as a network, for example, and the receiving side image decoding device performs a decoding process on the encoded data so that the original moving image is obtained. Can be obtained.
However, if an error or omission of encoded data occurs on the transmission path, an error occurs during the decoding process of the encoded data, and the original image cannot be correctly decoded. The decoded image may be significantly degraded until the head is received. Alternatively, the decoding process may not be continued.
If the picture in which the error has occurred is an I picture or a P picture, the error is also propagated to a picture that is to be decoded using that picture as a reference picture, so the head of the next GOP is received. Until then, the decoded image continues to deteriorate significantly.

In recent image decoding apparatuses, even if errors or omissions in encoded data occur on a transmission path and an error occurs during the decoding process of the encoded data, an error is detected as a means for reducing degradation of the decoded image. Some have implemented an error concealment function that hides the location where the error occurs.
For example, as an error concealment method for an I picture having a large influence of error, a method of displaying a decoded image by replacing an image at an error occurrence position in a decoded image with an image at the same position of the immediately preceding I picture / P picture, motion A method for displaying a decoded image by replacing it with the image of the previous I picture / P picture indicated by the motion vector using information, a method for displaying a decoded image by replacing with an image decoded from only a DC (direct current) component, etc. There is.

However, in the method of displaying the forward image in terms of time, a scene change or the like occurs, and when the correlation with the forward image is low, the decoded image is greatly deteriorated.
Further, in the method of decoding only from the DC component, when the correlation with the forward image is high, it is possible to obtain a decoded image with less visual discomfort by performing error concealment using the forward image. Therefore, it cannot be said that it is appropriate as an error concealment method.
Therefore, in the image decoding device disclosed in Patent Document 1 below, if an error occurs during the decoding process of the encoded data, the DC value, the magnitude of the motion vector, and the change in the motion vector obtained by the decoding process are changed. It is determined whether or not the amount is within a predetermined range, and the error concealment method to be used is determined according to the determination result.

JP 2001-69514 A (pages 9 to 16, FIG. 1, FIG. 7, FIG. 9)

Since the conventional image decoding apparatus is configured as described above, the error concealment method is switched for each macroblock to be encoded on the basis of a motion vector or the like. In the image encoding apparatus, for example, When a motion vector is calculated with an emphasis on coding efficiency, the motion vector may be different for each macroblock, unlike an actual motion. In such a case, when the error concealment method is switched on the basis of a motion vector or the like, there is a problem that image disturbance occurs in a fine unit and subjective deterioration becomes large.
Also, due to the effect of variable length coding processing in the image coding device, the macro block where the error actually occurs may be different from the macro block where the error is detected, but the error actually occurs. If the error concealment method based on the DC value is selected when the position from the macro block to the position of the macro block where the error is detected extends over a plurality of macro block lines, the decoded image is degraded. There was a growing problem.

  The present invention has been made to solve the above-described problems, and is less affected by variable length encoding processing in the image encoding apparatus, while maintaining the tracking capability with respect to motion and with less subjective deterioration. An object of the present invention is to obtain an image decoding apparatus capable of obtaining a decoded image.

When an error is detected during the decoding process of the image decoding means, the image decoding apparatus according to the present invention determines the entire slice including the macroblock in which the error has occurred as a concealment target, and selects the macroblock as a macroblock. The concealment target determining means for determining the entire picture in which an error is propagated from the included picture as the concealment target, and the picture type included in the encoding parameter output from the image decoding means, Reference data determination means for determining a reference decoded image and encoding parameter used for concealment processing to be concealed from among the stored decoded images and encoding parameters, and a code determined by the reference data determination means Motion vector for each macroblock included in the activation parameter A motion vector calculation means for calculating a motion vector for a concealment processing unit composed of a plurality of macroblocks using a data block, and a motion for each macroblock included in the encoding parameter determined by the reference data determination means The concealment method used for the concealment processing is referred to by referring to the vector and macroblock type and the motion vector of the concealment processing unit calculated by the motion vector calculating means , and the interframe concealment using the image data and the motion vector between the frames. a concealment method determination means for determining a frame concealment processing system using the image data of the instrument processing method or in the same frame provided, the concealment processing unit, which is determined by the concealment method determination means concealing The concealment process for concealing the concealment target determined by the concealment target determination unit is performed in accordance with the reference method, referring to the decoded image for reference determined by the reference data determination unit as necessary. It is a thing.

According to the present invention, when an error is detected during the decoding process of the image decoding means, the entire slice including the macroblock in which the error has occurred is determined as a concealment target, and from the picture including the macroblock. With reference to the picture type included in the encoding parameter output from the image decoding unit, the concealment target determining unit that determines the entire picture in which the error is propagated as the concealment target, and the data is stored in the data storage unit. Among the decoded image and the encoding parameter, the reference data determining means for determining the reference decoded image and the encoding parameter used for the concealment process to be concealed, and the encoding parameter determined by the reference data determining means. Using the included macroblock-based motion vector Motion vector calculation means for calculating a motion vector for a concealment processing unit composed of a plurality of macroblocks, and a motion vector and macroblock type for each macroblock included in the encoding parameter determined by the reference data determination means The concealment method used for the concealment process with reference to the motion vector of the concealment processing unit calculated by the motion vector calculating means is the same as the inter-frame concealment processing method using the image data and the motion vector between the frames. a concealment method determination means for determining a frame concealment processing system using the image data in the frame is provided, the concealment processing means, the concealment method determined by the concealment method determination means Therefore, the concealment process for concealing the concealment target determined by the concealment target determination unit is performed while referring to the decoded image for reference determined by the reference data determination unit as necessary. Therefore, there is an effect that it is possible to obtain a decoded image with less subjective deterioration while maintaining the followability to the motion without being affected by the variable length coding process in the image coding apparatus.

It is a block diagram which shows the image decoding apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows an example of the macroblock in which the error has generate | occur | produced, and the slice of concealment object. It is explanatory drawing which shows the example of an error propagation. It is explanatory drawing which shows an example of the decoded image for a reference in the case of performing an inter-frame concealment process using a motion vector, and an encoding parameter. It is explanatory drawing which shows the example of calculation of the motion vector of the concealment process unit in case a concealment object is a B picture. It is explanatory drawing which shows the determination conditions of a concealment system. It is a block diagram which shows the image decoding apparatus by Embodiment 3 of this invention. It is a block diagram which shows the image decoding apparatus by Embodiment 4 of this invention.

Embodiment 1 FIG.
1 is a block diagram showing an image decoding apparatus according to Embodiment 1 of the present invention.
In FIG. 1, an image decoding unit 1 is, for example, an international standard such as MPEG-2 or H.264. The decoder is applied to a standard such as H.264 / AVC. When encoded data of an image generated by performing motion compensation prediction processing and variable length encoding processing is input from an image encoding device, the encoding is performed. Perform decoding processing on the data, and include the decoded image that is the decoding result and encoding parameters (for example, picture type, motion vector and macroblock type in units of macroblocks, position information indicating the first macroblock of the slice, etc.) ) Is output.
Further, when detecting an error during the decoding process, the image decoding unit 1 performs a process of outputting error information including the position information of the macro block in which the error has occurred, an error detection flag, and the like.
The image decoding unit 1 constitutes an image decoding unit.

  The frame memory unit 2 is a recording medium such as a hard disk or a RAM for storing the decoded image and the encoding parameter output from the image decoding unit 1 in units of pictures. The frame memory unit 2 constitutes data storage means.

The concealment target determination unit 3 is composed of, for example, a semiconductor integrated circuit mounted with a CPU or a one-chip microcomputer, and the error detection flag included in the error information output from the image decoding unit 1 is significant. (When indicating that an error has been detected), the position information included in the error information and the first macroblock of the slice included in the encoding parameter output from the image decoding unit 1 The slice including the macroblock in which the error has occurred is identified, and the process of determining the entire slice as a concealment target is performed.
Further, the concealment target determining unit 3 refers to the picture type included in the encoding parameter, and an error is propagated from a picture including a macroblock in which an error has occurred (hereinafter referred to as “error occurrence picture”). A picture to be identified (hereinafter referred to as “error propagation picture”) is identified, and the entire error propagation picture is determined as a concealment target.
In addition, the concealment object determination part 3 comprises the concealment object determination means.

  The reference frame determination unit 4 is composed of, for example, a semiconductor integrated circuit mounted with a CPU or a one-chip microcomputer, and the error detection flag included in the error information output from the image decoding unit 1 is significant. In some cases, the concealment target determination unit is selected from the decoded image and the encoding parameter stored in the frame memory unit 2 with reference to the picture type included in the encoding parameter output from the image decoding unit 1. The reference decoded image and the encoding parameter used for the concealment processing of the concealment object determined by 3 are determined, and the reference decoded image and the encoding parameter and the concealment object image (the decoded image including the concealment object) are determined. The output instruction is output to the frame memory unit 2. The reference frame determination unit 4 constitutes reference data determination means.

  The estimated motion vector calculation unit 5 is composed of, for example, a semiconductor integrated circuit on which a CPU is mounted or a one-chip microcomputer, and is output from the frame memory unit 2 under an output instruction from the reference frame determination unit 4 A process of calculating a motion vector in a concealment process unit composed of a plurality of macroblocks is performed using the macroblock unit motion vector and the macroblock type included in the encoding parameter. The estimated motion vector calculation unit 5 constitutes a motion vector calculation unit.

  The concealment method determination unit 6 is composed of, for example, a semiconductor integrated circuit on which a CPU is mounted or a one-chip microcomputer, and the macroblock unit included in the encoding parameter output from the frame memory unit 2 With reference to the motion vector and macroblock type and the motion vector of the concealment processing unit calculated by the estimated motion vector calculation unit 5, processing for determining the concealment method used for the concealment processing is performed. The concealment method determination unit 6 constitutes a concealment method determination means.

The concealment processing unit 7 is composed of, for example, a semiconductor integrated circuit on which a CPU is mounted, a one-chip microcomputer, or the like, and the reference frame determining unit 4 according to the concealment method determined by the concealment method determining unit 6. The concealment for concealing the concealment target in the concealment target image output from the frame memory unit 2 while referring to the decoded image for reference output from the frame memory unit 2 as necessary under the output instruction Process.
That is, when the concealment method determined by the concealment method determination unit 6 is an inter-frame concealment processing method, the concealment processing unit 7 calculates a reference decoded image output from the frame memory unit 2 and an estimated motion vector. The concealment processing for hiding the concealment target in the concealment target image output from the frame memory unit 2 is performed using the motion vector of the concealment processing unit calculated by the unit 5, and the concealment method is a frame In the case of the inner concealment processing method, the concealment processing for concealing the concealment target in the concealment target image is performed using the macroblocks around the concealment target determined by the concealment target determination unit 3. .
The concealment processing unit 7 constitutes concealment processing means.

  In the example of FIG. 1, an image decoding unit 1, a frame memory unit 2, a concealment target determination unit 3, a reference frame determination unit 4, an estimated motion vector calculation unit 5, and a concealment method determination unit 6 that are components of the image decoding apparatus. And the concealment processing unit 7 are assumed to be configured by dedicated hardware, but when the image decoding device is configured by a computer, the image decoding unit 1, the frame memory unit 2, and the concealment target A program describing the processing contents of the determination unit 3, the reference frame determination unit 4, the estimated motion vector calculation unit 5, the concealment method determination unit 6 and the concealment processing unit 7 is stored in the memory of the computer, and the CPU of the computer May execute a program stored in the memory.

Next, the operation will be described.
When the image decoding unit 1 receives encoded data of an image generated by performing motion compensation prediction processing and variable length encoding processing from the image encoding device, the encoding method included in the encoded data With reference to the information, the encoding method in the image encoding device is recognized, and the decoding process is performed on the encoded data in accordance with the encoding method.
The image decoding unit 1 includes a decoded image obtained by decoding the encoded data and an encoding parameter (for example, a picture type, a motion vector and macroblock type in units of macroblocks, and position information indicating a macroblock at the beginning of a slice) And the like are output to the frame memory unit 2.
As a result, the frame memory unit 2 stores the decoded image and the encoding parameter output from the image decoding unit 1 for one frame time or more for each picture.

In addition, the image decoding unit 1 performs error detection processing during the decoding processing of the encoded data, and obtains error information and encoding parameters including the position information of the macro block in which the error has occurred and an error detection flag. The data is output to the concealment object determination unit 3 and the reference frame determination unit 4.
Note that the image decoding unit 1 detects, for example, a mismatch with the syntax defined in the encoding method standard or a syntax analysis error during the decoding process of the encoded data, or the decoded value is a predetermined value. Detect error when out of range.

When the error information is output from the image decoding unit 1 and the error detection flag included in the error information is significant (indicating that an error has been detected), the concealment target determination unit 3 By referring to the position information included in the error information and the position information indicating the first macroblock of the slice included in the encoding parameter output from the image decoding unit 1, the macroblock in which the error has occurred is determined. The slice to be included is specified, and the entire slice is determined as a concealment target.
Here, FIG. 2 is an explanatory diagram illustrating an example of a macroblock in which an error has occurred and a slice to be concealed.
In the example of FIG. 2, since the macro block in which an error has occurred is included in the second slice from the top, by specifying the first and last macro blocks of the second slice from the top, The entire second slice is determined as a concealment target.

  As described above, the concealment target determination unit 3 determines the entire slice including the macro block in which the error has occurred as the concealment target. The reason is that if an error occurs during the decoding process, an error occurs. The error continues from the macroblock where the error occurred to the beginning of the next coding unit (next slice), and the macroblock where the error actually occurred is different from the macroblock where the error is detected. Because there may be.

Further, the concealment target determination unit 3 refers to the picture type included in the encoding parameter, and specifies an error propagation picture in which an error is propagated from an error occurrence picture including a macroblock in which an error has occurred. The entire error propagation picture is determined as a concealment target.
That is, when the error-occurring picture is an I picture or a P picture, as shown in FIG. 3, the error also propagates to a decoded picture that refers to that picture by motion compensation prediction.
Therefore, when the picture from which an error is detected is an I picture or a P picture, all the pictures from the subsequent picture to the next I picture are determined as concealment targets. Further, the B picture between the I picture in which an error is detected or the I picture next to the P picture and the subsequent P picture is determined as a concealment target.
When the concealment target determination unit 3 determines the concealment target, the concealment target determination unit 3 outputs the concealment target information indicating the concealment target to the reference frame determination unit 4, the estimated motion vector calculation unit 5, and the concealment processing unit 7. As for concealment target information for a picture in which no error has occurred, zero is output.

When the error detection flag included in the error information output from the image decoding unit 1 is significant, the reference frame determination unit 4 determines the picture type included in the encoding parameter output from the image decoding unit 1. Referring to the decoded image and encoding for reference used in the concealment target concealment process determined by the concealment target determination unit 3 from the decoded image and encoding parameters stored in the frame memory unit 2 Determine the parameters.
Here, FIG. 4 is an explanatory diagram showing an example of a decoded image for reference and an encoding parameter when performing an interframe concealment process using a motion vector.

When the error occurrence picture or the error propagation picture is an I picture or a P picture, as shown in FIG. 4A, as the decoded picture for reference used in the interframe concealment process, the I picture immediately before in the coding order Or a P picture is used.
In addition, as an encoding parameter used when the estimated motion vector calculation unit 5 calculates the estimated motion vector, the encoding parameter of the immediately preceding P picture is used for an error-occurring picture, and the own picture is used for an error propagation picture. used.
The reason is that the coding parameter of the error occurrence picture is likely to be incorrect, but the coding parameter of the error propagation picture is correctly decoded.

When the error occurrence picture or the error propagation picture is a B picture, as shown in FIG. 4B, as a reference decoded image used in the inter-frame concealment process, two I frames in the forward direction in the coding order are used. A picture or a P picture (two I pictures or P pictures in forward and backward directions in display order) is used.
Also, as the encoding parameter used when the estimated motion vector calculation unit 5 calculates the estimated motion vector, the encoding parameter of the immediately preceding P picture in the encoding order (backward P picture in the display order) is used. Is done.

When the reference frame determining unit 4 determines a reference decoded image and encoding parameter to be used for the concealment process, the reference frame determining unit 4 gives an instruction to output the decoded image to the concealment processing unit 7 to the frame memory unit 2, and the encoding parameter To the estimated motion vector calculation unit 5 and the concealment method determination unit 6 is given to the frame memory unit 2.
In addition, the reference frame determination unit 4 gives the frame memory unit 2 an instruction to output the concealment target image (a decoded image including the concealment target) to the concealment processing unit 7.
Thereby, the concealment processing unit 7 is provided with the decoded image for reference and the concealment target image, and the estimated motion vector calculation unit 5 and the concealment method determination unit 6 are provided with the encoding parameters.

When the estimated motion vector calculation unit 5 receives the encoding parameter from the frame memory unit 2 under the output instruction of the reference frame determination unit 4, the estimated motion vector calculation unit 5 includes a macroblock unit motion vector and macroblock included in the encoding parameter. Using the type, a motion vector of a concealment processing unit composed of a plurality of macroblocks is calculated.
In other words, the estimated motion vector calculation unit 5 uses the motion vector in units of macroblocks, and when the inter-frame concealment process is performed, the disturbance of the image occurs in a fine unit when the motion vector is not hit. Therefore, an estimated motion vector is calculated for each concealment process using a plurality of macroblocks within the concealment target (for example, 4 macroblocks in the horizontal direction × 4 macroblocks in the vertical direction) as a concealment process unit. ing.

The estimated motion vector calculation unit 5 calculates the motion vector of the concealment processing unit based on the size of the motion vector of the plurality of macroblocks constituting the concealment processing unit. The vector calculation method will be specifically described.
For example, a median (intermediate value) is obtained for each horizontal component and vertical component of motion vectors of a plurality of macroblocks constituting a concealment processing unit, and the median is set as a motion vector of the concealment processing unit.
However, when the macroblock type of the macroblock is intra prediction, the motion vector of the macroblock is excluded from the target for calculating the median.

Here, an example is shown in which the estimated motion vector calculation unit 5 obtains a median for each of the horizontal and vertical components of the motion vectors of a plurality of macroblocks. However, the motion of a plurality of macroblocks constituting a concealment processing unit is shown. An average value of vectors may be obtained, and the average value may be used as a motion vector for a concealment processing unit.
However, after obtaining an average value of motion vectors of a plurality of macroblocks constituting a concealment processing unit, a difference value between those motion vectors and the average value is obtained, and the difference value is larger than a predetermined threshold value. Excluding the motion vector, the average value of the remaining motion vectors is calculated again, and the average value is used as the motion vector of the concealment processing unit.

  Further, the estimated motion vector calculation unit 5 classifies the motion vectors of a plurality of macroblocks constituting a concealment processing unit into a plurality of classes according to the size, and belongs to a motion belonging to the plurality of classes. The class having the largest number of vectors may be selected, and the median, average value, or value of the motion vector closest to the average value may be used as the motion vector of the concealment processing unit.

When the estimated motion vector calculation unit 5 calculates the motion vector for each concealment process, the estimated motion vector information including the motion vector and position information (information indicating the position where the concealment process is performed) is sent to the concealment method determination unit 6. Output.
However, if the concealment target is an I picture or a P picture with reference to the picture type included in the encoding parameter output from the frame memory unit 2, the concealment processing unit calculated as described above is used. If the concealment target is a B picture, the calculated motion vector of the concealment processing unit is expressed as the preceding or following I picture or P picture in the display order. Depending on the display time difference, the motion vector is replaced with a bidirectional motion vector (forward motion vector, backward motion vector), and the bidirectional motion vector is included in the estimated motion vector information as a motion vector for concealment processing. Output.

Here, FIG. 5 is an explanatory diagram illustrating an example of calculating a motion vector in units of concealment processing when the concealment target is a B picture.
In the example of FIG. 5, the forward motion vector MV1 and the backward motion vector MV2 are calculated using the motion vector of the concealment processing unit calculated from the motion vectors of a plurality of macroblocks constituting the concealment processing unit as MVcol. doing.
MV1 = T1 / (T1 + T2) × MVcol
MV2 = T2 / (T1 + T2) × (−MVcol)
However, T1 and T2 are display time differences.

When the concealment method determination unit 6 receives the encoding parameter from the frame memory unit 2 and receives the estimated motion vector information from the estimated motion vector calculation unit 5, the concealment method determination unit 6 receives the motion vector for each macroblock included in the encoding parameter, The concealment method used for the concealment process is determined with reference to the macroblock type and the motion vector of the concealment process unit included in the estimated motion vector information.
Hereinafter, the concealment method determination process by the concealment method determination unit 6 will be described in detail.
FIG. 6 is an explanatory diagram showing determination conditions for the concealment method.

  The concealment method determination unit 6 confirms macroblock types of a plurality of macroblocks constituting a concealment processing unit, and when the ratio of macroblocks whose macroblock type is intra prediction is equal to or greater than a predetermined value (intra Inter-frame concealment processing is appropriate because it is assumed that there are many macroblocks with large motion within the concealment processing unit). Therefore, the concealment method used for the concealment process is determined as the in-frame concealment process method (see FIG. 6A). This situation is assumed to be true even when a scene change occurs, for example.

  The concealment method determination unit 6 moves the motion within the concealment processing unit when the absolute difference between the maximum value and the minimum value of the motion vectors of the plurality of macroblocks constituting the concealment processing unit is equal to or greater than a predetermined threshold. Therefore, it is determined that the inter-frame concealment processing is not appropriate, and the concealment method used for the concealment processing is determined as the in-frame concealment processing method ( (See FIG. 6B).

  The concealment method determination unit 6 classifies the motion vectors of a plurality of macroblocks constituting a concealment processing unit into a plurality of classes according to the size, and the number of classified classes (number of motion vector classifications). Is greater than or equal to a predetermined threshold (when the motion vector size distribution covers a wide range), it is assumed that there are many macroblocks with different motions in the concealment processing unit. It is determined that the inter-concealment process is not appropriate, and the concealment method used for the concealment process is determined as the in-frame concealment process method (see FIG. 6C).

When the motion vector of the concealment processing unit calculated by the estimated motion vector calculation unit 5 indicates the outside of the picture screen, the concealment method determination unit 6 does not include a reference decoded image, and the pixel at the picture boundary Since it is assumed that subjective degradation of the decoded image will increase by replacing with a fixed value such as a value, it is determined that interframe concealment processing is not appropriate, and the concealment method used for concealment processing is determined. The in-frame concealment processing method is determined (see FIG. 6D).
The concealment method determination unit 6 determines an inter-frame concealment processing method using a motion vector of a concealment processing unit as the concealment method used for the concealment processing when none of the above determination conditions is satisfied.

When the concealment method determination unit 6 determines the concealment method, the concealment processing unit 7 determines that the reference decoded image output from the frame memory unit 2 and the concealment method are the inter-frame concealment processing method. Using the motion vector of the concealment processing unit calculated by the estimated motion vector calculation unit 5, concealment processing for concealing the concealment target in the concealment target image output from the frame memory unit 2 is performed.
On the other hand, when the concealment method is the in-frame concealment processing method, the concealment target image is used by using the surrounding macroblocks of the concealment target indicated by the concealment target information output from the concealment target determination unit 3. A concealment process is performed to conceal the concealment object.

That is, when the concealment method is an inter-frame concealment processing method, the concealment processing unit 7 detects an image at the position indicated by the motion vector of the concealment processing unit from the reference decoded image (the same size as the concealment processing unit). Concealment processing is performed to replace the concealment target in the concealment target image with the image at the position indicated by the motion vector, and a concealment processing image is generated.
On the other hand, when the concealment method is an in-frame concealment processing method, the lower end of a macroblock (a macroblock in which no error has occurred) positioned above the concealment target determined by the concealment target determination unit 3 The pixel of the line or the top line pixel of the macroblock located below (macroblock in which no error has occurred) was acquired, and the pixel to be concealed in the concealment target image was acquired. A concealment process for replacing pixels is performed to generate a concealment processed image.

  Here, when the concealment method is the intra-frame concealment processing method, the concealment processing unit 7 has been shown to be replaced with the pixel of the lower end line or the upper end line, but all the pixels constituting the concealment target image The maximum and minimum values of the luminance signal and color difference signal are calculated, and the luminance signal and color difference signal of each pixel constituting the image of the concealment processing unit in which the error has occurred are the maximum value and the minimum value. If it is significantly out of the range, it is judged that the image is distorted due to an error, the pixel is replaced with the average value of the surrounding pixels within the range, and the pixels within the range are output as they are You may do it.

When the concealment processing for all the pixels within the concealment target included in the picture is completed, the concealment processing unit 7 outputs a concealment processed image for one picture.
In addition, when the concealment process is performed, the concealment processing unit 7 stores the concealment processing image for one picture in the frame memory unit 2 and uses it as a decoded image for reference in subsequent concealment processing. It can be so.
However, only when the concealment processing image is an I picture or a P picture, the concealment processing image is stored in the frame memory unit 2 as a decoded image for reference.

  As apparent from the above, according to the first embodiment, when an error is detected during the decoding process of the image decoding unit 1, the entire slice including the macro block in which the error has occurred is set as a concealment target. A concealment target determining unit 3 that determines the entire picture in which an error is propagated from the picture including the macroblock as a concealment target, and a picture included in the encoding parameter output from the image decoding unit 1 A reference frame determination unit that determines a reference decoded image and encoding parameter to be used for concealment processing to be concealed from among the decoded image and encoding parameter stored in the frame memory unit 2 with reference to the type 4 and the macro included in the encoding parameter determined by the reference frame determination unit 4. Included in the estimated motion vector calculation unit 5 that calculates a motion vector in a concealment processing unit composed of a plurality of macroblocks using the motion vector in units of block and the encoding parameter determined by the reference frame determination unit 4 The concealment method for determining the concealment method used for the concealment process with reference to the motion vector of the macroblock unit, the macroblock type and the motion vector of the concealment process unit calculated by the estimated motion vector calculation unit 5 A determination unit 6, and the concealment processing unit 7 refers to the decoded image for reference determined by the reference frame determination unit 4 according to the concealment method determined by the concealment method determination unit 6 as necessary. While concealment Since the concealment process for concealing the concealment target determined by the fixing unit 3 is performed, the tracking ability to the motion is maintained without being affected by the variable length encoding process in the image encoding apparatus. There is an effect that a decoded image with little subjective deterioration can be obtained.

Embodiment 2. FIG.
In the first embodiment, when the concealment method determined by the concealment method determination unit 6 is the intra-frame concealment processing method ( when it is determined that the inter-frame concealment processing using the motion vector is not appropriate). The concealment processing unit 7 uses the macroblocks around the concealment object to perform the concealment process for concealing the concealment object image. However, the concealment processing unit 7 uses the macroblocks around the concealment object. Instead, the concealment target image is concealed using image data at the same position between the decoded images for reference (between frames) determined by the reference frame determination unit 4 (hereinafter referred to as “true back image data”). You may make it implement a concealment process.
In this case, it is not necessary for the concealment processing unit 7 to calculate the features (average value, maximum value, minimum value, etc.) of the past picture and surrounding pixels, so that a simple configuration can be taken.
In the second embodiment, the same effect as in the first embodiment can be obtained.

Embodiment 3 FIG.
FIG. 7 is a block diagram showing an image decoding apparatus according to Embodiment 3 of the present invention. In the figure, the same reference numerals as those in FIG.
The concealment method determining unit 8 performs the process of determining the concealment method used for the concealment process, similarly to the concealment method determining unit 6 of FIG. When the concealment processing method is not used, the ratio of the concealment target to the entire concealment target image (decoded image including the concealment target) output from the frame memory unit 2 is compared with a predetermined threshold, and the ratio of the concealment target If is less than the threshold value, the concealment method used for the concealment process is determined to be the in-frame concealment process method. However, if the concealment target ratio is equal to or greater than the threshold value, the concealment process is performed using the back image data. Method It carries out a process of determining the to Embodiment 2 of the method). The concealment method determining unit 8 constitutes concealment method determining means.

Next, the operation will be described.
However, since the configuration other than the concealment method determination unit 8 is the same as that of the first embodiment, only the processing content of the concealment method determination unit 8 will be described.
In the first embodiment, the concealment method used for the concealment process is determined as either the inter-frame concealment process method or the intra-frame concealment process method. In the third embodiment, the inter-frame concealment method is used. It is different in that it is determined as a processing method, an in-frame concealment processing method, or a method for performing concealment processing using true back image data.

When the inter-frame concealment processing method is not used as the concealment method used for the concealment process, the concealment method determination unit 8 determines the ratio of the concealment target to the entire concealment target image output from the frame memory unit 2. Compare with a predetermined threshold.
The concealment method determining unit 8 determines the concealment method used for the concealment process as the in-frame concealment process method when the concealment target ratio is less than the threshold, but the concealment target ratio is equal to or greater than the threshold. In this case, since there is little image data as a concealment source in the frame, it is determined that the intra-frame concealment processing is not appropriate, and the method for performing the concealment processing using the true back image data is determined.

  Further, the concealment method determination unit 8 calculates the maximum value and the minimum value of the luminance signal and the color difference signal of all the pixels constituting the concealment target image, and when the maximum value is larger than a predetermined threshold value, or When the minimum value is smaller than the predetermined threshold, it is impossible to detect a pixel that is significantly out of the range between the maximum value and the minimum value among the pixels constituting the concealment target. It is determined that the concealment process is not appropriate, and a method for performing the concealment process using the back image data is determined.

  As apparent from the above, according to the third embodiment, when the inter-frame concealment processing method is not used as the concealment method used for the concealment processing, the concealment target image output from the frame memory unit 2 is displayed. Compare the percentage of the concealment target to the predetermined threshold, and if the percentage of the concealment target is less than the threshold, the concealment method used for the concealment process is determined as the in-frame concealment process method. If the ratio is equal to or greater than the threshold value, it is determined that the concealment process is performed using the true back image data, so even if the subjective deterioration cannot be reduced by the in-frame concealment process Improved concealment processing image An effect that can be obtained.

Embodiment 4 FIG.
FIG. 8 is a block diagram showing an image decoding apparatus according to Embodiment 4 of the present invention. In the figure, the same reference numerals as those in FIG. 1 and FIG.
The concealment target determination unit 9 is composed of, for example, a semiconductor integrated circuit on which a CPU is mounted, a one-chip microcomputer, or the like, and the error detection flag included in the error information output from the image decoding unit 1 is significant. If it is, the maximum value and the minimum value of the luminance signal or color difference signal in the latest decoded image in which no error has occurred among the decoded images stored in the frame memory unit 2 are specified, and the pixels in the macroblock A value is compared with the maximum value and the minimum value, and a macroblock in which an error has occurred is detected.
When the concealment target determination unit 9 detects a macroblock in which an error has occurred in the detection process of a macroblock in which an error has occurred, the concealment target determination unit 9 performs the concealment from that macroblock to the end of the slice including the macroblock. If the macroblock in which an error has occurred is not detected in the macroblock detection process in which an error has occurred, the decoding process of the image decoding unit 1 is the same as in the first to third embodiments. The process of determining the entire slice including the macroblock in which the error is detected as the target of concealment is performed.
Further, the concealment target determination unit 9 can detect the error propagation picture concealment target from the macro block if the macro block in which the error has occurred is detected in the detection process of the macro block in which the error has occurred. A process of determining up to the end of the error propagation picture including the macroblock as a concealment target is performed.
The concealment target determining unit 9 constitutes an error macroblock detection unit and a concealment target determination unit.

Next, the operation will be described.
However, since the components other than the concealment target determining unit 9 are the same as those in the first to third embodiments, only the processing content of the concealment target determining unit 9 will be described.
In the first to third embodiments, the concealment target determining unit 3 has shown that the entire slice including the macro block in which the error has occurred is determined as the concealment target. However, an error actually occurs. Since the macroblock which is not yet subjected to the concealment process, the image quality of the macroblock may be deteriorated.
Therefore, in the fourth embodiment, the position of the macro block in which the error actually occurs is estimated, and by determining from the macro block to the end of the slice including the macro block as the concealment target, Is designed to prevent deterioration of the image quality of macroblocks in which no error has occurred.

The concealment target determination unit 9 generates an error in the decoded image stored in the frame memory unit 2 when the error detection flag included in the error information output from the image decoding unit 1 is significant. The maximum and minimum values of the luminance signal or color difference signal in the latest decoded image that has not been specified are identified, and the pixel value in the macroblock is compared with the maximum value and the minimum value, and the macroblock in which an error has occurred Execute the detection process.
That is, the concealment target determination unit 9 sequentially detects macroblocks in which errors are detected in the decoding process of the image decoding unit 1 (macroblocks in which no error may actually occur), starting from the first macroblock of the slice. Until the block), the pixel value in the macroblock is compared with the maximum value and the minimum value.
Then, the concealment target determination unit 9 detects a macroblock including a pixel value that is greatly deviated from the range of the maximum value and the minimum value (hereinafter referred to as “detected macroblock”). The detected macroblock determines that the image is disturbed due to an error.

The concealment target determination unit 9 starts detection processing from the first macroblock of the slice, and determines from the first detected macroblock to the last macroblock of the slice as a concealment target.
If the concealment target determining unit 9 cannot detect a macroblock in which an error has occurred in the macroblock detection processing, the decoding processing of the image decoding unit 1 is performed as in the first to third embodiments. The entire slice including the macroblock in which the error is detected is determined as a concealment target.

For the error propagation picture, the concealment target determination unit 9 performs the same detection process on the entire picture, and includes a macroblock that includes a pixel value that is significantly out of the range of the maximum value and the minimum value. Is detected, the first macroblock to the last macroblock of the picture is determined as a concealment target.
On the other hand, if the macroblock in which an error has occurred cannot be detected in the macroblock detection process, the entire error propagation picture is determined as a concealment object as in the first to third embodiments.

Here, the error propagation picture has been shown to be subjected to the same detection process, but the maximum value in the positive and negative direction of the vertical component of the motion vector included in the encoding parameter of the error propagation picture is obtained for each error propagation picture. A range obtained by extending the macroblock line including the maximum value of the motion vector from the concealment target of the generated picture to each of the positive and negative directions in the vertical direction may be set as the concealment target.
For the next error propagation picture, the maximum value of the motion vector is obtained in the same way, and the range of the macroblock line that includes the maximum value of the motion vector from the concealment range of the previous error propagation picture is used as the concealment target. Good.

  As is apparent from the above, according to the fourth embodiment, the concealment target determining unit 9 performs the macroblock detection process in which an error has occurred, and an error has occurred in the detection process. When a macro block is detected, the concealment target is determined from the macro block to the end of the slice including the macro block, so that the image quality of the macro block in which no error actually occurs is deteriorated. Therefore, there is an effect that it is possible to obtain a decoded image with little subjective deterioration while maintaining the followability to the motion.

  In the present invention, within the scope of the invention, any combination of the embodiments, or any modification of any component in each embodiment, or omission of any component in each embodiment is possible. .

  DESCRIPTION OF SYMBOLS 1 Image decoding part (image decoding means), 2 Frame memory part (Data storage means), 3 Concealment object determination part (Concealment object determination means), 4 Reference frame determination part (Reference data determination means), 5 Estimated motion vector Calculation unit (motion vector calculation unit), 6, 8 Concealment method determination unit (Concealment method determination unit), 7 Concealment processing unit (Concealment processing unit), 9 Concealment target determination unit (Error macroblock detection unit, Concealment target determination means).

Claims (12)

Image decoding means for performing decoding processing on encoded data of an image generated by performing motion compensation prediction processing and variable length encoding processing, and outputting a decoded image and an encoding parameter as a decoding result;
Data storage means for storing the decoded image and encoding parameters output from the image decoding means;
When an error is detected during the decoding process of the image decoding means, the entire slice including the macroblock in which the error has occurred is determined as a concealment target, and the error propagates from the picture including the macroblock A concealment target determining means for determining the whole as a concealment target,
With reference to the picture type included in the encoding parameter output from the image decoding unit, the concealment process to be concealed from the decoded image and the encoding parameter stored in the data storage unit Reference data determining means for determining a reference decoded image and encoding parameters to be used in
A motion vector calculating means for calculating a motion vector in a concealment process unit composed of a plurality of macroblocks using a motion vector in a macroblock unit included in an encoding parameter determined by the reference data determining means; ,
By referring to the motion vector and macroblock type included in the encoding parameter determined by the reference data determining means and the motion vector of the concealment processing unit calculated by the motion vector calculating means, concealment is performed. A concealment method determining means for determining an interframe concealment processing method using inter-frame image data and motion vectors or an intra-frame concealment processing method using image data in the same frame as the concealment method used for the concealment processing When,
According to the concealment method determined by the concealment method determining means, the concealment determined by the concealment target determining means while referring to the reference decoded image determined by the reference data determining means as necessary. An image decoding apparatus comprising: concealment processing means for performing concealment processing for concealing an object to be sent.   2. The image according to claim 1, wherein the motion vector calculation means calculates a motion vector of a concealment processing unit based on the magnitudes of motion vectors of a plurality of macroblocks constituting the concealment processing unit. Decoding device.   When the concealment method determined by the concealment method determination unit is an interframe concealment processing method, the concealment processing unit calculates the reference decoded image and the motion vector calculated by the reference data determination unit. The concealment process for concealing the concealment target determined by the concealment target determining means is performed using the motion vector of the concealment processing unit calculated by the means, and the concealment method is an in-frame concealment processing method. 3. The image decoding apparatus according to claim 1, wherein concealment processing for concealing the concealment target is performed using macroblocks around the concealment target.   The concealment method determining means confirms the macroblock types of a plurality of macroblocks constituting a concealment processing unit, and when the ratio of macroblocks in which the macroblock type is intra prediction is equal to or greater than a predetermined value, 4. The image decoding apparatus according to claim 3, wherein the concealment method used for the concealment process is determined as an in-frame concealment process method.   The concealment method determining means determines whether the concealment method used for the concealment processing is a frame when the difference between the maximum value and the minimum value in the motion vectors of a plurality of macroblocks constituting the concealment processing unit is a predetermined value or more. 4. The image decoding apparatus according to claim 3, wherein the inner concealment processing method is determined.   The concealment method determining means classifies motion vectors of a plurality of macroblocks constituting a concealment processing unit according to the size, and when the number of motion vector classifications is equal to or greater than a predetermined value, 4. The image decoding apparatus according to claim 3, wherein the concealment method to be used is determined as an in-frame concealment processing method.   The concealment method determining means, when the motion vector of the concealment processing unit calculated by the motion vector calculating means indicates the outside of the picture screen, the concealment method used for the concealment processing is the intra-frame concealment processing method. The image decoding apparatus according to claim 3, wherein the image decoding apparatus is determined as follows. Image decoding means for performing decoding processing on encoded data of an image generated by performing motion compensation prediction processing and variable length encoding processing, and outputting a decoded image and an encoding parameter as a decoding result;
When an error is detected during the decoding process of the decoded image output from the image decoding unit and the decoding parameter of the image decoding unit, the slice including the macroblock in which the error has occurred A concealment target determining means for determining the entire picture in which an error is propagated from the picture including the macroblock as a concealment target,
With reference to the picture type included in the encoding parameter output from the image decoding unit, the concealment process to be concealed from the decoded image and the encoding parameter stored in the data storage unit Reference data determining means for determining a reference decoded image and encoding parameters to be used in
A motion vector calculating means for calculating a motion vector in a concealment process unit composed of a plurality of macroblocks using a motion vector in a macroblock unit included in an encoding parameter determined by the reference data determining means; ,
By referring to the motion vector and macroblock type included in the encoding parameter determined by the reference data determining means and the motion vector of the concealment processing unit calculated by the motion vector calculating means, concealment is performed. A concealment method determining means for determining an interframe concealment processing method using a motion vector or an interframe concealment processing method using no motion vector as a concealment method used for
According to the concealment method determined by the concealment method determining means, the concealment determined by the concealment target determining means while referring to the reference decoded image determined by the reference data determining means as necessary. A concealment processing means for performing concealment processing for concealing the object to be sent,
When the concealment method determined by the concealment method determination unit is an interframe concealment processing method using a motion vector, the concealment processing unit is a decoded image for reference determined by the reference data determination unit. The concealment process for concealing the concealment target determined by the concealment target determination unit is performed using the motion vector of the concealment processing unit calculated by the motion vector calculation unit, and the motion vector is not used. If a concealment processing scheme between frames, using the image data of the same position between the decoded image of the reference which is determined by the upper Symbol reference data determining means, performing a concealment process to conceal the concealment subject images decoded you wherein Location.   When the ratio of the concealment target to the entire decoded image is equal to or greater than a predetermined value, the concealment processing unit does not use the macroblocks around the concealment target but instead uses the reference data determined by the reference data determination unit. 9. The image decoding apparatus according to claim 8, wherein concealment processing for concealing the concealment target is performed using image data at the same position between the decoded images. If an error is detected during the decoding process of the image decoding means, the maximum value of the luminance signal or color difference signal in the latest decoded image in which no error has occurred among the decoded images stored in the data storage means And an error macroblock detection means for specifying a minimum value, comparing a pixel value in the macroblock with the maximum value and the minimum value, and performing a process of detecting a macroblock in which an error has occurred,
The concealment target determining means, when a macro block in which an error has occurred is detected by the error macro block detecting means, determines from the macro block to the end of the slice including the macro block as a concealment target, When a macroblock in which an error has occurred is not detected by the error macroblock detection unit, the entire slice including the macroblock in which the error is detected in the decoding process of the image decoding unit is determined as a concealment target. The image decoding device according to any one of claims 1 to 9.   In the case of a picture in which an error is propagated, the concealment target determining means, from the macro block to the end of the picture including the macro block, if the error macro block detecting means detects a macro block in which an error has occurred. The image decoding apparatus according to claim 10, wherein the image is determined as a concealment target.   12. The image decoding apparatus according to claim 11, wherein the error macroblock detection unit detects a macroblock in which an error propagates with reference to a motion vector in units of macroblocks in a picture in which the error propagates.

Images