JP4360207B2 - Video decoding device - Google Patents

Description

The present invention relates to a video decoding KaSo location for decoding the video information that has been band compressed transmission.

  Conventionally, when a missing pixel block in a current frame of a moving image is interpolated using a previous frame, a motion vector of a missing pixel block is estimated from motion vectors of neighboring pixel blocks of the missing block, and according to the estimated motion vector The position (reference position) of the reference pixel block in the previous frame is determined, and the missing pixel block is interpolated with the reference pixel block (see, for example, Patent Document 1).

JP-A-6-20050

  However, in the above conventional technique, the reference position in the previous frame is determined only by the motion vector estimation regardless of the pixel values of the previous frame and the current frame. In many cases, the texture shift was conspicuous at the block boundaries.

The present invention has been made in order to solve such a conventional problem, and is a moving picture decoding capable of reducing deterioration in image quality at the boundary between a defective pixel block interpolated in a moving picture frame and its peripheral pixel block. an object of the present invention is to provide a KaSo location.

The video decoding device of the present invention is
In a moving picture decoding apparatus for decoding moving picture information transmitted by band compression,
Decoding means for decoding the input moving image information;
A first memory for storing pixel information of the frame decoded by the decoding means;
A second memory for storing pixel information of a neighboring pixel set that is a predetermined pixel set near a defective pixel block generated in the decoded frame;
A third memory for storing pixel information of the frame after interpolation processing;
A plurality of similar pixel sets near the pixel values of the neighboring pixel set of the current frame stored in the second memory and the reference pixel block candidates in the previous frame after interpolation processing stored in the third memory Search means for searching, using a candidate pixel value as a reference position, a position in the previous frame that is similar in texture to the neighboring pixel set,
Using the pixel block at the reference position in the previous frame obtained by the search means, the missing pixel block of the current frame is interpolated, and the pixel information of the current frame after interpolation is stored in the third memory. Interpolation means ;
A fourth memory for storing motion vector information obtained in the decoding;
Deformation detecting means for detecting deformation of the texture between the current frame stored in the first memory and the previous frame after interpolation processing stored in the third memory,
The search means further uses the motion vector information of the neighboring pixel blocks of the missing pixel block and the deformation information of the texture between the frames obtained by the deformation detection means, and the respective pixel values of the neighborhood pixel set Search for the position of the similar pixel set candidate that minimizes the sum of the differences of the corresponding pixel values of the similar pixel set candidate as the reference position,
When the deformation detection means detects that the texture of the current frame is reduced compared to the previous frame,
The search means enlarges and sets the position of the similar pixel set candidate of the previous frame by the enlargement magnification of the previous frame when viewed from the center position of the reduction, and sets the pixel of the similar pixel set candidate set by the enlarged movement Use the value to search the reference position,
When the deformation detection means detects that the texture of the current frame is enlarged compared to the previous frame,
The search means sets the position of the similar pixel set candidate of the previous frame by reducing and setting the position of the previous frame by the reduction magnification of the previous frame when viewed from the center position of the enlargement, and sets the pixel of the similar pixel set candidate set by the reduction movement The reference position is searched using the value .

  According to the present invention, a position in the previous frame whose texture is similar to a neighboring pixel set of the missing pixel block generated in the current frame is searched as a reference position in the previous frame to be referred to in order to interpolate the missing pixel block. By interpolating the defective pixel block with the obtained pixel block at the reference position in the previous frame, it is possible to reduce image quality deterioration at the boundary between the interpolated defective pixel block and its surrounding pixel blocks.

Embodiment 1
FIG. 1 is a block configuration diagram of a moving picture decoding apparatus according to Embodiment 1 of the present invention. 1 includes an input terminal 101, a decoding device 102, a frame memory 103, a neighboring pixel memory 104, an interpolation device 105, a search device 106, and a frame memory 107. And an output terminal 108.

  The decoding apparatus 102 decodes the moving image information that has been band-compressed and transmitted input from the input terminal 101, and the frame information of the decoded frame (information about the frame such as pixel values and other pixel information) is stored in the frame memory 103. To store. In addition, when a decoding pixel block has a missing pixel block in the decoded frame, the decoding apparatus 102 determines a neighboring image set that is a decoded pixel set in the vicinity of the missing pixel block, and pixel information of the neighboring pixel set Is stored in the neighboring pixel memory 104.

  The interpolator 105 performs different interpolation processing described below depending on whether the decoded current frame stored in the frame memory 103 has a missing pixel block or not. When the decoded current frame does not generate a defective pixel block, the frame information of the current frame output from the frame memory 103 is output as it is from the output terminal 108 and stored in the frame memory 107 as it is. When the decoded frame generates a missing pixel block, the missing pixel block is interpolated by the pixel block (reference pixel block) at the reference position in the previous frame obtained by the search device 106, and this interpolation is performed. The frame information of the current frame is output from the output terminal 108 and stored in the frame memory 107.

  The search device 106 uses the pixel values of the neighboring pixel set of the current frame stored in the neighboring pixel memory 104 and the pixel values of the previous frame after the interpolation processing stored in the frame memory 107 to search for the neighboring pixel set. Is searched as a reference position in the previous frame that is referred to in order to interpolate the missing pixel block, and the previous frame necessary as a reference pixel block for the interpolation and the above-mentioned interpolation. Is sent to the interpolation device 105.

  Described below is the operation of the moving picture decoding apparatus according to the first embodiment when a defect occurs in the decoded current frame and the defective pixel block is 8 pixels × 8 pixels.

  As shown in FIG. 2, assuming that the defect position in the current frame is the position of the upper left corner of the defective pixel block (x, y), the position of the upper right corner of the defective pixel block is (x + 7, y), The position of the lower left corner is (x, y + 7), and the position of the lower right corner is (x + 7, y + 7).

  As a neighboring pixel set for the missing position (x, y), for example, (x-1, y + β) (0 ≦ β ≦ 7), which is a set of pixels extending from the missing pixel block by 1 pixel in the vertical and horizontal directions. 8 pixels, (x + α, y−1) (0 ≦ α ≦ 7), 8 pixels, (x + 8, y + β) (0 ≦ β ≦ 7), (x + α, y + 8) (0 ≦ α ≦ 7) A total of 32 pixels is set.

  Also, consider a plurality of reference pixel block candidates, each of which is a pixel block of 8 pixels × 8 pixels, which is the same as the defective pixel block, for the previous frame. For each reference pixel block candidate, a similar pixel set candidate having the same size as the neighboring pixel set of the defective pixel block is considered.

  As shown in FIG. 3, when one position among the plurality of reference position candidates is (x ′, y ′), the position of the upper left corner of the reference pixel block candidate is (x ′, y ′), The position is (x ′ + 7, y ′), the position of the lower left corner is (x ′, y ′ + 7), and the position of the lower right corner is (x ′ + 7, y ′ + 7).

  As similar pixel set candidates for the reference position candidate (x ′, y ′), a set of pixels that are extended by one pixel from the reference pixel block candidate in the vertical and horizontal directions, similarly to the neighboring pixel set of the defective pixel block. (X′−1, y ′ + β) (0 ≦ β ≦ 7), 8 pixels (x ′ + α, y′−1) (0 ≦ α ≦ 7), (x ′ + 8, A total of 32 pixels of 8 pixels of y ′ + β) (0 ≦ β ≦ 7) and 8 pixels of (x ′ + α, y ′ + 8) (0 ≦ α ≦ 7) are set.

Further, assuming that the pixel value at the position (X, Y) of the current frame is V (X, Y) and the pixel value at the position (X ′, Y ′) of the previous frame is V ′ (X ′, Y ′), the following (1) Define an expression.

And, for example, the reference position candidate set in the previous frame is
{(X ′, y ′) | x−8 <x ′ <x + 8, y−8 <y ′ <y + 8} (2)
In the search device 106, the value of the function f _{(x, y)} (x ′, y ′) defined by the above equation (1) is calculated from a plurality of reference position candidates that satisfy the conditions of the reference position candidate set. A reference position candidate to be minimized is searched, and a reference position candidate having a minimum value of the function f _{(x, y)} (x ′, y ′) is set as a reference position (x _p , y _p ). At this time, the position of the upper left corner of the reference pixel block is (x _p , y _p ), the position of the upper right corner is (x _p +7, y _p ), the position of the lower left corner is (x _p , y _p +7), and the lower right corner position is _{(x p + 7, y p} +7) of.

In the interpolation device 105, the pixel value at the position (x + α, y + β) (0 ≦ α ≦ 7, 0 ≦ β ≦ 7) of the missing pixel block in the current frame is the position (x _p + α, y _p + β) of the previous frame. Interpolated with pixel values.

  As described above, according to the first embodiment, using the pixel value of the neighboring pixel set of the missing pixel block of the current frame and the pixel value of the previous frame, the position in the previous frame whose texture is similar to that of the neighboring pixel set. As a reference position in the previous frame to be referenced to interpolate the missing pixel block, and interpolating the missing pixel block by interpolating the missing pixel block with the obtained pixel block at the reference position in the previous frame. The boundary between the pixel block and the surrounding pixel block is easily connected smoothly, and image quality deterioration can be suppressed with a high probability.

  In the first embodiment, the case where the defective pixel block is 8 pixels × 8 pixels has been described. However, the shape of the defective pixel block and the shape of the neighboring pixel set may be arbitrary. Furthermore, the neighboring pixel set need not be adjacent to the defective pixel block.

Embodiment 2
FIG. 4 is a block configuration diagram of the moving picture decoding apparatus according to the second embodiment of the present invention. 4 includes an input terminal 401, a decoding device 402, a frame memory 403, a neighboring pixel memory 404, an interpolation device 405, a search device 406, and a frame memory 407. And an output terminal 408 and a motion vector memory 409.

  In the moving picture decoding apparatus according to the second embodiment, the configurations and operations of the input terminal 401, the decoding apparatus 402, the frame memory 403, the neighboring pixel memory 404, the interpolation apparatus 405, the frame memory 407, and the output terminal 408 are as described above. This is the same as the input terminal 101, the decoding device 102, the frame memory 103, the neighboring pixel memory 104, the interpolation device 105, the frame memory 107, and the output terminal 108 of the first embodiment shown in FIG.

  The motion vector memory 409 stores motion vector information obtained together by decoding when moving picture information input to the input terminal is decoded by the decoding device 402.

  In search device 106 (see FIG. 1) of the first embodiment, the reference position of the previous frame is obtained using the pixel values of the neighboring pixel set of the missing pixel block of the current frame and the pixel values of the previous frame after the interpolation process. In addition to these pixel values, the search device 406 of the second embodiment searches for motion vector information of the macroblock adjacent to the missing pixel block of the current frame stored in the motion vector memory 409 and the previous frame. The reference position of the previous frame is searched using the motion vector information.

For example, the motion vector (x _U , y _U ) of the macro block adjacent to the upper side of the defective pixel block in the current frame and the motion vector (x _L , y of the macro block adjacent to the left side of the defective pixel block in the current frame) _L ) and the motion vector (x _UL , y _UL ) of the defective pixel block is
x _UL = (x _U + x _L ) / 2 (3)
y _UL = (y _U + y _L ) / 2 (4)
Predict by.

Then, using the motion vector (x _UL , y _UL ) of the predicted missing pixel block as a reference position candidate set,
{(X ', y') | x-x UL -8 <x '<x-x UL + 8, y-y UL -8 <y'<y-y UL +8} ... (5)
And the central position (xx _UL , yy _UL ) of this reference position candidate set is the first calculation position of the value of the function f _{(x, y)} (x ′, y ′) in the above equation (1). Then, the next calculation position is determined so as to draw a spiral from this center position (xx _UL , yy _UL ).

In the search for the reference position (x _p , y _p ) by calculating the value of the function f _{(x, y)} (x ′, y ′), the value of the function f _{(x, y)} (x ′, y ′) If the value falls below the appropriately set threshold AllowDifference, the search is terminated at that time, and the position is determined as the reference position (x _p , y _p ).

Further, if the value of the function f _{(x, y)} (x ′, y ′) increases as the calculation is continued and exceeds an appropriately set threshold value AbortDifference, the search is performed at that time. , And the position where the value of the function f _{(x, y)} (x ′, y ′) has been minimized is determined as the reference position (x _p , y _p ).

According to the second embodiment as described above, they predict the motion vector of the defect image Motobu lock from the motion vector information of the peripheral pixel blocks of the defective pixel blocks, and the center position of the reference position candidates set the prediction motion vector At the same time, by setting appropriate threshold values AllowDifference and AbortDifference for the value of the function f _{(x, y)} (x ′, y ′) in the above formula (1), the redundant search can be omitted. Can be obtained at high speed.

  In the second embodiment, the motion vector of the missing pixel block is calculated from the average value of the motion vector components of the upper neighboring macroblock adjacent to the missing pixel block of the current frame and the motion vector component of the left neighboring macroblock. However, the motion vector information to be referred to is not limited to two, and is not necessarily a motion vector in the same frame. Further, the calculation order is not limited to the spiral shape, and other orders may be used.

Also, the thresholds AllowDifference and AbortDifference are not necessarily fixed constants, and the threshold AllowDifference increases as the number of search steps increases, and the threshold AbortDifferenece decreases. Alternatively, it may be determined adaptively. Thus, as a configuration for adaptively determining the threshold AllowDifference and AbortDifference, for example, the following processing algorithm can be considered. First, the initial value AllowDifferenceStart of the threshold AllowDifference and the initial value AbortDifferenceStart of the threshold AbortDifferenece,
AllowDifferenceStart ≦ AbortDifferenceStart
To be determined. A search step variable whose initial value is 0 and increases by a predetermined value for each increase in the number of search steps is StepCount, and the value of the function f _{(x, y)} (x ′, y ′) in the above equation (1) is Value. As
Value ≦ AllowDifferenceStart + StepCount
If so, the process ends with the value taking the smallest so far as the decision value,
Value ≧ AbortDifference−StepCount
If so, the process ends with that value as the decision value. According to this processing algorithm, as the search is performed, the threshold value is changed so as to increase the possibility that either threshold condition is satisfied and the processing is interrupted.

Embodiment 3
FIG. 5 is a block configuration diagram of a moving picture decoding apparatus according to Embodiment 3 of the present invention. 5 includes an input terminal 501, a decoding device 502, a frame memory 503, a neighboring pixel memory 504, an interpolation device 505, a search device 506, and a frame memory 507. An output terminal 508, a motion vector memory 509, and a deformation detection device 510.

  In the moving picture decoding apparatus according to the third embodiment, the configuration of an input terminal 501, a decoding apparatus 502, a frame memory 503, a neighboring pixel memory 504, an interpolation apparatus 505, a frame memory 507, an output terminal 508, and a motion vector memory 509 The operations are the same as those of the input terminal 401, the decoding device 402, the frame memory 403, the neighboring pixel memory 404, the interpolation device 405, the frame memory 407, the output terminal 408, and the motion vector memory 409 of the second embodiment shown in FIG. is there.

  In the third embodiment, after the frame information of the current frame decoded by the decoding device 502 is stored in the frame memory 503, the frame information is stored in the frame memory 503 before the search device 506 starts searching for the reference position. The frame information of the current frame and the frame information of the previous frame stored in the frame memory 507 are sent to the deformation detection device 510.

  The deformation detection device 510 uses the frame information of the current frame and the previous frame to detect how the texture of the current frame is deformed compared to the previous frame, and the detected magnification, orientation, reference point, The deformation information such as the center point is sent to the search device 506. The deformation is, for example, enlargement or reduction.

  The search device 406 (see FIG. 4) of the second embodiment uses the pixel values of the neighboring pixel set of the missing pixel block of the current frame, the pixel values of the previous frame after the interpolation process, and the motion vector information. The search device 606 of the third embodiment searches for the reference position of the previous frame using the deformation information from the deformation detection device 510 in addition to the pixel value and the motion vector information. Search.

  For example, when the center position of the defective pixel block is used as the center position of deformation and the texture of the current frame is reduced as compared with the previous frame, the search device 506 enlarges the magnification of the previous frame when viewed from the center position. The position of the similar pixel set candidate in the previous frame is enlarged and set by the amount, and the value of equation (1) is calculated using the pixel value of the similar pixel set candidate set by the enlarged movement, The reference position is searched in the same manner as the search device 406 in the second embodiment. Therefore, the pixel of the similar pixel set candidate of the previous frame from which the pixel value difference is calculated by the above equation (1) and the pixel of the neighboring pixel set of the current frame is the center of the current frame compared with the case where there is no reduction deformation of the current frame. It becomes an outer pixel when viewed from the position.

  Further, in this case, a pixel block enlarged at the reference position by the enlargement magnification of the previous frame from the defective pixel block is set as a reference pixel block, and the reference pixel block is reduced by the reduction magnification of the current frame to reduce the defective pixel. Interpolate blocks.

  Also, for example, when the texture of the current frame is enlarged compared to the previous frame with the center position of the missing pixel block as the center position of the deformation, the search device 506 looks for the previous frame as viewed from the center position. The position of the similar pixel set candidate of the previous frame is reduced and set by the reduction magnification, and the value of the above equation (1) is calculated using the pixel value of the similar pixel set candidate set by the reduced movement, Similar to the search device 406 of the second embodiment, the reference position is searched. Therefore, the pixel of the similar pixel set candidate of the previous frame from which the pixel value difference is obtained by the above equation (1) and the pixel of the neighboring pixel set of the current frame is the center of the current frame compared to the case where there is no enlarged deformation of the current frame. It becomes the inner pixel when viewed from the position.

  Further, in this case, a pixel block reduced by the reduction magnification of the previous frame from the defective pixel block of the current frame at the reference position is set as a reference pixel block, and the reference pixel block is enlarged by the enlargement magnification of the current frame. Interpolate the missing pixel block.

  As described above, according to the third embodiment, the position and range of the similar pixel set candidate of the previous frame are changed according to the texture deformation information such as reduction or enlargement obtained by the deformation detection device 510. In addition to parallel translation of the texture, the deformation of the texture such as reduction or enlargement is also interpolated by searching for the reference position and changing the position and range of the reference pixel block that interpolates the missing pixel block. The boundary between the pixel block and the surrounding pixel block is easily connected smoothly, and image quality deterioration can be suppressed with a high probability.

  In the third embodiment, reduction deformation and enlargement deformation have been described, but application to general geometric deformation is also possible.

Embodiment 4
FIG. 6 is a block configuration diagram of a video decoding apparatus according to Embodiment 4 of the present invention. 6 includes an input terminal 601, a decoding device 602, a frame memory 603, a neighboring pixel memory 604, an interpolation device 605, a search device 606, and a frame memory 607. An output terminal 608, a motion vector memory 609, and a deformation detection device 610.

  In the moving picture decoding apparatus according to the fourth embodiment, the configuration of an input terminal 601, a frame memory 603, a neighboring pixel memory 604, an interpolation apparatus 605, a search apparatus 606, an output terminal 608, a motion vector memory 609, and a deformation detection apparatus 610 The operations are the input terminal 501, the decoding device 502, the frame memory 503, the neighboring pixel memory 504, the interpolation device 505, the search device 506, the output terminal 508, the motion vector memory 509, and the deformation detection in the third embodiment shown in FIG. Similar to device 510.

  In the fourth embodiment, the frame information of the current frame decoded by the decoding device 602 is stored in the frame memory 503, the current frame is interpolated by the interpolation device 605, and the frame information of the processed current frame is converted into the frame information. After being stored in the memory 607, before the decoding device 602 starts decoding the next frame, the frame information of the current frame after the interpolation processing stored in the frame memory 607 is sent to the decoding device 602.

  In decoding apparatus 502 according to Embodiment 3, the next frame is decoded using the current frame before the interpolation process with the missing pixel block generated as a reference frame (using the frame information of the current frame before the interpolation process). However, the decoding apparatus 602 according to the fourth embodiment uses the current frame after the interpolation process stored in the frame memory 607 as a reference frame (using the frame information of the current frame after the interpolation process), and Is decrypted.

  As described above, according to the fourth embodiment, by decoding the next frame using the current frame after the interpolation process as a reference frame, it is possible to suppress the propagation of the missing pixel block and effectively suppress the image quality deterioration.

  In the fourth embodiment, the previous frame is used to interpolate the missing pixel block of the current frame, and the current frame after the interpolation process is used as a reference frame at the time of decoding the next frame. Not only the current frame but also a backward frame after decoding and interpolation processing before the next frame may be used as a reference frame. Further, the reference frame does not have to be adjacent to the next frame, and a plurality of frames may be separated.

  Also, in each of the above embodiments, the example in which the present invention is applied to a moving picture decoding apparatus that decodes moving picture information that has been subjected to band compression transmission has been described. The present invention can also be applied to interpolation of a defective pixel block generated in an image frame.

It is a block block diagram of the moving image decoding apparatus of Embodiment 1 of this invention. It is a figure explaining the position in the present frame. It is a figure explaining the position in a front frame. It is a block block diagram of the moving image decoding apparatus of Embodiment 2 of this invention. It is a block block diagram of the moving image decoding apparatus of Embodiment 3 of this invention. It is a block block diagram of the moving image decoding apparatus of Embodiment 4 of this invention.

Explanation of symbols

101, 401, 501 Input terminal 102, 402, 502, 602 Decoding device 103, 403, 503 Frame memory 104, 404, 504 Neighboring pixel memory 105, 405, 505 Interpolation device 106, 406, 506 Search device 107, 407, 507 Frame memory 108, 408, 508 Output terminal 409, 509 Motion vector memory 510 Deformation detection device

Claims (3)

In a moving picture decoding apparatus for decoding moving picture information transmitted by band compression,
Decoding means for decoding the input moving image information;
A first memory for storing pixel information of the frame decoded by the decoding means;
A second memory for storing pixel information of a neighboring pixel set that is a predetermined pixel set near a defective pixel block generated in the decoded frame;
A third memory for storing pixel information of the frame after interpolation processing;
A plurality of similar pixel sets near the pixel values of the neighboring pixel set of the current frame stored in the second memory and the reference pixel block candidates in the previous frame after interpolation processing stored in the third memory Search means for searching, using a candidate pixel value as a reference position, a position in the previous frame that is similar in texture to the neighboring pixel set,
Using the pixel block at the reference position in the previous frame obtained by the search means, the missing pixel block of the current frame is interpolated, and the pixel information of the current frame after interpolation is stored in the third memory. Interpolation means ;
A fourth memory for storing motion vector information obtained in the decoding;
Deformation detecting means for detecting deformation of the texture between the current frame stored in the first memory and the previous frame after interpolation processing stored in the third memory,
The search means further uses the motion vector information of the neighboring pixel blocks of the missing pixel block and the deformation information of the texture between the frames obtained by the deformation detection means, and the respective pixel values of the neighborhood pixel set Search for the position of the similar pixel set candidate that minimizes the sum of the differences of the corresponding pixel values of the similar pixel set candidate as the reference position,
When the deformation detection means detects that the texture of the current frame is reduced compared to the previous frame,
The search means enlarges and sets the position of the similar pixel set candidate of the previous frame by the enlargement magnification of the previous frame when viewed from the center position of the reduction, and sets the pixel of the similar pixel set candidate set by the enlarged movement Use the value to search the reference position,
When the deformation detection means detects that the texture of the current frame is enlarged compared to the previous frame,
The search means sets the position of the similar pixel set candidate of the previous frame by reducing and setting the position of the previous frame by the reduction magnification of the previous frame when viewed from the center position of the enlargement, and sets the pixel of the similar pixel set candidate set by the reduction movement A moving picture decoding apparatus , wherein a reference position is searched using a value .   The search means sets the position of the similar pixel set candidate of the previous frame by enlarging and setting the position of the previous frame similar to the enlargement magnification of the previous frame when viewed from the center position of the reduction, and sets the pixel of the similar pixel set candidate set by the enlarged movement If you search for a reference position using a value,
  The interpolation means sets a pixel block enlarged at the reference position by the enlargement magnification of the previous frame relative to the defective pixel block as a reference pixel block, and reduces the reference pixel block by the reduction magnification of the current frame to reduce the defective pixel block. Interpolate,
  The search means sets the position of the similar pixel set candidate of the previous frame by reducing the position corresponding to the reduction magnification of the previous frame when viewed from the center position of the enlargement, and the similar pixel set candidate pixels set by the reduction movement If you search for a reference position using a value,
  The interpolation means uses a pixel block reduced by the reduction magnification of the previous frame at the reference position as the reference pixel block, and enlarges the reference pixel block by the enlargement magnification of the current frame. Interpolate pixel blocks
  The moving picture decoding apparatus according to claim 1. The moving picture decoding apparatus according to claim 1 or 2 ,
The moving picture decoding apparatus, wherein the decoding means decodes the next frame with reference to the current frame after the interpolation processing stored in the third memory.

Images