JP2010206628A - Video image decoding apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video image decoding apparatus effectively complementing an error. <P>SOLUTION: An error complementing part 102 holds, as hold images, a predetermined number of decoded images sequentially decoded from a video ES by a video decoder 101 and arranged in chronological order. When an error is detected in the hold images by error information, an error complementing source image is selected from among the hold images temporally adjacent to each other for the detected hold images where error is complemented, and the error in the hold images is complemented by image information corresponding to the error of the error complementing source image. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a moving picture decoding apparatus that detects a macroblock error when decoding a video ES (Elementary Stream) and supplements the error with a decoded image.

  When a conventional video decoding device finds an error that appears in a decoded image due to a grammatical error in the input video data, etc., it is decoded before that point in time or before that image. Error compensation was performed using image information (see, for example, Patent Document 1).

Japanese Patent Laid-Open No. 9-154141

  In a conventional video decoding device that performs error complementation, only an image that is temporally prior to an image that is subject to error compensation or an image that needs to be decoded before an image that is subject to error compensation is used as a complement source image. If there are fewer errors in the image after the target than the complementary image in the conventional technology, or the image after the target than the complementary image in the conventional technology There is a problem that it is not possible to cope with the case where the error compensation can be effectively performed by using the image after the target image, such as when the image is similar to the image of the image.

  The present invention has been made to solve the above-described problems, and a moving image that can effectively perform error compensation by complementing images that need error compensation from temporally preceding and following images. An object is to obtain a decoding device.

  The moving image decoding apparatus according to the present invention obtains a decoded image by decoding an image stream indicating input moving image information, detects an error portion on the decoded image due to an error in the moving image information, and detects the error portion. A decoding unit that generates error location information to be shown, and a predetermined number of decoded images that are sequentially decoded by the decoding unit, are held as a retained image, and the presence of an error location in the retained image is detected by the error location information And the error complement source image is selected from among the retained images before and after the detected error complement destination retained image, and the error location of the error complement destination retained image is selected as the error location of the error complement source image. And error complementing means for complementing errors with image information corresponding to.

According to the present invention, the error complementing means holds the predetermined number of decoded images arranged in time series sequentially decoded by the decoding means as the retained image, and the presence of the error location is detected in the retained image based on the error location information. And the error complement source image is selected from among the retained images before and after the detected error complement destination retained image, and the error location of the error complement destination retained image is selected as the error location of the error complement source image. Error compensation is performed with image information corresponding to.
In this way, since error complementation is performed using the error complementing source image selected from the retained images before and after the time series for images that need error compensation, it can be selected as an error complementing source image compared to the conventional case. The image can be increased. Thereby, there is an effect that error compensation can be performed effectively. Further, by evaluating the error complement source image based on the degree of similarity with an image that needs error complementation, it is possible to complement an error generated by transmission or the like so as to be less noticeable.

It is a block diagram which shows the structure of the moving image decoding apparatus by Embodiment 1 of this invention. It is a block diagram which shows the structure of the error complementation part in FIG. It is a figure for demonstrating operation | movement with the image memory in FIG. 2, and a complement operation part. It is a block diagram which shows the structure of the error complementation part in the moving image decoding apparatus by Embodiment 2 of this invention. It is a block diagram which shows the structure of the error complementation part in the moving image decoding apparatus by Embodiment 3 of this invention. It is a block diagram which shows the structure of the error complementation part in the moving image decoding apparatus by Embodiment 4 of this invention. It is a block diagram which shows the structure of the error complementation part in the moving image decoding apparatus by Embodiment 5 of this invention. It is a block diagram which shows the structure of the moving image decoding apparatus by Embodiment 6 of this invention.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing the configuration of a moving picture decoding apparatus according to Embodiment 1 of the present invention. 1, the moving picture decoding apparatus according to Embodiment 1 includes a video decoder 101 and an error complementing unit 102. The video decoder (decoding means) 101 decodes (decodes) the video ES (Elementary Stream), decodes the decoded image, the image type indicating the representation format of the decoded image in the video ES, and the decoded image This is means for sequentially outputting error location information (coordinates of the portion where the error occurred).

  Here, for example, the video ES is a bit stream encoded in the MPEG-Video format, the image type is any one of I, P, and B in the MPEG-Video format, and the error location information is a macroblock including an error. Coordinates. It should be noted that the error location information is such that, in the decoding process, if the next start code is obtained before the codes to be described in the macro block of the decoding target image are obtained, the macro block is considered to have an error. It can be obtained by detecting grammatical errors.

  An error complementing unit (error complementing unit) 102 is a unit that inputs a decoded image, an image type, and error location information from the video decoder 101, complements an error, and sequentially outputs the complemented image. For example, a decoded image, an image type, and error location information are input from the video decoder 101, and the decoded image is complemented according to the image type and the error location information, and a complement result is obtained as a complement image.

  FIG. 2 is a block diagram showing a configuration of the error complementing unit in FIG. In FIG. 2, the error complementing unit 102 includes an image memory 201, a complement planning unit 202, and a complement operation unit 203. When the decoded images decoded by the video decoder 101 are sequentially input, the image memory 201 holds a plurality of images arranged in chronological order from the newest one as a retained image, and the complementary operation unit 203 performs the above chronological sequence. When a result obtained by complementing one of the held images arranged is input, the image is output as a complemented image.

  The supplementary planning unit 202 sequentially inputs the image type and error location information from the video decoder 101, holds the image type and error location information of the held image held in the image memory 201, and stores the image in the image memory 201. When the presence of an error is detected based on the error location information, a search is performed for the first stored image whose image type is an I frame for the first time after the stored image. Information indicating whether or not and error location information are output as a complementary instruction.

  The complement operation unit 203 inputs a complement instruction from the complement planning unit 202, reads out image information about a portion indicated by the error location information of the number of retained images according to the complement instruction from the image memory 201, and retains an error. Complement processing is executed by writing in the part indicated by the error location information of the image.

Next, the operation will be described.
When the video decoder 101 receives a video ES of a moving image to be played back, the video decoder 101 decodes the video ES, decodes a decoded image, an image type indicating an expression format of the decoded image in the video ES, and the decoded image Is output to the error complementing unit 102.

  In the error complementing unit 102, the image memory 201 sequentially inputs the decoded images and, as shown in FIG. 3, N decoded images (N frames arranged in time series) from the newest in time are displayed. Hold as a holding image. Further, the supplementary planning unit 202 sequentially inputs the image type and error location information from the video decoder 101, and holds the image type and error location information for the N held images held in the image memory 201.

  When the supplementary planning unit 202 detects the presence of an error based on the error location information with respect to the M (M is a predetermined natural number satisfying M ≦ N) number of retained images in the image memory 201, the image type is the first time after the Mth number of images. Is searched for the number of held images that are the I frame, and the fact that the held image is the x-th image and the error location information are output to the complementary operation unit 203 as a complementary instruction.

  In the complement operation unit 203, in accordance with the complement instruction input from the complement planning unit 202, the image information of the portion indicated by the error location information in the complement instruction in the x-th retained image is read from the image memory 201, and the M-th retained Complement processing is executed by writing in the part indicated by the error location information of the image.

  As shown in FIG. 3, the image memory 201 outputs the image as a complementary image when the result obtained by complementing the M-th previous retained image by the complementary operation unit 203 is input.

  In the above description, the I-frame of the MPEG-Video format is used as the image type. However, the image type may be an image represented by an encoding type in which the entire image is held in a single frame, and the MPEG-Video format. The present invention can also be applied to video encoding methods other than those described above. Also, in a moving image coding format that does not have an image type or has no effect on error compensation even if it is determined by the image type, the immediately following image can be used without determining the image type.

  As described above, according to the first embodiment, the error complementing unit 102 holds a predetermined number of decoded images arranged in a time series sequentially decoded from the video ES by the video decoder 101 as held images. When the presence of an error location is detected in the retained image based on the error location information, the error complement source image is selected from the retained images before and after the time series with respect to the detected error complement destination retained image, and an error is detected. The error part of the retained image of the complement destination is error-compensated with the image information corresponding to the error part of the error complement source image. In this way, errors in the decoded image can be complemented using an image that was not decoded at the time of decoding the image, and only when the image that has been decoded at the time of decoding is used. Compared to this, it is possible to increase the number of image candidates that are complementary sources. As a result, effective error compensation can be expected.

  Further, according to the first embodiment, when the error complementation unit 102 detects the presence of an error location in the retained image based on the error location information, the encoding type in which the entire image is retained in a single image type frame. The held image that first appears after the time series with respect to the detected error-complemented retained image is selected as the error-complementing source image. Thereby, the influence of an error can be reduced. In particular, the decoded image having the image type P or B accumulates the error of the image referred to by the decoded image, whereas the decoded image having the image type I has no error accumulation. If error compensation is performed using a decoded image of type I frame, it is possible to minimize the influence of the error.

Embodiment 2. FIG.
In the second embodiment, an error portion in a decoded image is configured to be error-complemented by a more similar one of the preceding and subsequent decoded images. The overall configuration of the video decoding apparatus according to the second embodiment is the same as that shown in FIG. 1 in the first embodiment, but the configuration and operation of the error complementing unit are different. In the second embodiment, it is not necessary for the video decoder to acquire the image type, and the error complementing unit performs error compensation by inputting only error location information from the video decoder.

  FIG. 4 is a block diagram showing a configuration of an error complementation unit in the video decoding apparatus according to Embodiment 2 of the present invention. The error complementing unit (error complementing unit) 102 according to the second embodiment includes an image memory 401, a complementation planning unit 402, and a complementing operation unit 403. The complementation planning unit 402 includes an error detection unit 404 and similarity calculation units 405a and 405b. And a comparison unit 406.

  The error detection unit 404 sequentially inputs error location information from the video decoder 101, holds the held image held in the image memory 201, and detects the presence of an error with respect to the held image in the image memory 201 based on the error location information. Then, the retained image and the retained images before and after that are acquired from the image memory 201.

  The similarity calculation unit 405a calculates the similarity between the retained image in which the error acquired by the error detection unit 404 exists and the retained image existing in time with respect to the retained image. The similarity calculation unit 405b calculates the similarity between the retained image with the error acquired by the error detection unit 404 and the retained image existing later in time with respect to this retained image.

  The comparison unit 406 compares the similarities calculated by the similarity calculation units 405a and 405b, and among the stored images existing before and after in time from the comparison result, the comparison image is more similar to the stored image having an error. Information indicating the number of the retained image and the error location information are output to the complementary operation unit 403 as a complementary instruction.

  The complement operation unit 403 inputs a complement instruction from the complement planning unit 402, reads out image information about the portion indicated by the error location information of the number of retained images according to the complement instruction from the image memory 401, and retains an error. Complement processing is executed by writing in the part indicated by the error location information of the image.

Next, the operation will be described.
In the video decoder 101 according to the second embodiment, when the video ES of the moving image to be reproduced is input, the video ES is decoded, and the decoded image of the decoding result and the error location information of the decoded image are output to the error complementing unit 102. To do.

  In the error complementing unit 102, the image memory 401 sequentially inputs decoded images, and as shown in FIG. 4, N decoded images (N frames arranged in time series) from the newest in time are displayed. Hold as a holding image. Further, the error detection unit 404 of the supplementary planning unit 402 sequentially inputs and holds error location information from the video decoder 101.

  When the error detection unit 404 detects the presence of an error in the M (M is a predetermined natural number satisfying M ≦ N) number of held images in the image memory 401 based on the error location information, Are acquired from the image memory 401 and output to the similarity calculation units 405a and 405b.

  The similarity calculation unit 405a has a range of a predetermined width centered on the part specified by the error location information for the Mth retained image and the M−1th retained image acquired by the error detection unit 404. The similarity (first similarity) of the part (not including the part itself specified by the error part) is calculated. Similarly, the similarity calculation unit 405b calculates the similarity (second similarity) between the Mth retained image and the M + 1th retained image. Here, the similarity is, for example, a value indicating a higher value as the correlation is higher according to a calculation result such as a sum of absolute values of differences or a normalized correlation calculation with respect to the luminance of the pixels constituting the image.

  The comparison unit 406 inputs and compares the respective similarities calculated by the similarity calculation units 405a and 405b, and if the similarity calculated by the similarity calculation unit 405a is higher than the similarity calculation unit 405b, x = M −1, if the similarity calculated by the similarity calculation unit 405b is high, x = M + 1, and information indicating that the error complement source retained image is the xth image and the error location are used as a complement instruction to the complement operation unit 403. Output.

  In the complement operation unit 403, in accordance with the complement instruction input from the complement planning unit 402, the image information of the portion indicated by the error location information in the complement instruction in the x-th retained image is read from the image memory 401, and the M-th retained Complement processing is executed by writing in the part indicated by the error location information of the image.

  As described above, according to the second embodiment, the error complementing unit 102 is configured as shown in FIG. 4 so that the error part in the decoded image is more similar to the decoded image before and after that. It can be complemented with. Since complementation is performed from a more similar decoded image, it can be expected that the image will not be affected by sudden changes in the image such as a scene change.

Embodiment 3 FIG.
In the third embodiment, as in the second embodiment, the error part in the decoded image is configured such that the error is complemented by a more similar one of the preceding and succeeding decoded images. The overall configuration of the video decoding apparatus according to the third embodiment is the same as that shown in FIG. 1 in the first embodiment, but the configuration and operation of the error complementing unit are different. In the third embodiment, as in the second embodiment, it is not necessary for the video decoder to acquire the image type, and the error complementing unit performs error compensation by inputting only error location information from the video decoder. .

  FIG. 5 is a block diagram showing the configuration of the error complementation unit in the video decoding apparatus according to Embodiment 3 of the present invention. The error complementing unit (error complementing unit) 102 according to the third embodiment includes an image memory 601, a complement planning unit 602, and a complement operation unit 603. The supplement planning unit 602 includes an error detection unit 604, a similarity calculation unit 605, a threshold value. A memory 606 and a comparison unit 607 are provided.

  The error detection unit 604 sequentially inputs error location information from the video decoder 101, holds the held image held in the image memory 601, and detects the presence of an error from the error location information with respect to the held image in the image memory 601. Then, the retained images before and after the retained image are acquired from the image memory 601.

  The similarity calculation unit 605 calculates the similarity between held images that exist temporally before and after the held image with the error acquired by the error detection unit 604. The threshold memory 606 stores a predetermined threshold related to similarity.

  The comparison unit 607 compares the similarity calculated by the similarity calculation unit 605 with a predetermined threshold read from the threshold memory 606, and when the similarity exceeds the predetermined threshold from the comparison result, Information indicating the number of images held before and after each of the held images and the error location information are output as a complementary instruction to the complementary operation unit 603.

  The complement operation unit 603 receives a complement instruction from the complement planning unit 602, acquires from the image memory 601 the image information of the portion indicated by the error location information in each of the number of retained images according to the complement instruction, and outputs these intermediate images. And the interpolation processing is executed by writing the image information of the intermediate image into the portion indicated by the error location information of the retained image in which an error exists.

Next, the operation will be described.
In the video decoder 101 according to the third embodiment, when the video ES of the moving image to be reproduced is input, the video ES is decoded, and the decoded image of the decoding result and the error location information of the decoded image are output to the error complementing unit 102. To do.

  In the error complementing unit 102, the image memory 601 sequentially inputs the decoded images and, as shown in FIG. 5, N decoded images (N frames arranged in time series) from the newest in time are displayed. Hold as a holding image. In addition, the error detection unit 604 of the supplementary planning unit 602 sequentially inputs error location information from the video decoder 101 and holds error location information about N held images held in the image memory 601.

  When the error detection unit 604 detects the presence of an error based on the error location information in the M (M is a predetermined natural number satisfying M ≦ N) number of held images in the image memory 601, the error detection unit 604 detects the presence of an error between the M number of held images. The (M−1) th retained image and the (M + 1) th retained image that are temporally before and after are acquired from the image memory 601 and output to the similarity calculation unit 605.

  The similarity calculation unit 605 has a range of a predetermined width centered on a part specified by the error location information for the M−1th retained image and the M + 1th retained image acquired by the error detection unit 604. The similarity of the part (not including the part itself specified in the error part) is calculated. Here, the similarity is, for example, a value indicating a higher value as the correlation is higher according to a calculation result such as a sum of absolute values of differences or a normalized correlation calculation with respect to the luminance of the pixels constituting the image.

  The comparison unit 607 inputs and compares the similarity calculated by the similarity calculation unit 605 and the predetermined threshold read from the threshold memory 606, and the similarity calculated by the similarity calculation unit 605 is greater than the predetermined threshold. If it is higher, x = M−1 and y = M + 1 are output to the complementary operation unit 603 as information indicating that the retained image as the error complement source is the x-th image and the y-th image and the error location as a complement instruction.

  In the complement operation unit 603, according to the complement instruction input from the complement planning unit 602, the image information of the part indicated by the error location information in the x-th and y-th retained images is obtained from the image memory 601, respectively. Is generated and written in the portion indicated by the error location information of the M-th retained image to execute the complementing process. Note that, as the intermediate image, an average value of luminance of the two held images is used.

  As described above, according to the third embodiment, by configuring the error complementing unit 102 as shown in FIG. 5, it is expected that the error location in the decoded image is similar to both the decoded images before and after that. It is possible to supplement only when possible. Thereby, it is possible to perform complementation only when there is little change in the image, and it can be expected to avoid erroneous complementation.

Embodiment 4 FIG.
In the fourth embodiment, even if there is an error in the image immediately before or immediately after the error complement source, the error location in the decoded image is more similar to that in the decoded images before and after that as in the second embodiment. It is configured so that error correction is performed by the user. The overall configuration of the moving picture decoding apparatus according to the fourth embodiment is the same as that shown in FIG. 1 in the first embodiment, but the configuration and operation of the error complementing unit are different. In the fourth embodiment, as in the second embodiment, it is not necessary for the video decoder to acquire the image type, and the error complementing unit performs error compensation by inputting only error location information from the video decoder. .

  FIG. 6 is a block diagram showing the configuration of the error complementation unit in the video decoding apparatus according to Embodiment 4 of the present invention. The error complementing unit (error complementing unit) 102 according to the fourth embodiment includes an image memory 801, a complement planning unit 802, and a complement operation unit 803. The complement planning unit 802 includes an error detection unit 804 and similarity calculation units 805a and 805b. And a comparison unit 806.

  The error detection unit 804 sequentially inputs error location information from the video decoder 101, holds the held image held in the image memory 801, and detects the presence of an error from the error location information with respect to the held image in the image memory 801. Then, from the holding image immediately before the holding image in which the error is detected (hereinafter referred to as a complement target holding image), the holding image existing before that is sequentially referred to and the portion indicated by the error location information It is determined whether or not the same error location information exists at the position, and firstly, the holding image in which the error location information no longer exists is specified, and from the holding image immediately after the complement target holding image, The stored images existing thereafter are sequentially referenced to determine whether or not similar error location information exists at the position indicated by the error location information. Identifies the holding image over part information is no longer present, and acquires the respective holding image above complemented holding image and the specific from the image memory 801.

  The similarity calculation unit 805a calculates the similarity between the complement target holding image acquired by the error detection unit 804 and the holding image specified from the holding image existing in time. The similarity calculation unit 805b calculates the similarity between the complementing target retained image acquired by the error detection unit 804 and the retained image specified from the retained image existing later in time.

  The comparison unit 806 compares the similarities calculated by the similarity calculation units 805a and 805b, and determines which of the holding images existing before and after the comparison result is more similar to the complement target holding image. Specifically, information indicating the number of the retained image and the error location information are output to the complementary operation unit 803 as a complementary instruction.

  The complement operation unit 803 inputs a complement instruction from the complement planning unit 802, reads out image information about the portion indicated by the error location information of the number of retained images according to the complement instruction from the image memory 801, and stores the complement target retained image. Complementary processing is executed by writing in the portion indicated by the error location information.

Next, the operation will be described.
In the video decoder 101 according to the fourth embodiment, when the video ES of the moving image to be reproduced is input, the video ES is decoded, and the decoded image of the decoding result and the error location information of the decoded image are output to the error complementing unit 102. To do.

  In the error complementing unit 102, the image memory 801 sequentially inputs the decoded images and, as shown in FIG. 6, N decoded images (N frames arranged in time series) from the newest in time are displayed. Hold as a holding image. In addition, the error detection unit 804 of the supplementary planning unit 802 sequentially inputs error location information from the video decoder 101 and holds error location information about N held images held in the image memory 801.

  When the error detection unit 804 detects the presence of an error in the M (M is a predetermined natural number satisfying M ≦ N) number of held images in the image memory 801 based on the error location information, k is set to the M−kth held image. While sequentially increasing from 1, it is determined whether similar error location information exists at the position of the portion indicated by the error location information, and the value of k of the retained image in which error location information no longer exists is first set to i. As specified. Further, the error detection unit 804 determines whether or not similar error location information exists at the position of the portion indicated by the error location information while increasing k from the 1st M + k-th retained image in order. The value of k of the retained image in which error location information no longer exists is specified as j. After that, the error detection unit 804 obtains the Mth retained image, the M−ith retained image, and the M + jth retained image to be complemented from the image memory 801, and the similarity calculation units 805a and 805b. Output to.

  The similarity calculation unit 805a has a range of a predetermined width centered on a part specified by the error location information for the Mth retained image and the M-ith retained image acquired by the error detection unit 804. The similarity (third similarity) of the part (not including the part itself specified by the error part) is calculated. In addition, the similarity calculation unit 805b has a predetermined range of the center of the portion specified by the error location information regarding the Mth retained image and the M + jth retained image acquired by the error detection unit 804. The similarity (fourth similarity) of the part (not including the part itself specified by the error part) is calculated. Here, the similarity is, for example, a value indicating a higher value as the correlation is higher according to a calculation result such as a sum of absolute values of differences or a normalized correlation calculation with respect to the luminance of the pixels constituting the image.

  The comparison unit 806 inputs and compares both similarities calculated by the similarity calculation units 805a and 805b, and if the similarity calculated by the similarity calculation unit 805a is higher than the similarity calculation unit 805b, x = M− If i is high and the similarity calculated by the similarity calculation unit 805b is high, y = M + j, and information indicating that the retained image of the error complement source is the Mi-th or M + j-th image and the error location are instructed To the complementary operation unit 803.

  In the complement operation unit 803, according to the complement instruction input from the complement planning unit 802, the image information of the portion indicated by the error location information in the M−i th or M + j th retained image is acquired from the image memory 801, and the M th The supplement processing is executed by writing in the portion indicated by the error location information of the retained image.

  As described above, according to the fourth embodiment, by configuring the error complementing unit 102 as shown in FIG. 6, even when there is an error in the image immediately before or immediately after the error complementing source, The same effect as in the second embodiment can be obtained.

Embodiment 5 FIG.
In the fifth embodiment, even if there is an error in the image immediately before or immediately after the error complement source, the error location information in the decoded image is similar to both the preceding and subsequent decoded images as in the third embodiment. It is possible to supplement only when it can be expected. The overall configuration of the moving picture decoding apparatus according to the fifth embodiment is the same as that shown in FIG. 1 in the first embodiment, but the configuration and operation of the error complementing unit are different. In the fifth embodiment, as in the second embodiment, it is not necessary for the video decoder to acquire the image type, and the error complementing unit performs error compensation by inputting only error location information from the video decoder. .

  FIG. 7 is a block diagram showing a configuration of an error complementation unit in the video decoding apparatus according to Embodiment 5 of the present invention. The error complementing unit (error complementing unit) 102 according to the fifth embodiment includes an image memory 901, a complementing planning unit 902, and a complementing operation unit 903. The complementing planning unit 902 includes an error detection unit 904, a similarity calculation unit 905, a threshold value. A memory 906 and a comparison unit 907 are provided.

  The error detection unit 904 sequentially inputs error location information from the video decoder 101, holds the held image held in the image memory 901, and detects the presence of an error from the error location information with respect to the held image in the image memory 901. Then, from the holding image immediately before the holding image in which the error is detected (hereinafter referred to as a complement target holding image), the holding image existing before that is sequentially referred to and the portion indicated by the error location information It is determined whether or not there is similar error location information at the position, and the first holding image in which the error location information no longer exists is identified, and from the holding image immediately after the complement target holding image, The stored images existing thereafter are sequentially referenced to determine whether or not similar error location information exists at the position indicated by the error location information. Identify the holding image over part information is no longer present, and acquires the respective holding image identified from the image memory 901.

  The similarity calculation unit 905 calculates the similarity between each retained image acquired by the error detection unit 904. The threshold memory 906 stores a predetermined threshold related to similarity. The comparison unit 907 compares the similarity calculated by the similarity calculation unit 905 with a predetermined threshold read from the threshold memory 906, and when the similarity exceeds the predetermined threshold based on this comparison result, Information indicating how many images the retained images exist before and after and the error location information are output to the complementary operation unit 903 as complementary instructions.

  The complement operation unit 903 receives a complement instruction from the complement planning unit 902, acquires the image information of the portion indicated by the error location information in each held image of the number according to the complement instruction from the image memory 901, and outputs these intermediate images. Is generated, and the interpolation process is executed by writing the image information of the intermediate image into the portion indicated by the error location information of the complement target holding image.

Next, the operation will be described.
In the video decoder 101 according to the fifth embodiment, when the video ES of the moving image to be reproduced is input, the video ES is decoded, and the decoded image of the decoding result and the error location information of the decoded image are output to the error complementing unit 102. To do.

  In the error complementing unit 102, the image memory 901 sequentially inputs the decoded images, and as shown in FIG. 7, N decoded images (N frames arranged in time series) from the newest in time are displayed. Hold as a holding image. In addition, the error detection unit 904 of the supplementary planning unit 902 sequentially inputs error location information from the video decoder 101 and holds error location information about N held images held in the image memory 901.

  When the error detection unit 904 detects the presence of an error in the M (M is a predetermined natural number satisfying M ≦ N) image in the image memory 901 based on the error location information, the error detection unit 904 sets k to the Mk image. While sequentially increasing from 1, it is determined whether similar error location information exists at the position of the portion indicated by the error location information, and the value of k of the retained image in which error location information no longer exists is first set to i. As specified. Further, the error detection unit 904 determines whether or not the same error location information exists at the position of the portion indicated by the error location information while increasing k from 1 in the M + k-th retained image in order. The value of k of the retained image in which error location information no longer exists is specified as j. Thereafter, the error detection unit 904 acquires the M−ith retained image and the M + jth retained image from the image memory 901, and outputs them to the similarity calculation unit 905.

  The similarity calculation unit 905 has a range of a predetermined width centered on a part specified by the error location information with respect to the M−ith retained image and the M + jth retained image acquired by the error detection unit 904. The similarity of the part (not including the part itself specified in the error part) is calculated. Here, the similarity is, for example, a value indicating a higher value as the correlation is higher according to a calculation result such as a sum of absolute values of differences or a normalized correlation calculation with respect to the luminance of the pixels constituting the image.

  The comparison unit 907 inputs and compares the similarity calculated by the similarity calculation unit 905 and the predetermined threshold read from the threshold memory 906, and the similarity calculated by the similarity calculation unit 905 is greater than the predetermined threshold. If it is higher, x = M−i, y = M + j, and information indicating that the retained image of the error complement source is the xth and yth images and the error location are output to the complement operation unit 903 as a complement instruction.

  In the complement operation unit 903, according to the complement instruction input from the complement planning unit 902, the image information of the portion indicated by the error location information in the x-th and y-th retained images is obtained from the image memory 901, and these intermediate images are obtained. Is generated and written in the portion indicated by the error location information of the M-th retained image to execute the complementing process. Note that, as the intermediate image, an average value of luminance of the two held images is used.

  As described above, according to the fifth embodiment, by configuring the error complementing unit 102 as shown in FIG. 7, even when there is an error in the image immediately before or immediately after the complementing source, The same effect as in the third mode can be obtained.

Embodiment 6 FIG.
FIG. 8 is a block diagram showing a configuration of a moving picture decoding apparatus according to Embodiment 6 of the present invention. In FIG. 8, the moving picture decoding apparatus according to the sixth embodiment includes a video decoder 101a, an error complementing unit 102a, and an error tracking unit 103. The video decoder (decoding unit) 101 decodes (decodes) the video ES, decodes the decoded image, an image type indicating the representation format of the decoded image in the video ES, motion vector information of the decoded image, and This is means for sequentially outputting error location information (coordinates of a portion where an error has occurred) of the decoded image. Note that the motion vector information of the decoded image is information including the reference source decoded image, the reference source coordinates, and the movement amount of the decoded image.

  The error complementing unit (error complementing unit) 102a inputs the decoded image and the image type from the video decoder 101a, inputs the tracking error location information from the error tracking unit 103, and performs error compensation described in the first to fifth embodiments. To output complementary images sequentially. For example, a decoded image, an image type, and tracking error location information are input, and the decoded image is complemented according to the image type and tracking error location information, and the complement result is obtained as a complementary image.

  The error tracking unit (error tracking means) 103 receives the motion vector information of the decoded image and the detected error location information from the video decoder 101a, and is specified by the error location information in the decoded image sequentially decoded by the video decoder 101a. This is a means for tracking the spillover range of the error and outputting the error location information of the decoded image affected by the error as tracking error location information.

Next, the operation will be described.
When the video decoder 101a receives a video ES of a moving image to be played back, the video decoder 101a decodes the video ES, and displays an error type indicating a decoded image as a decoding result and an image type indicating an expression format of the decoded image in the video ES. 102a, and outputs the motion vector information of the decoded image and error location information detected from the decoded image to the error tracking unit 103.

  In the error tracking unit 103, when the coordinates of the part specified by the error location information input from the video decoder 101a in the reference source decoded image included in the motion vector information overlap with the reference source coordinates, The error location coordinates are moved by the movement amount included in the motion vector information, and the destination coordinate (reference location coordinates of the error location) is added as the error location coordinates to be output to the error complementing unit 102a as tracking error location information. To do.

  The error complementing unit 102a treats the tracking error location information input from the error tracking unit 103 in the same manner as the error location information described in the first to fifth embodiments, and performs error compensation described in the first to fifth embodiments. Is executed, and complementary images are sequentially output.

  As described above, according to the sixth embodiment, since the error tracking unit 103 shown in FIG. 8 is provided, it is possible to track an error that has spread by referring to an error image and use it as a complement target.

  101, 101a Video decoder (decoding unit), 102, 102a Error complementing unit (error complementing unit), 103 Error tracking unit (error tracking unit), 201, 401, 601, 801, 901 Image memory, 202, 402, 602 802, 902 complement planning unit, 203, 403, 603, 803, 903 complement operation unit, 405a, 405b, 605, 805a, 805b, 905 similarity calculation unit, 406, 607, 806, 907 comparison unit, 606, 906 threshold memory.

Claims (10)

Decoding that decodes an input image stream indicating moving image information to obtain a decoded image, detects an error portion on the decoded image due to an error in the moving image information, and generates error portion information indicating the error portion Means,
When a predetermined number of decoded images arranged in time series sequentially decoded by the decoding unit are held as a retained image, and the presence of an error location is detected in the retained image by the error location information, the detected error compensation is performed The error complement source image is selected from the retained images before and after the time series with respect to the previous retained image, and the error part of the retained image of the error complement destination is the image information corresponding to the error part of the error supplement source image. A moving picture decoding apparatus comprising error complementing means for error compensation. When decoding the image stream indicating the input moving image information, the decoding means acquires an image type indicating an expression format in the moving image information of the decoded image,
When the presence of an error location is detected in the retained image based on the error location information, the error complementing means is an image represented by an encoding type in which the entire image is retained in a single frame. The retained image that appears first after the time series with respect to the retained image of the error complement destination is selected as the error complement source image, and the error location of the retained image of the error complement destination corresponds to the error location of the error complement source image The moving image decoding apparatus according to claim 1, wherein error compensation is performed using image information to be processed.   3. The moving picture decoding apparatus according to claim 2, wherein an image represented by an encoding type in which the entire image is held in a single image type is an I frame in MPEG format. Error completion means are
An image memory for holding a predetermined number of decoded images arranged in time series sequentially decoded by the decoding means as a retained image;
When the presence of an error location is detected in the retained image based on the error location information input from the decoding unit, the detected error complement destination retained image, the retained image immediately before the time series with respect to the error complement destination retained image, and A retained image immediately after the time series is acquired from the image memory, and a range of a predetermined area around the error portion between the retained image at the error complement destination and the retained image immediately before the time series A first similarity is calculated, and a second similarity is calculated for a range of a predetermined area around the error portion between the error complement destination retained image and the retained image immediately after the time series. A complement planning unit that calculates and selects, from the comparison result between the first and second similarities, the higher-similarity of the retained images immediately before or after the time series as the error complementing source image ,
And a complement operation unit that complements an error part of the error complement destination retained image with image information corresponding to the error part of the error complement source image selected by the complement plan part. Item 4. A moving picture decoding apparatus according to Item 1. Error completion means are
An image memory for holding a predetermined number of decoded images arranged in time series sequentially decoded by the decoding means as a retained image;
When the presence of an error location is detected in the retained image based on the error location information input from the decoding unit, the retained image immediately before the time series and the retained image immediately after the time series with respect to the detected retained image of the error complement destination Is calculated from the image memory, and a similarity is calculated for a range of a predetermined width around the error portion between the retained image immediately before the time series and the retained image immediately after the time series. If the similarity is higher than the threshold from the comparison result with a predetermined threshold, a complement planning unit that generates an intermediate image of the retained images immediately before and immediately after the time series and serves as an error complement source image,
And a complement operation unit that complements an error part of the error complement destination retained image with image information corresponding to the error part of the error complement source image obtained by the complement planning unit. Item 4. A moving picture decoding apparatus according to Item 1. Error completion means are
An image memory for holding a predetermined number of decoded images arranged in time series sequentially decoded by the decoding means as a retained image;
When the presence of an error location is detected in the retained image based on the error location information input from the decoding unit, the detected error supplement destination retained image is acquired from the image memory, and the error supplement destination retained image is obtained. It is determined whether there is an error location at the position indicated by the error location information by sequentially referring to the retention image existing before the retention image immediately before the time series, and the error location no longer exists first The retained image is acquired from the image memory, and the error location is indicated at the position indicated by the error location information by sequentially referring to the retained image immediately after the retained image immediately after the time series with respect to the retained image of the error complement destination. A determination is made as to whether or not there is an error, and a retained image that no longer has an error portion is first obtained from the image memory, and the error A third similarity is calculated for a range of a predetermined width around the error portion between the complete retained image and the retained image acquired from the previous retained images immediately before the time series. A fourth range of a predetermined area around the error portion between the error complement destination retained image and a retained image acquired immediately after the time series and thereafter. Calculating a similarity, and selecting a retained image having a high similarity from the comparison result between the third and fourth similarities as an error complementing source image;
And a complement operation unit that complements an error part of the error complement destination retained image with image information corresponding to the error part of the error complement source image obtained by the complement planning unit. Item 4. A moving picture decoding apparatus according to Item 1. Error completion means are
An image memory for holding a predetermined number of decoded images arranged in time series sequentially decoded by the decoding means as a retained image;
When the presence of an error location is detected in the retained image based on the error location information input from the decoding unit, the retained image existing before that from the retained image immediately before the time series with respect to the detected retained image of the error complement destination Are sequentially referred to to determine whether or not there is an error location at the position indicated by the error location information, and a retained image in which no error location exists first is acquired from the image memory, and the error complement destination is retained. With respect to the image, the retained image immediately after the time series is sequentially referred to to determine whether or not there is an error location at the position indicated by the error location information. The lost retained image is acquired from the image memory, and the retained image acquired from the previous retained images immediately before the time series and the previous The degree of similarity is calculated for a range of a predetermined area around the error portion between the stored images acquired immediately after the time series and the stored images after that, and the result of comparison with a predetermined threshold is calculated. If the similarity is higher than the threshold value, a complement planning unit that generates an intermediate image of the retained images immediately before and after the time series and serves as an error complement source image;
And a complement operation unit that complements an error part of the error complement destination retained image with image information corresponding to the error part of the error complement source image obtained by the complement planning unit. Item 4. A moving picture decoding apparatus according to Item 1.   8. The moving picture decoding apparatus according to claim 5, wherein the supplementary planning unit generates an intermediate image by taking an average of the pixel values of the two held images. The decoding unit generates motion vector information of the decoded image when decoding the image stream indicating the moving image information and obtaining the decoded image,
When the coordinates of the error location of the retained image of the error complement destination detected by the error location information overlaps the reference source coordinates of the reference source decoded image for the retained image of the error complement destination specified by the motion vector information, The error tracking means for generating the tracking error location information by moving the coordinates of the error location from the reference source coordinates by the movement amount included in the motion vector information and adding the coordinates of the movement destination as the coordinates of the error location,
The moving image according to any one of claims 1 to 8, wherein the error complementing means performs error compensation by treating the tracking error location information input from the error tracking means as error location information. Image decoding device.   10. The moving picture decoding apparatus according to claim 1, wherein the moving picture information is information encoded in an MPEG-Video format.

Images