JP5292355B2 - Signal correction apparatus, encoding apparatus, decoding apparatus, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a signal correction device which corrects a component signal of a moving image, an encoder, and a decoder. <P>SOLUTION: A signal correction device 10 as a first embodiment of the present invention includes: a teaching signal generation unit 11 which inputs a reference image signal and a first component signal of a decoded signal of an image signal, and normalizes the first component based upon the reference image signal to generate it as a teaching signal for a second component signal; a correction determination unit 12 which generates flag information representing a correction in a signal element to be corrected; a correction processing unit 13 which corrects the signal element to be corrected with the signal element of the teaching signal according to the flag information for the second component signal, and thus generates the second component signal having been corrected; a correction estimation unit 14 which generates estimation flag information obtained by comparing a signal element of the normalized first component signal with the signal element of the second component signal of the decoded signal; and a flag comparison unit 15 which compares the flag information and estimation flag information with each other to generate flag correction information representing a correction in an unmatched signal element. <P>COPYRIGHT: (C)2012,JPO&amp;INPIT

Description

  The present invention relates to an image processing technique, and more particularly to a signal correction device, an encoding device, a decoding device, and a program for correcting an image signal that has been encoded by an irreversible encoding method and has undergone encoding degradation.

  As a signal interpolation technique using the correlation between component signals constituting an image signal, a technique for improving the accuracy of a pixel interpolation method in a single-plate primary color image sensor has been reported (for example, see Non-Patent Document 1). Since the image signal interpolation technique in such an image sensor is intended to interpolate RGB signals (R: red signal, G: green signal, B: blue signal) in the RGB color space, signal deterioration due to encoding is assumed. It has not been.

  Further, as a signal interpolation technique focusing on the difference in sampling frequency of YUV signals in the YUV color space, a color difference component interpolation technique for format color images has been reported (for example, see Non-Patent Document 2). In this technique, high-precision interpolation is performed by generating an interpolated signal of a color difference signal (U signal = BY, V signal = RY) using the sampling frequency of the luminance (Y) signal. . The signal interpolation technique that pays attention to such a difference in the sampling frequency of the YUV signal is also intended for the interpolation of the YUV signal, so that signal degradation due to encoding is not assumed.

  These signal interpolation techniques are suitable for interpolation on an image signal before encoding when encoding an image signal by an irreversible encoding method (for example, MPEG-2, H.264, etc.). It is not suitable for interpolation on the image signal after conversion. For example, when a YUV signal is encoded by irreversible encoding processing, the deterioration of the luminance signal propagates to the color difference signal based on the luminance signal as the luminance signal deteriorates. In addition, these signal interpolation techniques are not processes for reducing the degradation of the luminance signal itself, and therefore do not reduce the degradation of the luminance signal.

  In addition, there are various deblocking filters (for example, deblocking filters in H.264 etc.) in order to reduce the degradation of encoding, but these deblocking filters visually degrade each of the image signal components. Are processed independently so as not to stand out, and deterioration after encoding of the original image signal cannot be reduced.

Kuno, Sugiura, “Higher accuracy of pixel interpolation in single-plate primary color image sensor”, Journal of the Institute of Image Information and Television Engineers, Vol. 61, no. 7. July 1, 2007, pp. 1006 to 1016 Sugita, Taguchi, "YUV4: 2: 0 format color difference component interpolation method", IEICE Transactions, Vol. J88-A, no. 6, June 1, 2005, pp. 751-760

  In the conventional signal interpolation technique that pays attention to the difference in sampling frequency of the YUV signal in the YUV color space, signal interpolation is performed based on the signal deterioration caused at the time of encoding. Therefore, the effect of reducing deterioration of the YUV signal before encoding is reduced. It does not play.

  Even if the component signals representing the same area of the decoded image are compared and corrected, the correction is not necessarily correct for the YUV signal before encoding.

  In view of the above-described problems, an object of the present invention is to provide a signal correction device, an encoding device, a decoding device, and a program that can correct a YUV signal before encoding.

  The signal correction apparatus according to the present invention compares signal degradation caused during encoding between component signals representing the same region of a decoded image (hereinafter, typically described with respect to a YUV signal), and performs encoding. Based on the previous YUV signal (hereinafter referred to as “reference YUV signal”), the signal element of the component signal (for example, the pixel value of the block or the orthogonal transformation coefficient) to be corrected is determined, and the signal element of the component signal is corrected. The estimation flag is generated by generating flag information representing a location and / or a normalization parameter representing a modification amount of a signal element of a component signal, and comparing and estimating between component signals representing the same region of a decoded image. Only when the information and the estimated correction parameter are generated and the flag information and the estimated flag information are compared and different, Sends the flag correction information representing the difference amount to the outside, only when the difference by comparing the normalized parameter and the estimated correction parameters, and sends the parameter correction information representing the difference amount to the outside. The component signal may be in any color space such as RGB, YCrCb, LUV, Lab, XYZ.

  That is, the signal correction apparatus according to the first aspect of the present invention is a signal correction apparatus that corrects a component signal of a moving image including a signal element of a pixel value or an orthogonal transform coefficient, and performs encoding through an irreversible encoding method. The image signal of the predetermined image format before the input is input as a reference image signal, and the first component signal of the decoded signal in the image signal is input and normalized based on the reference image signal. One component signal is generated as a teacher signal of a second component signal of the decoded signal, input from the second component signal of the decoded signal, and the teacher signal generator A correction determination means for comparing the signal element with the teacher signal to be generated and generating flag information indicating a correction portion of the signal element to be corrected; A correction process for correcting the signal component to be corrected with the signal component of the teacher signal in accordance with the flag information with respect to the second component signal of the signal and generating a corrected second component signal And a signal element obtained by comparing the signal element of the first component signal normalized based on the second component signal with the signal element of the second component signal of the decoded signal Correction estimation means for generating estimated flag information representing a location; and flag comparison means for comparing the flag information with the estimated flag information to generate flag correction information representing a corrected location of a different signal element. It is characterized by.

  In the signal correction device according to the first aspect of the present invention, the teacher signal generation means and the correction estimation means adjust the amplitude of the corresponding first component signal based on the corresponding second component signal. It is characterized by performing normalization.

  Further, in the signal correction device according to the first aspect of the present invention, the flag information represents a correction point in signal element units or divided block units for the signal elements of each component signal.

  Further, in the signal correction device according to the first aspect of the present invention, the teacher signal generation unit is configured to perform processing between the second component signal of the decoded signal and the teacher signal input from the teacher signal generation unit. Means for comparing the signal elements and generating a normalization parameter representing a correction amount of the signal element to be corrected, wherein the correction estimation means is the first normalized based on the second component signal Means for generating an estimated correction parameter representing a correction amount of the signal element obtained by comparing the signal element of the component signal with the signal element of the second component signal of the decoded signal, and the normalization parameter; The apparatus further comprises parameter comparison means for comparing the estimated correction parameters and generating parameter correction information representing a correction amount of a different signal element.

Furthermore, the signal correction device according to the second aspect of the present invention is a signal correction device for correcting a component signal of a moving image including a signal element of a pixel value or an orthogonal transform coefficient, and is obtained through an irreversible encoding method. The first component obtained by inputting and normalizing the first component signal of the decoded signal in the image signal of the predetermined image format, and normalizing based on the second component signal of the decoded signal in the image signal Generating estimated flag information indicating a correction portion of the signal element obtained by comparing the signal element of the signal with the signal element of the second component signal of the decoded signal, and the normalized first component Modified estimation means for generating a signal as a teacher signal of a second component signal of the decoded signal;
Input flag correction information indicating a correction portion of a different signal element obtained by comparing flag information indicating a correction portion of a signal element to be corrected and the estimation flag information, and performing the estimation based on the flag correction information Flag correction determination means for correcting flag information and reproducing the flag information, and correcting the signal element to be corrected with the signal element of the teacher signal according to the flag information, and correcting the second component signal after correction Correction processing means to be generated.

  Furthermore, the signal correction apparatus according to the third aspect of the present invention is a signal correction apparatus that corrects a component signal of a moving image including a signal element of a pixel value or an orthogonal transform coefficient, and is obtained through an irreversible encoding method. The first component obtained by inputting and normalizing the first component signal of the decoded signal in the image signal of the predetermined image format, and normalizing based on the second component signal of the decoded signal in the image signal Generating estimated flag information indicating a correction portion of the signal element obtained by comparing the signal element of the signal with the signal element of the second component signal of the decoded signal, and based on the second component signal The normalized signal component of the first component signal is obtained by comparing the signal component of the second component signal of the decoded signal. A correction estimation unit for generating an estimated correction parameter representing a correction amount of a signal element to be corrected, a flag information indicating a correction part of the signal element to be corrected, and the correction flag information obtained by comparing the estimated flag information Flag correction information indicating the amount of correction of the signal element to be corrected, and flag correction determination means for correcting the estimated flag information based on the flag correction information and reproducing the flag information The parameter correction information representing the correction amount of the different signal element obtained by comparing the estimated correction parameter and the estimated correction parameter is input, the estimated correction parameter is corrected based on the parameter correction information, and the normalized parameter is reproduced. Parameter correction determination means, teacher signal generation means for generating a teacher signal of a second component signal of the decoded signal, According lag information, it modifies the signal elements to the corrected in signal component of the teacher signal, characterized by comprising a correction processing means for generating a second component signal after correction, a.

  Furthermore, each of the signal correction apparatuses of the first to third aspects can be configured by a computer.

  According to the present invention, since the YUV signal is corrected based on the YUV signal before encoding, the image deterioration can be further reduced as compared with the case where the present invention is not applied.

It is a block diagram of the encoding side signal correction apparatus of one Example by this invention. It is a block diagram of the encoding side signal correction apparatus of one Example by this invention. It is a figure which shows the structural example which corrects a to-be-corrected YUV signal by replacement | exchange of an orthogonal transformation coefficient as a signal element of a component signal in the encoding side signal correction apparatus of one Example by this invention. It is a figure which shows the structural example which corrects a to-be-corrected YUV signal by replacement | exchange of an orthogonal transformation coefficient as a signal element of a component signal in the encoding side signal correction apparatus of one Example by this invention. (A) shows how the U and V signals of the color difference signal in the 4: 2: 0 format are corrected using the down-converted luminance signal Y, and (b) shows the luminance in the 4: 2: 0 format. It is a figure which shows a mode that the signal Y is corrected using the U signal or V signal of an up-converted color difference signal. (A) shows how the U and V signals of the color difference signal in the 4: 2: 2 format are corrected using the down-converted luminance signal Y, and (b) shows the luminance in the 4: 2: 2 format. It is a figure which shows a mode that the signal Y is corrected using the U signal or V signal of an up-converted color difference signal. It is the schematic of the encoding apparatus to which the encoding side signal correction apparatus of one Example by this invention is applied. It is a block diagram of the decoding side signal correction apparatus of one Example by this invention. It is a block diagram of the decoding side signal correction apparatus of one Example by this invention. It is a figure which shows the structural example which corrects a to-be-corrected YUV signal by replacement | exchange of an orthogonal transformation coefficient as a signal element of a component signal in the decoding side signal correction apparatus of one Example by this invention. It is the schematic of the decoding apparatus to which the decoding side signal correction apparatus of one Example by this invention is applied. It is an operation | movement flow of the encoding side signal correction apparatus of one Example by this invention. It is an operation | movement flow of the decoding side signal correction apparatus of one Example by this invention. (A) is a figure which shows the example which corrects the signal element of a component signal per signal element, (b) is a figure which shows the example which corrects in the unit of the block divided | segmented about the signal element of the component signal. It is a figure which illustrates the relationship between the luminance signal and color difference signal in a to-be-corrected YUV signal, and the YUV signal after correction. It is a block diagram of the encoding side signal correction apparatus of the modification by this invention. (A) is a figure which shows the example of a signal of 4: 2: 2 format in a frame image, (b) is a figure which shows the example of a signal of 4: 2: 0 format in a frame image.

  A signal correction apparatus according to an embodiment of the present invention will be described below.

  First, in order to understand the present invention, a general image format used in image coding will be described. In general, in image encoding, one image frame is composed of a combination of signals of different pixel sizes because of differences in sensitivity to luminance and color differences in human perception. As such an image format, as shown in FIG. 17A, for the pixel S1 in the frame image F, the 4: 2: 2 format (in the horizontal direction, the number U of pixels of the luminance signal (Y) is 1). As shown in FIG. 17B, the number of pixels of the signal and the V signal is 1/2), and for the pixel S2 in the frame image F, the 4: 2: 0 format (the luminance signal in the horizontal and vertical directions). The number of pixels of each of the U signal and the V signal is ½) with respect to the number of pixels 1.

  In a typical block coding such as MPEG, such an image signal is coded for every certain number of pixels without distinguishing between a luminance signal and a color difference signal. For this reason, the image ranges occupied by the encoding blocks of the luminance signal and the color difference signal are different, and the range in which the encoding deterioration occurs is also different.

  The difference in the number of samplings expressed by the luminance signal and the color difference signal means that the sampling frequency of the luminance signal and the color difference signal is different. For example, in the case of 4: 2: 2 format, two color difference signals (U signal and V signal) corresponding to an image range formed by luminance signals of 16 × 16 pixels are each configured by 8 × 16 pixels. Therefore, in the case of the 4: 2: 2 format, the sampling frequency of the color difference signal is halved with respect to the luminance signal in the horizontal direction. Similarly, in the 4: 2: 0 format, the sampling frequency of the color difference signal is halved with respect to the luminance signal in the horizontal and vertical directions.

  In the following description, typically, based on the reference YUV signal (reference color difference signal), the signal element of the color difference signal (U signal / V signal) in the corrected YUV signal is represented in the corrected YUV signal. An example in which a corrected YUV signal (corrected color difference signal) is obtained by correcting with a signal element of a teacher signal generated from the luminance signal (Y signal) will be described.

  From the relationship between the component signals shown in FIG. 17, the luminance signal (Y signal) of the corrected YUV signal and the corrected YUV signal of the corrected YUV signal are based on the reference YUV signal (reference luminance signal). The signal component of the luminance signal (Y signal) in the corrected YUV signal is compared with the color difference signal (U signal / V signal) and the signal component of the color difference signal (U signal / V signal) in the corrected YUV signal. It is also possible to obtain a corrected YUV signal (corrected luminance signal) by correcting with elements. Therefore, both the luminance signal and the color difference signal can be corrected from each other from the correlation between the luminance signal and the color difference signal.

  In the following description, an example in which the YUV signal is composed of a luminance signal and a color difference signal having different sampling frequencies as shown in FIG. 17 will be described. The reference YUV signal may be considered as a YUV signal before encoding, and the corrected YUV signal may be considered as a decoded signal of the encoded YUV signal.

  FIG. 1 is a block diagram of an encoding-side signal modification device 10 according to an embodiment of the present invention. The encoding-side signal correction device 10 includes a teacher signal generation unit 11, a correction determination unit 12, a correction processing unit 13, a correction estimation unit 14, and a flag comparison unit 15.

  The encoding-side signal correction device 10 inputs, for example, a corrected YUV signal of a predetermined image format output from a decoder (not shown) that decodes an image signal encoded by an irreversible encoding method, Using the difference in sampling frequency between component signals, a new YUV signal (modified luminance signal and color difference signal) is generated in accordance with the predetermined image format while suppressing encoding deterioration of the luminance signal and color difference signal. .

  That is, the teacher signal generation unit 11 receives the luminance signal of the corrected YUV signal and the color difference signal of the reference YUV signal, and based on the color difference signal of the reference YUV signal, The normalized luminance signal is generated, and the normalized luminance signal is generated as a teacher signal of the color difference signal, and is transmitted to the correction determination unit 12 and the correction processing unit 13.

  Here, an example in which a normalized luminance signal is generated as a teacher signal of a color difference signal will be described.

As shown in the following equation, the absolute value is the maximum value among the values in the component coordinates (u, v) of the orthogonal transform coefficient of each color difference signal (excluding the (0, 0) component which is a direct current component). When the component coordinates are given by (a, b), the teacher signal T_U [v] [u] for the U signal and the teacher signal T_V [v] [u] for the V signal are orthogonal transform coefficients U [ b] [a] (or V [b] [a]) The luminance signal is orthogonalized with a sampling rate equal to the ratio based on the orthogonal transformation coefficient Y [b] [a] of the luminance signal of the component coordinates. It can be obtained by normalizing each component of the conversion coefficient Y [v] [u].
T_U [v] [u] = Y [v] [u] * U [b] [a] / Y [b] [a]
T_V [v] [u] = Y [v] [u] * V [b] [a] / Y [b] [a]
Here, u and v are 0 to 7 for an 8 × 8 pixel block. The normalization coefficient here is given by U [b] [a] / Y [b] [a] or V [b] [a] / Y [b] [a].

The teacher signal T_Y [v] [u] for the luminance signal can be obtained in the same manner. For example, as shown in the following equation, the maximum absolute value of the values in the component coordinates (u, v) of the orthogonal transformation coefficient of the luminance signal (excluding the (0, 0) component which is a direct current component) Is given by (a, b), the teacher signal T_Y [v] [u] for the luminance signal has a component coordinate corresponding to the orthogonal transformation coefficient Y [b] [a] of the luminance signal. An orthogonal transform coefficient U [v] [u] (or a color difference signal with a sampling rate equal to the ratio based on the orthogonal transform coefficient U [b] [a] (or V [b] [a]) of the color difference signal. V [v] [u]) can be obtained by normalizing each component.
T_Y [v] [u] = U [v] [u] * Y [b] [a] / U [b] [a], or
= V [v] [u] * Y [b] [a] / V [b] [a]
The normalization coefficient here is given by Y [b] [a] / U [b] [a] or Y [b] [a] / V [b] [a].
Since the luminance signal is composed of a plurality of orthogonal transform blocks with respect to the chrominance signal, for example, in the case of 4: 2: 0 format, the chrominance signal up-converted to equalize the sampling rate is divided into four luminances. Normalization is performed for each block with respect to the orthogonal transform block of the signal.

  Since two teacher signals T_Y [v] [u] for a certain luminance signal are obtained, the larger one of these two teacher signals is used as the teacher signal or the average of the absolute values A teacher signal having a larger value (or mean square value) is used as a teacher signal, a teacher signal having a higher correlation of two teacher signals is selected, or a teacher selected based on a predetermined selection criterion The teacher signal can be calculated in a manner suitable for the purpose of use of the image processing apparatus 1.

  In the above example, the example in which the maximum value of the absolute value of the orthogonal transform coefficient is detected and normalized in order to generate the teacher signal has been described, but the average value (or the mean square value) of the absolute value of the orthogonal transform coefficient is described. It is also possible to normalize based on.

  Based on the color difference signal in the input reference YUV signal, the correction determination unit 12 is configured to generate a signal element between the color difference signal in the corrected YUV signal and the teacher signal input from the teacher signal generation unit 11. (For example, pixel values in the block or orthogonal transform coefficients) are compared, and flag information representing the correction location (coordinates) of the signal element to be corrected for the color difference signal in the YUV signal to be corrected is generated and the flag comparison unit 15 and to the correction processing unit 13. In this comparison processing, since the signal element of the reference YUV signal is the original correct signal, in this case, based on the signal element of the color difference signal in the reference YUV signal, the teacher is instructed about the color difference signal in the corrected YUV signal. The signal element correction point (coordinates) to be corrected is specified.

  The correction processing unit 13 receives the color difference signal of the corrected YUV signal, and corrects the signal element to be corrected for the color difference signal of the corrected YUV signal according to the flag information input from the correction determination unit 12. The position (coordinates) is corrected (for example, replaced) with the signal element of the teacher signal, and a YUV signal including the corrected color difference signal is transmitted to the outside.

  The correction estimation unit 14 receives the luminance signal and the color difference signal of the corrected YUV signal, normalizes the luminance signal of the corrected YUV signal based on the color difference signal of the corrected YUV signal, and corrects the corrected signal. Estimated flag information indicating the correction location (coordinates) of the signal element corrected and estimated for the color difference signal in the YUV signal is generated and sent to the flag comparison unit 15. This estimation flag information is generated when the normalized luminance signal is used as a corrected estimation signal of a color difference signal. The normalization process in the correction estimation unit 14 and the normalization process in the teacher signal generation unit 11 are the same.

  The flag comparison unit 15 compares the flag information input from the correction determination unit 12 with the estimation flag information input from the correction estimation unit 14, and uses flag correction information indicating the correction location (coordinates) of the signal elements that do not match. Generate and send out. Therefore, when the flag information and the estimated flag information completely match, the flag correction information is not transmitted, or is transmitted with a parity bit indicating that the flag information completely matches.

  As a result, on the receiving side that receives the signal from the encoding-side signal correction device 10, when receiving the YUV signal including the corrected color difference signal, which signal of which signal is relative to the signal element of the reference YUV signal. Whether or not the element has been corrected can be determined by flag correction information.

  FIG. 2 is a block diagram of the encoding-side signal modification device 20 according to an embodiment of the present invention. Components similar to those in FIG. 1 are described with the same reference numerals. The encoding-side signal correction device 20 is the same as that of FIG. 1 in that it includes a teacher signal generation unit 11, a correction determination unit 12, a correction processing unit 13, a correction estimation unit 14, and a flag comparison unit 15. However, there is a further difference in that a parameter comparison unit 16 is provided.

  Further, the correction estimation unit 14 in the encoding-side signal correction device 20 inputs the luminance signal and the color difference signal of the corrected YUV signal, and based on the color difference signal of the corrected YUV signal, the correction YUV signal is corrected. The luminance signal is normalized, and estimation flag information indicating the correction portion (coordinate) of the signal element that is corrected and estimated for the color difference signal in the corrected YUV signal, and the estimation that indicates the correction amount of the signal element that is corrected and estimated Correction parameters are generated and sent to the flag comparison unit 15 and the parameter comparison unit 16, respectively. The estimated correction parameter is generated as a parameter for normalization processing in the generation of the corrected estimated signal when the normalized luminance signal is used as the corrected estimated signal of the color difference signal.

  Further, the teacher signal generation unit 11 in the encoding-side signal correction device 20 receives and normalizes the luminance signal of the YUV signal to be corrected, and generates the normalized luminance signal as a teacher signal of the color difference signal. The difference is that a normalization parameter representing a correction amount generated by the difference between the signal element of the teacher signal of the color difference signal and the signal element of the color difference signal in the corresponding reference YUV signal is generated and sent to the parameter comparison unit 16. .

  The flag comparison unit 15 compares the flag information input from the correction determination unit 12 with the estimation flag information input from the correction estimation unit 14, and flag correction information (for example, coincidence) indicating a correction portion of a different signal element Flag correction information indicating the correction location (coordinates) of the signal element not to be generated and sent to the outside. Therefore, when the flag information and the estimated flag information completely match, the flag correction information is not transmitted, or is transmitted with a parity bit indicating that the flag information completely matches. As a result, the amount of information to be transmitted can be further reduced.

  The parameter comparison unit 16 compares the normalization parameter input from the teacher signal generation unit 11 with the estimated correction parameter input from the correction estimation unit 14, and flag correction information (for example, a correction point of a different signal element) , Parameter correction information indicating a correction amount that does not match the correction location (coordinates) of the signal element is generated and sent to the outside. Accordingly, when the normalization parameter and the estimated correction parameter completely match, the parameter correction information is not transmitted, or is transmitted with a parity bit indicating that they are completely matched.

  In addition, in the flag comparison unit 15 and the parameter comparison unit 16, on the receiving side when the comparison results match, as described later, the flag information and the flag correction information, and the estimated correction parameter and the normalization parameter are treated as the same. Will be able to.

  The correction amount in the normalization parameter and the estimated correction parameter can be a linear conversion parameter as the simplest example. For example, when approximating the signal element (Y) of the teacher signal of the color difference signal with the signal element (X) of the color difference signal in the corresponding reference YUV signal, when using an approximation function of Y = aX + b, the normalization parameter (estimation correction) Parameters a) and b) can be obtained.

  Therefore, the encoding-side signal correction device 20 receives the color difference signal of the corrected YUV signal by the correction processing unit 13 and includes the corrected YUV signal according to the flag information input from the correction determination unit 12. When the YUV signal including the corrected color difference signal is sent to the outside by correcting the correction portion (coordinates) of the signal element to be corrected with the signal element of the teacher signal and sending out the corrected color difference signal to the outside, it is generated based on the reference YUV signal If the flag information and the normalization parameter are compared with the estimated flag information and the estimated correction parameter that are corrected and estimated based on the corrected YUV signal, respectively, the flag correction information and the parameter correction information are Transmit to the outside.

  Thus, on the receiving side that receives the signal from the encoding-side signal correction device 20, when receiving the YUV signal including the corrected color difference signal, which signal element of which signal has been corrected is indicated by the flag correction information. It is possible to know the parameter correction information and how much correction amount is added to the corrected signal element to get close to the signal element of the reference YUV signal, as well as flag correction information and parameter When it is determined that there is no correction information, the result of correction estimation using only the received component signal can be decoded as a signal very close to the reference YUV signal.

  Therefore, a more specific example will be described below based on the most preferable configuration of the encoding-side signal correction device 20.

  FIGS. 3 and 4 are configuration examples for correcting the corrected YUV signal by replacing the orthogonal transformation coefficient as the signal element of the component signal in the encoding-side signal correction device 20. As illustrated in FIG. 3, the teacher signal generation unit 11 includes a sampling rate conversion unit 11a, a third orthogonal conversion unit 11b, and a normalization processing unit 11c. The correction determination unit 12 includes a second orthogonal transform unit 12a and a comparison unit 12b. The correction processing unit 13 includes a first orthogonal transform unit 13a, an orthogonal transform coefficient replacement unit 13b, and an inverse orthogonal transform unit 13c.

  In the case of this example, the sampling rate conversion unit 11a down-converts the luminance signal of the corrected YUV signal to the sampling rate of the corresponding color difference signal and sends it to the third orthogonal conversion unit 11b. When correcting the luminance signal of the corrected YUV signal, the sampling rate converter 11a up-converts the color difference signal of the corrected YUV signal to the sampling rate of the corresponding luminance signal and performs the third orthogonal operation. The data is sent to the conversion unit 11b.

  As described above, one luminance signal pixel block (16 × 16 pixels), which is the minimum unit of the encoding process, and a corresponding color difference signal pixel block (8 × 8 pixels in the 4: 2: 0 format). ) Is different in number of pixels. Deterioration due to quantization differs for each pixel block of 8 × 8 pixels, and the tendency of deterioration of each block is uncorrelated, so that visually noticeable deterioration may occur particularly at the boundary between pixel blocks.

  However, considering that the area of the color difference signal after encoding is twice as large as the area of the luminance signal after encoding (or horizontal and vertical), the corresponding luminance is calculated from the color difference signal. It is possible to know the correlation of signals between adjacent pixel blocks of the signal. Also, since the number of luminance signal samplings is higher than that of chrominance signals and has a higher resolution, the luminance signal including coding degradation is down-converted to the sampling rate of the chrominance signal to reduce the degradation, and this luminance is reduced. The color difference signal can be restored by using the signal as a teacher signal for the color difference signal. Therefore, in order to compare the correlation between the luminance signal and the color difference signal, it is preferable that the sampling rates of the luminance signal and the color difference signal are the same.

  When correcting the luminance signal of the corrected YUV signal, the sampling rate conversion unit 11a converts one or both of the two color difference signals in the corrected YUV signal. Predefined.

  In the case of this example, the third orthogonal transform unit 11b performs orthogonal transform similar to the orthogonal transform process used in the first orthogonal transform unit 13a on the down-converted luminance signal input from the sampling rate transform unit 11a. The obtained orthogonal conversion coefficient of the down-converted luminance signal is sent to the normalization processing unit 11c.

  The second orthogonal transform unit 12a performs orthogonal transform similar to the orthogonal transform process used in the first orthogonal transform unit 13a on the color difference signal of the input reference YUV signal, and orthogonal transform of the obtained color difference signal The coefficient is sent to the normalization processing unit 11c and the comparison unit 12b.

  The normalization processing unit 11c receives the orthogonal transformation coefficient of the down-converted luminance signal from the third orthogonal transformation unit 11b, and based on each value of the orthogonal transformation coefficient of the color difference signal of the reference YUV signal input separately from this. The orthogonal transformation coefficient of the luminance signal is normalized by performing amplitude adjustment (for example, linear transformation), the normalized orthogonal transformation coefficient of the luminance signal is generated as a teacher signal of the color difference signal, and the comparison unit 12b and the orthogonal transformation coefficient replacement unit 13b To send. Further, when the normalization processing unit 11c generates the orthogonal transform coefficient of the normalized luminance signal as the teacher signal of the color difference signal, the signal element of the teacher signal of the color difference signal and the signal of the color difference signal in the corresponding reference YUV signal A normalization parameter representing a correction amount caused by the difference between the elements is generated and sent to the parameter comparison unit 16.

  In the case of this example, the first orthogonal transform unit 13a performs a predetermined orthogonal transform process (for example, DCT) on the color difference signal in the input corrected YUV signal, and performs orthogonal transform of the obtained color difference signal. The coefficients are sent to the orthogonal transform coefficient replacement unit 13b and the comparison unit 12b.

  The comparison unit 12b receives the orthogonal transformation coefficient of the color difference signal in the corrected YUV signal from the first orthogonal transformation unit 13a, and the orthogonal transformation coefficient of the teacher signal input from the normalization processing unit 11c. The signal elements (orthogonal transform coefficients in this example) are compared, flag information representing the correction portion (coordinates) of the signal element to be corrected for the color difference signal in the corrected YUV signal is generated, and the flag comparison unit 15 At the same time, it is sent to the orthogonal transform coefficient replacement unit 13b. In this comparison processing, since the signal element of the reference YUV signal input to the comparison unit 12b is the original correct signal, in this case, based on the signal element of the color difference signal in the reference YUV signal, the corrected YUV signal Among these, the correction part (coordinates) of the signal element to be corrected by the signal element of the teacher signal is specified.

  The orthogonal transform coefficient replacing unit 13b receives the orthogonal transform coefficient of the color difference signal in the corrected YUV signal from the first orthogonal transform unit 13a, and the corrected YUV signal of the corrected YUV signal according to the flag information input from the comparison unit 12b. The correction part (coordinates) of the orthogonal transformation coefficient to be corrected for the color difference signal is corrected (for example, replaced) with the orthogonal transformation coefficient of the teacher signal, and the corrected orthogonal conversion coefficient of the color difference signal is sent to the inverse orthogonal transformation unit 13c. To do.

  The inverse orthogonal transform unit 13c performs inverse orthogonal transform (for example, IDCT) on the orthogonal transform coefficient of the corrected color difference signal input from the orthogonal transform coefficient replacement unit 13b, and sends it to the outside. The above describes the case of correcting the color difference signal from the luminance signal, but the same applies to the case of correcting the luminance signal from the color difference signal.

  As shown in FIG. 4, the correction estimation unit 14 includes a sampling rate conversion unit 11a-1, a third orthogonal conversion unit 11b-1, a normalization processing unit 11c-1, a comparison unit 12b-1, and a first It consists of the orthogonal transformation part 13a-1. The processing operations of the sampling rate conversion unit 11a-1, the third orthogonal transformation unit 11b-1, the normalization processing unit 11c-1, the comparison unit 12b-1, and the first orthogonal transformation unit 13a-1 are shown in FIG. The sampling rate converter 11a, the third orthogonal transformer 11b, the normalization processor 11c, the comparator 12b, and the first orthogonal transformer 13a are the same. However, the normalization processing unit 11c-1 uses the orthogonal transformation coefficient of the down-converted luminance signal inputted from the third orthogonal transformation unit 11b-1 as the color difference signal of the corrected YUV signal inputted separately from this. The amplitude adjustment (for example, linear conversion) is performed based on each value of the orthogonal transform coefficient to normalize the orthogonal transform coefficient of the luminance signal, and the normalized orthogonal transform coefficient of the luminance signal is generated as a teacher signal of the chrominance signal. 12b-1, and when generating the orthogonal transformation coefficient of the normalized luminance signal as the chrominance signal teacher signal, the signal element of the chrominance signal teacher signal and the signal of the chrominance signal in the corresponding corrected YUV signal An estimated correction parameter representing a correction amount caused by the difference between the elements is generated and sent to the parameter comparison unit 16.

  5 and 6 are diagrams schematically showing a series of processing operations in the encoding-side signal correction device 20. FIG. 5A shows a state in which the U and V signals of the color difference signal in the 4: 2: 0 format are corrected using the down-converted luminance signal Y, and FIG. It shows how the luminance signal Y in the 2: 0 format is corrected using the U signal or V signal of the up-converted color difference signal. FIG. 6A shows a state in which the U and V signals of the color difference signal in the 4: 2: 2 format are corrected using the down-converted luminance signal Y, and FIG. It shows how the luminance signal Y in the 2: 2 format is corrected using the U signal or V signal of the up-converted color difference signal.

  As a result, according to the encoding-side signal correction device 20 of the present embodiment, even an image signal (corrected YUV signal) that has been decoded through an irreversible encoding method is decoded based on the reference YUV signal. Therefore, it can be corrected as an image signal with little image deterioration (YUV signal after correction), and image deterioration caused by encoding can be further reduced. Furthermore, according to the encoding-side signal correction device 20 of the present embodiment, when the flag comparison unit 15 and the parameter comparison unit 16 do not match the comparison results, the flag information generated by the correction determination unit 12 is used as flag correction information. Since the normalization parameter transmitted to the outside and the normalization parameter generated by the teacher signal generation unit 11 can be transmitted to the outside as parameter correction information, the transmission information is simply compared with the case of transmitting the flag information and the normalization parameter. It can be further reduced.

  Note that MPEG-2 and MPEG-4 AVC / H. Although DCT is used as orthogonal transform in many encoding systems such as H.264, according to the signal correction apparatus of this embodiment, the difference in the sampling frequency of the luminance signal of the image signal and the color difference signal that are generally used is used. Can be applied to any coding scheme. In particular, when an image deteriorated by an irreversible encoding method is input and the purpose is to recover the deterioration due to the encoding method, the same orthogonal transformation as the encoding method used for input is used. Thus, the effect of reducing deterioration will be exhibited.

  Hereinafter, a case where the encoding side signal correction device 20 of the present embodiment is applied to a moving image encoding device will be described.

[Encoding device]
In general, a moving image is encoded by motion compensation prediction, orthogonal transform, quantization, and variable length encoding. In the case of an encoding method using motion compensated prediction, a decoding device is included in an encoding device in order to use a decoded image for prediction, and the signal correction device of this embodiment is applied to the output of the decoding device. can do.

  FIG. 7 shows an example in which the encoding side signal correction device 20 of this embodiment is applied to a moving image encoding device 60. In the following description, the same components will be described with the same reference numerals.

  The encoding device 60 includes an encoding-side signal correction device 20 according to the present embodiment, a rearrangement unit 124, a subtraction unit 119, an orthogonal transformation unit 114, a quantization unit 115, a variable length encoding unit 116, An inverse quantization unit 117, an inverse orthogonal transform unit 118, a frame memory 121, a motion compensation prediction unit 122, and an addition unit 123 are provided. The encoding device 60 according to the present embodiment is different from the known encoding device (for example, the encoding device for MPEG-2, H.264) in that the encoding-side signal correction device 20 according to the present embodiment is the same. It has been added. Note that the encoding-side signal correction device 20 of the present embodiment can be applied to encoding of so-called intra prediction and motion compensated prediction, but an example in which motion compensated prediction is performed will be typically described.

  The rearrangement unit 124 temporarily stores the input video signal, rearranges the order of the frame images, and sends the frame image necessary for the encoding process to the subtraction unit 119 and the motion compensation prediction unit 122. In this example, the rearrangement unit 124 sends the input image signal to the encoding side signal correction device 20 as a reference YUV signal.

  The motion compensation prediction unit 122 performs motion vector detection on the input image signal supplied from the rearrangement unit 124 using a reference image acquired from the frame memory 121 via the encoding side signal correction device 20, and is obtained. The motion compensation is performed using the obtained motion vector, and the prediction image obtained as a result is output to the subtraction unit 119 and the addition unit 123. The motion vector information is sent to the variable length encoding unit 116.

  The subtraction unit 119 generates a difference signal between the input image signal from the rearrangement unit 124 and the predicted image from the motion compensation prediction unit 122 and sends it to the orthogonal transformation unit 114.

  The orthogonal transform unit 114 performs orthogonal transform (for example, DCT) on the difference signal supplied from the subtraction unit 119 for each pixel block in the small area, and sends the result to the quantization unit 115.

  The quantization unit 115 selects a quantization table corresponding to the pixel block in the small area supplied from the orthogonal transform unit 114, performs quantization processing, and sends the quantization table to the variable length coding unit 116 and the inverse quantization unit 117. .

  The variable length encoding unit 116 performs variable length encoding processing on the quantized signal supplied from the quantization unit 115 to generate a bitstream, and information on the motion vector supplied from the motion compensation prediction unit 122 is also variable length. Encode and output. The variable length encoding unit 116 can encode and transmit the flag correction information and parameter correction information obtained from the encoding side signal modification device 20.

  The inverse quantization unit 117 performs inverse quantization processing on the quantized signal supplied from the quantization unit 115 and outputs the result to the inverse orthogonal transform unit 118.

  The inverse orthogonal transform unit 118 performs inverse orthogonal transform (for example, IDCT) on the orthogonal transform coefficient supplied from the inverse quantization unit 117 and outputs the result to the addition unit 123.

  The adding unit 123 adds the signal obtained by the inverse orthogonal transform obtained from the inverse orthogonal transform unit 118 and the prediction image obtained from the motion compensated prediction unit 122 to generate a decoded image, and stores the decoded image in the frame memory 121.

  Here, the encoding-side signal correction device 20 in FIG. 7 can acquire the corrected YUV signal from the frame memory 121 and can acquire the reference YUV signal (corresponding to the input image signal) from the rearrangement unit 124. 7 performs the above-described correction process on the YUV signal to be corrected based on the reference YUV signal, and sends flag correction information (and parameter correction information) to the variable length encoding unit 116. To do. The variable length coding unit 116 may perform coding on the flag correction information and the parameter correction information, or may transmit without performing the coding.

  In this way, by applying the encoding-side signal correction devices 10 and 20 in the encoding device 60, the corrected YUV signal is compared with the teacher signal based on the reference YUV signal, and among the corrected YUV signals, By correcting the signal element of the component signal with the signal element of the teacher signal generated from the component signal different from the component signal, it is possible to obtain a corrected YUV signal that is more approximate to the original signal.

  Next, the decoding side signal correction devices 70 and 80 for correcting the corrected YUV signal using the flag correction information (and parameter correction information) will be described in order.

  In the following description, based on the flag correction information (and parameter correction information), the signal elements of the color difference signal (U signal / V signal) in the YUV signal to be corrected are typically represented in the YUV signal to be corrected. An example of obtaining a corrected YUV signal (corrected color difference signal) by correcting with a signal element of a teacher signal generated from a luminance signal (Y signal) will be described. An example in which the YUV signal is composed of a luminance signal and a color difference signal having different sampling frequencies as shown in FIG. 17 will be described.

  From the relationship between the component signals shown in FIG. 17, flag information (and normalization parameters) is generated based on the input flag correction information (and parameter correction information), and the luminance of the corrected YUV signal By correcting the signal (Y signal) with the color difference signal (U signal / V signal) of the YUV signal to be corrected, a corrected YUV signal (corrected luminance signal) can be obtained, and flag correction information ( When the parameter correction information) is not input, both the luminance signal and the color difference signal can be corrected from the correlation between the luminance signal and the color difference signal.

  First, the decoding-side signal correction device 70 shown in FIG. 8 is a device corresponding to the decoding-side signal correction device 10 shown in FIG. 1, and includes a correction processing unit 72, a correction estimation unit 73, and a flag correction determination unit 74. Prepare.

  The correction processing unit 72 has the same operation as that of the correction processing unit 13 on the encoding side, inputs a corrected YUV signal, and based on the flag information input from the flag correction determination unit 74, the correction processing unit 72 The signal element of the component signal to be corrected is corrected (for example, replaced) to the signal element of the correction estimation signal obtained from the correction processing unit 72 (functioning as a teacher signal), and a corrected YUV signal is generated.

  The correction estimation unit 73 operates in the same manner as the correction estimation unit 14 on the encoding side, and normalizes a signal component of a component signal different from the component signal of the YUV signal to be corrected (functions as a teacher signal). The estimation signal is sent to the correction processing unit 72.

  The flag correction determination unit 74 receives the flag correction information from the encoding-side signal correction device 10 shown in FIG. 1, corrects the correction portion indicated in the estimation flag information obtained from the correction processing unit 72 according to the flag correction information, The flag information is restored and sent to the correction processing unit 72.

  When there is no flag correction information (indicating that there is no correction part), the flag correction determination unit 74 sends the estimated flag information to the correction processing unit 72 as flag information. In this case, the decoding side signal correction device 70 shown in FIG. 8 uses the flag information as the coordinate information of the signal element to be corrected with respect to the component signal of the reference YUV signal, despite the prediction between component signals. Since it is used and corrected, the accuracy of the corrected portion of the signal element is maintained.

  However, the correction estimation signal generated by the correction estimation unit 73 shown in FIG. 8 is a signal component of a component signal different from the component signal of the corrected YUV signal, and the component of the corrected YUV signal is the same as in FIG. Since the signal is generated by normalization using the signal, an error may occur with respect to the component signal of the reference YUV signal from the viewpoint of the correction amount even if the correction portion is accurate.

  9 is a device corresponding to the encoding-side signal correction device 20 shown in FIG. 2, and is based on the luminance signal of the corrected YUV signal based on the input normalization parameter. Based on the parameter correction information input to the teacher signal generation unit 71 that generates the teacher signal, the estimated correction parameter generated by the correction estimation unit 73 is corrected to restore the normalization parameter, and is sent to the teacher signal generation unit 71 And a parameter correction determination unit 75. The correction estimation unit 73 of this example sends an estimated correction parameter representing a normalization process parameter in the generation of the correction estimation signal to the parameter correction determination unit 75 for the correction estimation signal generated in the same manner as the example shown in FIG. . When there is no parameter correction information (indicating that there is no need for correction), the parameter correction determining unit 75 sends the estimated correction parameter corresponding to the corrected estimated signal to the teacher signal generating unit 71 as a normalization parameter.

  That is, the correction processing unit 72 of the decoding-side signal correction device 80 shown in FIG. 9 is the same operation as the correction processing unit 72 shown in FIG. 8, and receives the corrected YUV signal and based on the input flag information. Then, the signal element of a certain component signal of the YUV signal to be corrected is corrected with the signal element of the teacher signal obtained from the teacher signal generation unit 71 to generate a corrected YUV signal. On the other hand, the teacher signal generation unit 71 of the decoding-side signal correction device 80 shown in FIG. 9 can correct the signal element of the component signal different from the component signal of the YUV signal to be corrected based on the reference YUV signal. Therefore, the teacher signal is generated by normalization using the normalization parameter. As a result, the decoding-side signal correction device 80 shown in FIG. 9 maintains accuracy with respect to the correction location and the correction amount of the signal element.

  Therefore, a more specific example will be described below based on the most preferable configuration of the decoding-side signal correction device 80.

  FIG. 10 shows an example of correcting the corrected YUV signal by replacing the orthogonal transformation coefficient as the signal element of the component signal in the decoding-side signal correction device 80. As will be described in comparison with FIG. 9, the teacher signal generation unit 71 includes a sampling rate conversion unit 71a, a third orthogonal conversion unit 71b, and a normalization processing unit 71c. The correction processing unit 72 includes a first orthogonal transform unit 72a, an orthogonal transform coefficient replacement unit 72b, and an inverse orthogonal transform unit 72c. The correction estimation unit 73 includes a sampling rate conversion unit 73a, a second orthogonal conversion unit 73b, a normalization processing unit 73c, and a comparison unit 73d.

  In the case of this example, the sampling rate conversion unit 71a down-converts the luminance signal of the corrected YUV signal to the sampling rate of the corresponding color difference signal and sends it to the third orthogonal conversion unit 71b. When correcting the luminance signal of the corrected YUV signal, the sampling rate conversion unit 71a up-converts the color difference signal of the corrected YUV signal to the sampling rate of the corresponding luminance signal and performs the third orthogonal operation. The data is sent to the conversion unit 71b.

  When correcting the luminance signal in the corrected YUV signal, the sampling rate conversion unit 71a converts one or both of the two color difference signals in the corrected YUV signal. It is preferable to preliminarily prescribe and prescribe it to be the same on the encoding side and the decoding side.

  In the case of this example, the third orthogonal transform unit 71b performs orthogonal transform similar to the orthogonal transform process used in the first orthogonal transform unit 72a on the down-converted luminance signal input from the sampling rate conversion unit 71a. The obtained orthogonal transform coefficient of the down-converted luminance signal is sent to the normalization processing unit 71c. It is preferable that the encoding side and the decoding side are defined to have the same orthogonal transform type.

  The normalization processing unit 71c receives the orthogonal transformation coefficient of the down-converted luminance signal from the third orthogonal transformation unit 71b, and the orthogonality of the color difference signal of the reference YUV signal according to the normalization parameter input from the parameter correction determination unit 75. Amplitude adjustment (for example, linear conversion) is performed on the basis of each value of the transform coefficient to normalize the orthogonal transform coefficient of the luminance signal, and the orthogonal transform coefficient of the normalized luminance signal is generated as a teacher signal of the color difference signal. The data is sent to the replacement unit 72b.

  In the case of this example, the first orthogonal transform unit 72a performs a predetermined orthogonal transform process (for example, DCT) on the color difference signal in the input corrected YUV signal, and performs orthogonal transform of the obtained color difference signal. The coefficients are sent to the orthogonal transform coefficient replacement unit 72b, the normalization processing unit 73c, and the comparison unit 73d.

  In the case of this example, the sampling rate conversion unit 73a down-converts the luminance signal of the corrected YUV signal to the sampling rate of the corresponding color difference signal and sends it to the second orthogonal conversion unit 73b. When correcting the luminance signal of the corrected YUV signal, the sampling rate conversion unit 73a up-converts the color difference signal of the corrected YUV signal to the sampling rate of the corresponding luminance signal and performs second orthogonality. The data is sent to the conversion unit 73b.

  When correcting the luminance signal of the corrected YUV signal, the sampling rate conversion unit 73a converts one or both of the two color difference signals in the corrected YUV signal. It is preferable to preliminarily prescribe and prescribe it to be the same on the encoding side and the decoding side.

  In the case of this example, the second orthogonal transform unit 73b performs orthogonal transform similar to the orthogonal transform process used in the first orthogonal transform unit 72a on the down-converted luminance signal input from the sampling rate conversion unit 73a. The obtained orthogonal transform coefficient of the down-converted luminance signal is sent to the normalization processing unit 73c. It is preferable that the encoding side and the decoding side are defined to have the same orthogonal transform type.

  The normalization processing unit 73c receives the orthogonal transformation coefficient of the down-converted luminance signal from the second orthogonal transformation unit 73b, and each value of the orthogonal transformation coefficient of the color difference signal of the corrected YUV signal from the first orthogonal transformation unit 72a. The orthogonality conversion coefficient of the luminance signal is normalized by performing amplitude adjustment (for example, linear conversion) based on this, and the normalized orthogonal transformation coefficient of the luminance signal is generated as the orthogonal transformation coefficient of the estimated correction signal of the color difference signal, and the comparison unit 73d To send.

  In addition, the normalization processing unit 73 c sends an estimated correction parameter corresponding to the estimated correction signal to the parameter correction determination unit 75.

  The comparison unit 73d receives the orthogonal transformation coefficient of the color difference signal in the corrected YUV signal from the first orthogonal transformation unit 72a, and the orthogonal transformation coefficient of the estimated correction signal input from the normalization processing unit 73c. The signal elements (orthogonal transformation coefficients in this example) are compared, and the estimated flag information indicating the correction location (coordinates) of the signal element to be corrected for the color difference signal in the corrected YUV signal is generated to determine the flag correction. The data is sent to the unit 74.

  The orthogonal transform coefficient replacing unit 72b receives the orthogonal transform coefficient of the color difference signal in the corrected YUV signal from the first orthogonal transform unit 72a, and the corrected YUV according to the flag information input from the flag correction determination unit 74. The correction part (coordinates) of the orthogonal transformation coefficient to be corrected for the color difference signal in the signal is corrected (for example, replaced) with the orthogonal transformation coefficient of the teacher signal obtained from the normalization processing unit 71c, and the corrected color difference signal The orthogonal transform coefficient is sent to the inverse orthogonal transform unit 72c.

  The inverse orthogonal transform unit 72c performs inverse orthogonal transform (for example, IDCT) on the orthogonal transform coefficient of the corrected color difference signal input from the orthogonal transform coefficient replacement unit 72b, and transmits the result to the outside. The above describes the case of correcting the color difference signal from the luminance signal, but the same applies to the case of correcting the luminance signal from the color difference signal.

  Hereinafter, the case where the decoding side signal correction device 80 of the present embodiment is applied to a moving image decoding device will be described.

[Decoding device]
An example in which the decoding-side signal correction device 80 of the present embodiment is applied to a moving image decoding device 90 is shown in FIG. In the following description, the same components will be described with the same reference numerals.

  The decoding device 90 according to the present embodiment includes a variable length decoding unit 132, an inverse quantization unit 133, an inverse orthogonal transform unit 134, an addition unit 135, a decoding side signal modification device 80, a frame memory 136, a motion A compensation prediction unit 137 and a rearrangement unit 138 are provided. The decoding device 90 according to this embodiment is obtained by adding the decoding-side signal correction device 80 according to this embodiment to a known decoding device (for example, a decoding device for MPEG-2, H.264). It is.

  The variable-length decoding unit 132 receives the encoded bitstream, performs variable-length decoding processing, sends it to the inverse quantization unit 133, decodes motion vector information, and sends it to the motion compensation prediction unit 137. . In addition, the variable length decoding unit 132 sends the flag correction information and the parameter correction information to the decoding side signal correction device 80.

  The inverse quantization unit 133 obtains an orthogonal transform coefficient of the differential signal that has been subjected to inverse quantization processing on the quantized signal supplied from the variable length decoding unit 132 and motion-compensated, and sends it to the inverse orthogonal transform unit 134. .

  The inverse orthogonal transform unit 134 performs inverse orthogonal transform (for example, IDCT) on the orthogonal transform coefficient of the difference signal supplied from the inverse quantization unit 133, and sends the obtained difference signal to the addition unit 135.

  The motion compensation prediction unit 137 generates a prediction image using the reference image obtained from the frame memory 136 and the motion vector obtained from the variable length decoding unit 132, and outputs the prediction image to the addition unit 135.

  The adding unit 135 adds the difference signal obtained from the inverse orthogonal transform unit 134 and the prediction image supplied from the motion compensation prediction unit 137 to restore the image signal, and the restored image signal is related to the present embodiment. The data is sent to the decoding side signal correction device 80 in the decoding device 90.

  The decoding-side signal correction device 80 in the decoding device 90 uses the flag correction information and the parameter correction information to generate a corrected image signal in which the image signal obtained from the adding unit 135 is suppressed as described above. Are stored in the frame memory 136 and sent to the rearrangement unit 138.

  The rearrangement unit 138 rearranges the corrected image signals obtained from the decoding-side signal correction device 80 in the order of display frames and outputs them.

  As described above, the decoding-side signal correction device 80 of this embodiment can be applied to the decoding device 90.

  In the encoding device 60 and the decoding device 90, the signal correction device of the present invention has been incorporated at a specific arrangement position. You can also.

  Hereinafter, operations of the encoding-side signal correction device 20 and the decoding-side signal correction device 80 in the encoding device 60 and the decoding device 90 according to the present embodiment will be described with reference to FIGS. 12 and 13, respectively.

  Referring to FIG. 12, the encoding-side signal correction device 20 receives the reference YUV signal and the corrected YUV signal (step S1).

  The encoding-side signal correction device 20 performs down-conversion of the luminance signal in the corrected YUV signal by the sampling rate converter 11a (step S2), and up-converts the color difference signal in the corrected YUV signal. Execute (Step S3).

  The encoding-side signal correction device 20 performs orthogonal transformation of the luminance signal and the color difference signal in the reference YUV signal by the second orthogonal transformation unit 12a (step S4).

  The encoding-side signal correction device 20 performs orthogonal transformation on each of the luminance signal after down-conversion and the color difference signal after up-conversion by the third orthogonal transformation unit 11b (steps S5 and S6), and normalization processing The unit 11c normalizes each of the orthogonal transform coefficients of the down-converted luminance signal and the up-converted color difference signal based on the color difference signal and the luminance signal orthogonal transform coefficient of the reference YUV signal, respectively. (Steps S7 and S8).

  The encoding-side signal correction device 20 performs a normalization process on each orthogonal transform coefficient of the luminance signal after down-conversion and the color difference signal after up-conversion, so that the color difference signal of the YUV signal to be corrected respectively. In addition, a teacher signal of the luminance signal is generated, and a normalization parameter indicating the correction amount of each signal element (orthogonal transform coefficient) is generated (steps S9, S10, S11).

  The encoding-side signal correction device 20 uses the comparison unit 12b to calculate the orthogonal transform coefficients of the color difference signal and the luminance signal in the corrected YUV signal and the orthogonal transform coefficients of the corresponding teacher signals based on the reference YUV signal. Comparison is made to determine a correction location of the corrected YUV signal (step S12), and information on the correction location is generated as flag information (step S13).

Here, a specific example of flag information will be described. FIG. 14A is a diagram illustrating an example in which the signal elements of the component signal are modified in units of signal elements. FIG. 14B is a diagram illustrating an example in which the signal elements of the component signal are modified in units of divided blocks. It is. For example, as shown in FIG. 14A, each of the signal elements of the reference YUV signal, the corrected YUV signal, and the teacher signal is composed of 8 × 8 blocks (for example, pixel values or orthogonal transform coefficients). It shall be. For example, whether or not a certain signal element A1 of the corrected YUV signal should be corrected by the signal element A2 at the corresponding coordinate position of the teacher signal is determined by comparing the difference based on the signal element A0 of the reference YUV signal. Judgment can be made. For example, when (A1-A0) ² > (A2-A0) ² , it is determined that a certain signal element A1 of the YUV signal to be corrected is corrected by the signal element A2 at the corresponding coordinate position of the teacher signal. Similarly for other signal elements, if (B1-B0) ² > (B2-B0) ^2, a signal element B1 of the corrected YUV signal is corrected by the signal element B2 at the corresponding coordinate position of the teacher signal. to decide.

  On the other hand, as shown in FIG. 14B, each of the signal elements of the reference YUV signal, the corrected YUV signal, and the teacher signal is composed of 8 × 8 blocks (for example, pixel values or orthogonal transform coefficients). In this case, for example, the block is divided into four blocks, and a value representing the sum of evaluation values for each signal element in the block can be set as a comparison and replacement target. For example, whether a certain signal element (block A1) of the corrected YUV signal should be corrected by the signal element (block A2) at the corresponding coordinate position of the teacher signal indicates whether or not the signal element (block A0) of the reference YUV signal is to be corrected. It can be judged by comparing the magnitude of the difference based on the above. For example, if the sum of squares of the difference between the signal element of block A0 and the signal element of block A1 is greater than the sum of squares of the difference between the signal element of block A0 and the signal element of block A2, there is a certain corrected YUV signal. It is determined that the signal element (block A1) is corrected by the signal element (block A2) at the corresponding coordinate position of the teacher signal. Similarly, it is determined that other signal element blocks are corrected in units of blocks. By setting the signal element for comparing and replacing the corrected YUV signal as a block unit, the signal amount of flag information can be reduced, and the coding rate can be improved when applied to an encoding apparatus. . The dividing method is not limited to the above-described example, and can be modified in various ways according to individual designs.

  The encoding-side signal correction device 20 uses the orthogonal transformation coefficient replacement unit 13b to use the orthogonal transformation coefficient signal element of each teacher signal and the orthogonal transformation coefficients of the color difference signal and the luminance signal in the YUV signal to be modified. The correction of the correction portion corresponding to the flag information is applied, and the inverse orthogonal transform unit 13c performs inverse orthogonal transform on the orthogonal transform coefficient of the corrected YUV signal (step S14).

  In the encoding-side signal correction device 20, the correction estimation unit 14 performs normalization processing between the orthogonal transform coefficients of the luminance signal and the color difference signal in the corrected YUV signal, and generates estimation flag information and an estimation correction parameter ( Step S15).

  When there is a difference between the flag information and the estimation flag information, and between the normalization parameter and the estimation correction parameter, respectively, the encoding side signal correction device 20 is determined by the flag comparison unit 15 and the parameter comparison unit 16, respectively. Flag correction information and parameter correction information representing these differences (difference values) are sent to the outside (step S16).

  Thus, the encoding-side signal correction device 20 generates the corrected YUV signal, flag information, and normalization parameters.

  Referring to FIG. 13, the decoding-side signal correction device 80 inputs a corrected YUV signal and flag correction information and parameter correction information associated with the corrected YUV signal (step S21).

  In the decoding-side signal correction device 80, the correction estimation unit 73 performs normalization processing between the orthogonal transform coefficients of the luminance signal and the color difference signal in the corrected YUV signal, and generates estimation flag information and an estimation correction parameter (step). S22-1).

  The encoding-side signal correction device 20 uses the flag correction determination unit 74 and the parameter correction determination unit 75 to determine, from the flag correction information and the parameter correction information, between the flag information and the estimated flag information, and the normalization parameter and the estimated correction parameter, respectively. When it is determined that there is a difference between the two, the flag information and the normalization parameter are reproduced from the estimated flag information and the estimated correction parameter (step S22-2). When there is no difference (when flag correction information and parameter correction information are not transmitted), the flag information and the estimated flag information are equal, and the normalization parameter and the estimated correction parameter are equal.

  The decoding-side signal correction device 80 performs down-conversion of the luminance signal of the corrected YUV signal by the sampling rate conversion unit 71a (step S23) and up-conversion of the color difference signal of the corrected YUV signal. (Step S24).

  On the other hand, the decoding-side signal correction device 80 performs orthogonal transformation of the luminance signal and the color difference signal in the YUV signal to be modified by the first orthogonal transformation unit 72a (step S25).

  In the decoding-side signal correction device 80, the third orthogonal transform unit 71b performs orthogonal transform on each of the down-converted luminance signal and the up-converted color difference signal (steps S26 and S27), and a normalization processing unit By 71c, each of the orthogonal transform coefficients of the luminance signal after down-conversion and the color difference signal after up-conversion is subjected to normalization processing based on the normalization parameters (steps S28 and S29).

  The decoding-side signal correction device 80 performs normalization processing on each of the orthogonal transform coefficients of the luminance signal after down-conversion and the color difference signal after up-conversion, so that the color difference signal of the YUV signal to be corrected and A teacher signal for the luminance signal is generated (step S30).

  The decoding-side signal correction device 80 uses the orthogonal transform coefficient replacement unit 72b to flag the orthogonal transform coefficients of the color difference signal and the luminance signal in the corrected YUV signal using the signal element of the orthogonal transform coefficient of each teacher signal. The correction of the correction portion corresponding to the information is applied, and the inverse orthogonal transform is performed on the orthogonal transform coefficient of the corrected YUV signal by the inverse orthogonal transform unit 72c (step S31).

  Thus, the decoding-side signal correction device 80 can obtain a corrected YUV signal using the flag information and the normalization parameter. For example, as shown in FIG. 15, the luminance signal and the color difference signal in the corrected YUV signal can be corrected for each signal element in consideration of the mutual correlation and the difference with respect to the reference signal to obtain a corrected YUV signal. The corrected YUV signal is different from the one corrected based only on the correlation between the component signals, and is corrected based on the reference YUV signal. Therefore, the YUV signal is restored as an image signal with little deterioration. Keep in mind that there are.

  Furthermore, as one aspect of the present invention, the encoding-side signal modification devices 10 and 20 or the decoding-side signal modification devices 70 and 80 of each embodiment can be configured as a computer. A program for causing a computer to realize each component in each signal correction device is stored in a storage unit provided inside or outside the computer. Such a storage unit can be realized by an external storage device such as an external hard disk or an internal storage device such as ROM or RAM. The control part with which a computer is provided is realizable by control of a central processing unit (CPU). In other words, the CPU can appropriately read from the storage unit a program in which the processing content for realizing the function of each component is described, and realize the function of each component on the computer. Here, the function of each component may be realized by a part of hardware.

  In addition, the program describing the processing contents can be distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM, and such a program can be distributed on a server on a network, for example. Can be distributed by transferring the program from the server to another computer via the network.

  In addition, a computer that executes such a program can temporarily store, for example, a program recorded on a portable recording medium or a program transferred from a server in its own storage unit. As another embodiment of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and each time the program is transferred from the server to the computer. In addition, the processing according to the received program may be executed sequentially.

  While the embodiments of the present invention have been described in detail with specific examples, it will be apparent to those skilled in the art that various modifications and changes can be made without departing from the scope of the claims of the present invention.

  For example, if the sampling rate of the signal input to the signal correction device is the same, the luminance signal down-conversion unit and the color difference signal up-conversion unit can be bypassed.

  Further, the component signals to be corrected by the signal correction device are not limited to inputting YUV signals, respectively. For example, as in the encoding-side signal correction device 20b shown in FIG. It can also be configured as input. In this case, the first to third orthogonal transform units 13a, 12a, and 11b in the encoding-side signal modification device 20 described above are not necessary, and the sampling rate conversion unit 11d inputs the orthogonal transform coefficient and executes the sampling rate conversion. It will be. Note that a similar modification can be configured for the decoding-side signal correction device.

  Therefore, not only is it configured to compare the component signals in the image signal and / or the signals of these orthogonal transform coefficients, but also to compare the signals that have been input with the component signals and converted in sampling rate. , Compare the orthogonal transform coefficients of the signals whose component signals are input and converted the sampling rate, or compare after quantizing the signals of the orthogonal transform coefficients of each component signal that are input and converted the sampling rate Various modifications are possible such as performing the above-mentioned or using a combination of these.

  According to the present invention, for example, even if an image signal is composed of signal components in color spaces having different sampling frequencies, the image signal can be corrected so as to reduce the deterioration of the image signal. This is useful for any application that uses an image signal that handles encoding processing by a method.

DESCRIPTION OF SYMBOLS 10 Coding side signal correction apparatus 11 Teacher signal production | generation part 11a Sampling rate conversion part 11a-1 Sampling rate conversion part 11b 3rd orthogonal transformation part 11b-1 3rd orthogonal transformation part 11c Normalization process part 11c-1 Normalization process part 11d Sampling rate conversion unit 12 Correction determination unit 12a Second orthogonal transformation unit 12b Comparison unit 12b-1 Comparison unit 13 Correction processing unit 13a First orthogonal transformation unit 13a-1 First orthogonal transformation unit 13b Orthogonal transformation coefficient substitution unit 13c Inverse orthogonality Conversion unit 14 Modification estimation unit 15 Flag comparison unit 16 Parameter comparison unit 20 Encoding side signal modification device 20a Encoding side signal modification device 60 Encoding device 70 Decoding side signal modification device 71 Teacher signal generation unit 71a Sampling rate conversion unit 71b First 3-orthogonal transformation unit 71c normalization processing unit 72 correction processing unit 72a first 1 orthogonal transform unit 72b orthogonal transform coefficient replacement unit 72c inverse orthogonal transform unit 73 correction estimation unit 73a sampling rate conversion unit 73b second orthogonal transform unit 73c normalization processing unit 73d comparison unit 74 flag correction determination unit 75 parameter correction determination unit 80 decoding Side signal modification device 90 Decoding device 114 Orthogonal transformation unit 115 Quantization unit 116 Variable length coding unit 117 Inverse quantization unit 118 Inverse orthogonal transformation unit 119 Subtraction unit 121 Frame memory 122 Motion compensation prediction unit 123 Addition unit 124 Rearrangement unit 132 Variable length decoding unit 133 Inverse quantization unit 134 Inverse orthogonal transform unit 135 Adder unit 136 Frame memory 137 Motion compensation prediction unit 138 Rearrangement unit

Claims (12)

A signal correction device for correcting a component signal of a moving image including a signal element of a pixel value or an orthogonal transform coefficient,
The image signal of the predetermined image format before encoding through an irreversible encoding method is input as a reference image signal, and a first component signal of the decoded signal in the image signal is input, and the reference Teacher signal generation means for generating the first component signal normalized based on the image signal as a teacher signal of the second component signal of the decoded signal;
The signal component is compared between the second component signal of the decoded signal and the teacher signal input from the teacher signal generation means, and flag information indicating the correction location of the signal element to be corrected is generated. Correction determination means to
For the second component signal of the decoded signal, the signal element to be corrected is corrected with the signal element of the teacher signal in accordance with the flag information, and a corrected second component signal is generated. Correction processing means;
The signal component of the first component signal normalized based on the second component signal represents a correction portion of the signal component obtained by comparing the signal component of the second component signal of the decoded signal. Modified estimation means for generating estimation flag information;
Flag comparison means for comparing the flag information with the estimated flag information and generating flag correction information indicating a correction location of a different signal element;
A signal correction apparatus comprising:   The teacher signal generation unit and the correction estimation unit perform normalization by adjusting the amplitude of the corresponding first component signal based on the corresponding second component signal, according to claim 1. The signal correction apparatus as described.   The signal correction apparatus according to claim 1, wherein the flag information represents a correction portion in a signal element unit or a divided block unit for the signal element of each component signal. The teacher signal generation means compares signal elements between the second component signal of the decoded signal and the teacher signal input from the teacher signal generation means, and corrects a signal element to be corrected Means for generating a normalization parameter representing the quantity;
The correction estimation means is a signal obtained by comparing the signal component of the first component signal normalized based on the second component signal with the signal component of the second component signal of the decoded signal. Means for generating an estimated correction parameter representing the correction amount of the element;
The parameter comparison means which produces | generates the parameter correction information showing the correction amount of a different signal element by comparing the said normalization parameter and the said estimated correction parameter is further provided, It is characterized by the above-mentioned. A signal correction device according to claim 1.   An encoding apparatus provided with the signal correction apparatus as described in any one of Claims 1-4. A signal correction device for correcting a component signal of a moving image including a signal element of a pixel value or an orthogonal transform coefficient,
The first component signal of the decoded signal in the image signal of a predetermined image format obtained through the irreversible encoding method is input and normalized, and the second component signal of the decoded signal in the image signal is Generating estimated flag information representing a corrected portion of the signal element obtained by comparing the signal element of the first component signal normalized based on the signal element of the second component signal of the decoded signal; Correction estimation means for generating the normalized first component signal as a teacher signal of a second component signal of the decoded signal;
Input flag correction information indicating a correction portion of a different signal element obtained by comparing flag information indicating a correction portion of a signal element to be corrected and the estimation flag information, and performing the estimation based on the flag correction information Flag correction judging means for correcting flag information and reproducing the flag information;
Correction processing means for correcting the signal element to be corrected with the signal element of the teacher signal according to the flag information, and generating a corrected second component signal;
A signal correction apparatus comprising: A signal correction device for correcting a component signal of a moving image including a signal element of a pixel value or an orthogonal transform coefficient,
The first component signal of the decoded signal in the image signal of a predetermined image format obtained through the irreversible encoding method is input and normalized, and the second component signal of the decoded signal in the image signal is Generating estimated flag information representing a corrected portion of the signal element obtained by comparing the signal element of the first component signal normalized based on the signal element of the second component signal of the decoded signal; A correction amount of the signal element obtained by comparing the signal element of the first component signal normalized based on the second component signal with the signal element of the second component signal of the decoded signal. A correction estimation means for generating an estimated correction parameter representing;
Input flag correction information indicating a correction portion of a different signal element obtained by comparing flag information indicating a correction portion of a signal element to be corrected and the estimation flag information, and performing the estimation based on the flag correction information Flag correction judging means for correcting flag information and reproducing the flag information;
Input parameter correction information indicating a correction amount of a different signal element obtained by comparing the normalization parameter indicating the correction amount of the signal element to be corrected and the estimated correction parameter, and based on the parameter correction information Parameter correction determination means for correcting the estimated correction parameter and reproducing the normalization parameter;
Teacher signal generation means for generating a teacher signal of a second component signal of the decoded signal;
Correction processing means for correcting the signal element to be corrected with the signal element of the teacher signal according to the flag information, and generating a corrected second component signal;
A signal correction apparatus comprising:   A decoding device comprising the signal correction device according to claim 6. A computer configured as a signal correction device for correcting a component signal of a moving image including a signal element of a pixel value or an orthogonal transform coefficient;
The image signal of the predetermined image format before encoding through an irreversible encoding method is input as a reference image signal, and a first component signal of the decoded signal in the image signal is input, and the reference Generating the first component signal normalized based on an image signal as a teacher signal of a second component signal of the decoded signal;
The signal component is compared between the second component signal of the decoded signal and the teacher signal input from the teacher signal generation means, and flag information indicating the correction location of the signal element to be corrected is generated. And steps to
For the second component signal of the decoded signal, the signal element to be corrected is corrected with the signal element of the teacher signal in accordance with the flag information, and a corrected second component signal is generated. Steps,
The signal component of the first component signal normalized based on the second component signal represents a correction portion of the signal component obtained by comparing the signal component of the second component signal of the decoded signal. Generating estimated flag information;
Comparing the flag information with the estimated flag information, and generating flag correction information representing a corrected portion of a different signal element;
A program for running A computer configured as a signal correction device for correcting a component signal of a moving image including a signal element of a pixel value or an orthogonal transform coefficient;
The image signal of the predetermined image format before encoding through an irreversible encoding method is input as a reference image signal, and a first component signal of the decoded signal in the image signal is input, and the reference Generating the first component signal normalized based on an image signal as a teacher signal of a second component signal of the decoded signal;
The signal component is compared between the second component signal of the decoded signal and the teacher signal input from the teacher signal generation means, and flag information indicating the correction location of the signal element to be corrected is generated. And steps to
For the second component signal of the decoded signal, the signal element to be corrected is corrected with the signal element of the teacher signal in accordance with the flag information, and a corrected second component signal is generated. Steps,
The signal component of the first component signal normalized based on the second component signal represents a correction portion of the signal component obtained by comparing the signal component of the second component signal of the decoded signal. Generating estimated flag information;
Comparing the flag information with the estimated flag information, and generating flag correction information representing a corrected portion of a different signal element;
A signal parameter is compared between the second component signal of the decoded signal and the teacher signal input from the teacher signal generation means, and a normalization parameter indicating a correction amount of the signal element to be corrected is obtained. Generating step;
The signal component correction amount obtained by comparing the signal component of the first component signal normalized based on the second component signal with the signal component of the second component signal of the decoded signal is represented. Generating an estimated correction parameter;
Comparing the normalized parameter and the estimated correction parameter to generate parameter correction information representing a correction amount of a different signal element;
A program for running A computer configured as a signal correction device for correcting a component signal of a moving image including a signal element of a pixel value or an orthogonal transform coefficient;
The first component signal of the decoded signal in the image signal of a predetermined image format obtained through the irreversible encoding method is input and normalized, and the second component signal of the decoded signal in the image signal is Generating estimated flag information representing a corrected portion of the signal element obtained by comparing the signal element of the first component signal normalized based on the signal element of the second component signal of the decoded signal; Generating the normalized first component signal as a teacher signal of a second component signal of the decoded signal;
Input flag correction information indicating a correction portion of a different signal element obtained by comparing flag information indicating a correction portion of a signal element to be corrected and the estimation flag information, and performing the estimation based on the flag correction information Correcting the flag information and reproducing the flag information;
Modifying the signal element to be corrected with the signal element of the teacher signal according to the flag information, and generating a corrected second component signal;
A program for running A computer configured as a signal correction device for correcting a component signal of a moving image including a signal element of a pixel value or an orthogonal transform coefficient;
The first component signal of the decoded signal in the image signal of a predetermined image format obtained through the irreversible encoding method is input and normalized, and the second component signal of the decoded signal in the image signal is Generating estimated flag information representing a corrected portion of the signal element obtained by comparing the signal element of the first component signal normalized based on the signal element of the second component signal of the decoded signal; A correction amount of the signal element obtained by comparing the signal element of the first component signal normalized based on the second component signal with the signal element of the second component signal of the decoded signal. Generating an estimated correction parameter representing;
Input flag correction information indicating a correction portion of a different signal element obtained by comparing flag information indicating a correction portion of a signal element to be corrected and the estimation flag information, and performing the estimation based on the flag correction information Correcting the flag information and reproducing the flag information;
Input parameter correction information indicating a correction amount of a different signal element obtained by comparing the normalization parameter indicating the correction amount of the signal element to be corrected and the estimated correction parameter, and based on the parameter correction information Correcting the estimated correction parameter and regenerating the normalization parameter;
Generating a teacher signal of a second component signal of the decoded signal;
Modifying the signal element to be corrected with the signal element of the teacher signal according to the flag information, and generating a corrected second component signal;
A program for running

Images