JP4057503B2 - Resolution conversion filter coefficient determination method, image resolution conversion method, image resolution conversion apparatus, video re-encoding method, video re-encoding apparatus, resolution conversion filter coefficient determination program, image resolution conversion program, video re-encoding program, and the like Recording medium on which the program is recorded - Google Patents

Description

  The present invention relates to a video processing technique for performing enlargement / reduction processing of image and video resolution, and a video encoding technique accompanied therewith, and in particular, when the resolution of a video signal is enlarged and then reduced to the resolution before enlargement. Filter coefficient determination method for resolution conversion, image resolution conversion method, image resolution conversion device, video re-encoding method, video re-encoding device, for resolution conversion The present invention relates to a filter coefficient determination program, an image resolution conversion program, a video re-encoding program, and a recording medium on which these programs are recorded.

  In general, in order to enlarge the resolution of a video image in the horizontal or vertical one-dimensional direction, each pixel of the enlarged image is interpolated by weighted linear sum of neighboring pixel data of pixels between pixels of the image before enlargement. Obtain pixel data. Depending on the relationship between the pixel positions before and after the enlargement, the data of each pixel of the enlarged image is obtained while repeating the processing at several types of interpolation positions, and a different weighting coefficient is used for each interpolation position. Since the calculation of the weighted linear sum can be interpreted as a kind of one-dimensional filter, the weighting coefficient is hereinafter referred to as a filter coefficient.

  This filter coefficient is generally determined in consideration of the correlation between pixel positions before and after enlargement based on the FIR filter coefficient having the Nyquist frequency of the image before enlargement as a cutoff frequency. In general, by limiting the number of taps of the FIR filter at the filter design stage, the number of neighboring pixels used for the enlargement process is generally several to several tens (for example, non-patent literature). 1).

  Similarly, also in video reduction, each pixel data of the reduced image is obtained by a weighted linear sum of neighboring pixel data of the image before reduction. The filter coefficient is generally determined in consideration of the correlation between the pixel positions before and after reduction based on the FIR filter coefficient having the Nyquist frequency of the reduced image as a cutoff frequency.

Note that video re-encoding in video encoding such as MPEG-2 (for example, see Non-Patent Document 2) described later is described in Non-Patent Document 3 below.
Masaru Kanazawa, Gen Sone, “Basic Video Signal System and Its Conversion Processing Technology in the Digital TV Era”, Interface January 2000 ISO / IEC 13818-2 International Standard, Jan. 1995 P.Guilote1, et a1., "Adaptive Encoders: The New Generation of MPEG-2 Encoders", SMPTE journa1, April 2000

  However, when the resolution of the image is enlarged and reduced to the original resolution by using the filter coefficient determined in consideration of the cutoff frequency before and after the conversion, both of them are usually not reversible, and before the enlargement. The image after and the image after reduction do not match. This causes problems in various situations.

  One example is video re-encoding. For example, in video encoding such as MPEG-2 described in Non-Patent Document 2, when re-encoding a decoded image, re-encoding with almost no degradation is achieved by completely inheriting the encoding parameters of the previous stage. Is known to be possible (see, for example, Non-Patent Document 3).

  Often in video encoding, the resolution of the color difference signal of the input video is reduced by a half in the vertical direction from a format called 4: 2: 2 to a format called 4: 2: 0 in the encoder. In the decoder, the decoded video is doubled in the vertical direction from the 4: 2: 0 format to the 4: 2: 2 format and then output. When re-encoding is performed, the re-encoder again converts the 4: 2: 2 format into the 4: 2: 0 format. At this time, since the enlargement process of the normal decoder and the reduction process of the re-encoder are not reversible, a color difference signal mismatch occurs.

  As a result, even if the previous encoding parameters are completely inherited, it becomes impossible to realize re-encoding with almost no deterioration. Incidentally, since most video signal transmission interfaces are in the 4: 2: 2 format, transmission of the decoded video in the 4: 2: 0 format is not generally performed.

  In addition, in video coding, there are many known methods for reducing the resolution of the input video at the time of coding and increasing the resolution of the decoded video at the time of decoding. A similar problem occurs when using the re-encoding technique that inherits completely.

  For these reasons, enlargement / reduction processing is required so that the image before enlargement matches the image after reduction. Here, when it is not necessary to pay attention to the image quality of the enlarged image, for example, in the case of the 4: 2: 0-4: 2: 2 conversion described above, the pixel data is copied in the vertical direction at the time of enlargement and reduced. Sometimes, reversible conversion can be achieved by thinning out pixel data in the vertical direction. However, when the image quality of the enlarged image becomes a problem, such a method results in an enlarged image with extremely poor image quality.

  The present invention solves the above-mentioned problems of the prior art, and when the resolution of an image signal is enlarged using a one-dimensional filter and then reduced to a resolution before enlargement using a one-dimensional filter, the resolution enlargement process is almost the same. It is an object of the present invention to provide a video resolution conversion technique capable of performing a reduction process as inverse conversion.

  In order to solve the above problem, in the present invention, the filter coefficient for the enlargement process is determined by a normal method, the filter coefficient for the reduction process that is the inverse transform is obtained, and the reduction process is performed using the coefficient.

That is, the present invention relates to the resolution when the image signal obtained by expanding the resolution of the decoded video obtained by decoding the video encoded stream using the enlargement processing filter is re-encoded after being reduced to the resolution before the enlargement. A resolution conversion filter coefficient determination method for determining a filter coefficient of a reduction processing filter used for image reduction, wherein the filter coefficient of the enlargement processing filter used for the resolution enlargement process is at least an image edge boundary process. A process of inputting a filter coefficient of a fixed pattern applied to a region other than the affected area and storing it in the storage means, a matrix of 1 column N rows representing 1-dimensional image data of N pixels before enlargement, X, and M after enlargement (M> N) Y is a matrix with 1 column and M rows representing 1-dimensional image data of pixels, and A is an expansion conversion matrix with N columns and M rows with Y = AX representing enlargement processing, and N pixels after reduction. A matrix of 1 row N rows representing dimensional image data Z, the reduction transformation matrix M rows N row Z = a BY representing the reduction process B, and a unit matrix of N rows N row when the I, the storage BA = I from the expansion transformation matrix A determined by the filter coefficient including the row direction elements in the matrix of the region portion where the image edge boundary processing does not affect at least the filter coefficient of the fixed pattern of the expansion processing filter. a process of calculating the reduction transformation matrix B, and characterized by having a step of determining a filter coefficient of the reduction processing for the filter based on the calculated reduction transformation matrix B.

Further, in the process of calculating the reduced transformation matrix B with BA = I, the product BA of the reduced transformation matrix B and the enlarged transformation matrix A is at least in an area portion that is not affected by the image edge boundary processing in the enlargement / reduction processing. In order to obtain a filter coefficient for reduction processing that is equal to the unit matrix I of N columns and N rows, the array of element values in the row direction of the matrix in the region where the image end boundary processing of the matrices B, A, and I is not affected Are extracted from the matrices B, A, and I as matrices B ′, A ′, and I ′ , respectively, and a matrix B ′ in which B′A ′ = I ′ is obtained. The filter coefficient of the reduction processing filter is determined based on the obtained value.

  If there is a degree of freedom in determining the filter coefficient of the reduction processing filter, filter coefficient candidates of a plurality of reduction processing filters are calculated within the range of the degree of freedom, and the calculated plurality of filter coefficient candidates Among them, the one closest to the filter coefficient for the scale having the same scale ratio determined based on another method is selected.

The present invention also provides an image resolution conversion method for reducing an image signal obtained by enlarging the resolution of a decoded video obtained by decoding a video encoded stream using an enlargement processing filter to a resolution before enlargement before re-encoding . An image signal whose resolution is enlarged using the enlargement processing filter is input, and the input image signal is input using a reduction processing filter composed of filter coefficients determined using the resolution conversion filter coefficient determination method. The resolution is reduced to the resolution before enlargement.

  In addition, the present invention increases the resolution of the decoded video when decoding the video encoded stream by the decoder, and reduces the resolution of the input video to the resolution before enlargement when re-encoding the video by the re-encoder. In the video re-encoding method for encoding, the information on the resolution conversion filter coefficient used to increase the resolution of the decoded video in the decoder is transmitted to the re-encoder, and the transmitted resolution conversion filter coefficient is transmitted. From the above information, the filter coefficient of the reduction processing filter is determined using the resolution conversion filter coefficient determination method described above, and the resolution of the input video signal is determined using the reduction processing filter composed of the determined filter coefficients. The video signal is reduced to the resolution before enlargement, and the video signal with the reduced resolution is re-encoded.

  In addition, the present invention increases the resolution of the decoded video when decoding the video encoded stream by the decoder, and reduces the resolution of the input video to the resolution before enlargement when re-encoding the video by the re-encoder. In the video re-encoding method for encoding, the reduction processing determined using the resolution conversion filter coefficient determination method described above from the information of the resolution conversion filter coefficient used for enlarging the resolution of the decoded video in the decoder The filter coefficient information of the filter and the video signal are input to the re-encoder, and the resolution of the input video signal is reduced to the resolution before enlargement using a reduction processing filter based on the input filter coefficient information. The video signal with reduced resolution is re-encoded.

  The above processing can be realized by a computer and a software program, and the program can be provided by being recorded on a computer-readable recording medium or can be provided through a network.

  According to the present invention, when the resolution of an image signal is enlarged using a one-dimensional filter and then reduced to a pre-enlargement resolution using a one-dimensional filter, a reduction process that is substantially the inverse conversion of the resolution enlargement process is performed. be able to.

  Further, in the resolution conversion in the two-dimensional direction of the video realized by performing the resolution conversion in the one-dimensional direction a plurality of times, the two-dimensional resolution conversion that is almost the inverse conversion by using the resolution conversion processing according to the present invention a plurality of times. Processing can be performed.

  This also enables re-encoding with almost no deterioration by using the previous-stage encoding information in video re-encoding accompanied with video enlargement / reduction.

  Normally, the enlargement / reduction process is a linear sum operation of pixel data, and therefore can be expressed by a matrix (vector) representing one-dimensional image data and a transformation matrix having filter coefficients as components. 1-column N-row matrix X representing 1-dimensional image data of N pixels before enlargement, 1-column M-row matrix Y representing 1-dimensional image data of M (M> N) pixels after enlargement, N pixels after reduction A matrix Z having 1 column and N rows representing one-dimensional image data consisting of

X = (x ₀ , x ₁ , x ₂ ,..., X _N−2 , x _N−1 ) ^T (1)
Y = (y ₀ , y ₁ , y ₂ ,..., Y _M−2 , y _M−1 ) ^T (2)
Z = (z ₀ , z ₁ , z ₂ ,..., Z _N−2 , z _N−1 ) ^T (3)
Further, A is a conversion matrix of N columns and M rows where Y = AX representing the enlargement process, and B is a conversion matrix of M columns and N rows where Z = BY representing the reduction process.

Y = AX (4)
Z = BY (5)
Here, M = 2N, and in the enlargement process, the following filter coefficients (a ₁ , a ₂ , a ₃ , a ₄ ), (b ₁ , b ₂ , b ₃ , Suppose b ₄ ) is determined.

_{y 2n = a 1 × x n} -2 + a 2 × x n-1 + a 3 × x n + a 4 × x n + 1 (6)
_{y 2n + 1 = b 1 ×} x n-1 + b 2 × x n + b 3 × x n + 1 + b 4 × x n + 2 (7)
When N = 10 and M = 20, the expansion conversion represented by the expression (4) is expressed as the following expression (8) using the conversion matrix A.

ここで，

here,

である。変換行列Ａにおいて，＊は鏡像反転処理により係数が変化する成分を表している。

It is. In the transformation matrix A, * represents a component whose coefficient changes due to mirror image inversion processing.

In general, mirror image inversion processing is performed near the image boundary. In the mirror image inversion processing, when using the data of the neighboring pixels of the pixels near the image boundary, if the neighboring pixels are located outside the image boundary, x ₀ or x _N-1 (N is the image size) is the axis of symmetry. Assuming that mirror-inverted data of the image inside the image boundary exists outside, the calculation is performed, and x ₋₁ = x ₁ , x ₋₂ = x ₂ ,..., X _−n = _{x n, x n = x n} -2, x n + 1 = x n-3, ..., is a process to _{x n + n = x Nn-} 2.

  The component corresponding to the edge of the image in the transformation matrix A (the component indicated by * in the transformation matrix A) has a value different from that of the other regions because the coefficient for the mirror image inversion data is added. In other areas, the pattern repeats regularly. It is assumed that the following conversion is used for the reduction processing for the enlarged image.

_{z n = s 1 × y 2n} -3 + s 2 × y 2n-2 + s 3 × y 2n-1 + s 4 × y 2n + s 5 × y 2n + 1 + s 6 × y 2n + 2 + s 7 × y 2n + 3 + S ₈ × y _{2n + 4} (9)
Here, it is assumed that the same coefficient is used as the coefficient of the reduction filter processing in the case of the same interpolation position. In this case, the reduction conversion represented by the equation (5) is represented by the following equation (10) using the transformation matrix B.

ここで，

here,

である。変換行列Ｂにおいて，＊は鏡像反転処理により係数が変化する成分を表している。

It is. In the transformation matrix B, * represents a component whose coefficient changes due to mirror image inversion processing.

Looking at the enlargement process using the matrix A described above, y ₀ to y ₂ and y _M-3 to y _M-1 (y ₁₇ to y _{19 in} equation (8)) show the influence of the mirror image inversion process during the enlargement process. receive. Therefore, in the matrix B, the 0th to 2nd rows, the 0th to 2nd rows of the matrix B having components other than 0 in the 0th column, the M-3th column to the M-1th column, and the M-3th row to the M-1th row. Are affected by the mirror image inversion process.

In the matrix shown in the above formula (10), the outer part of the line is an area affected by the mirror image reversal process at the time of enlargement. That is, z ₀ to z ₂ and z _M−3 to z _M−1 are affected by mirror image inversion processing. Incidentally, the size of the area affected by the mirror image inversion process changes depending on the number of pixels used for the enlargement process, the enlargement ratio, and the correlation position of the pixels before and after the conversion.

In the above scaling conversion, the condition for the reversible conversion, that is, the condition for Z = X, uses the unit matrix I of N columns and N rows from Z = BAX,
BA = I (11)
It becomes.

  That is, for a given matrix A, if a reduced transformation matrix B in which the matrix product BA is equal to the unit matrix I is obtained, a filter coefficient for the reduction process is determined based on the reduced conversion matrix B. Reduction processing that is inverse transformation can be performed.

Here, the components of the region affected by the image edge boundary processing are ignored, and the reduced filter coefficients (s ₁ , s ₂ ,..., S ₈ ) in which reversible transformation is established for the components of the region not affected by the image edge boundary processing. Think about how to ask.

ここで，＊は鏡像反転処理により係数が変化する成分，線の外側は拡大時の鏡像反転処理が影響する領域，？は可逆とならない成分を表している。

Where * is a component whose coefficient changes due to mirror image reversal processing, the area outside the line is the area affected by the mirror image reversal processing during enlargement,? Represents a component that is not reversible.

  In the above equation (12), since the components in the region that is not affected by the image edge boundary processing are the same pattern repetition, the same repetition pattern in the region where the image edge boundary processing of the matrices B, A, and I is not affected. Extracted as matrices B ′, A ′, and I ′, a matrix B ′ that satisfies B′A ′ = I ′ is obtained.

このＢ’を求めることで，画像境界の影響のある成分以外において等式が成り立ち，可逆変換とすることができる。上記の例は，等式に対し未知数が１つ多い連立１次方程式であり，自由度が１で解が存在する。

By obtaining this B ′, an equation is established except for the component having an influence on the image boundary, and reversible conversion can be performed. The above example is a simultaneous linear equation with one unknown greater than the equation, with a degree of freedom of 1 and a solution.

  The degree of freedom of the solution varies depending on the enlargement ratio, and there may be no solution if the enlargement ratio is less than twice. If there are multiple solutions, selecting a filter coefficient close to the reduction filter of the same scale obtained by the normal method will not result in an unnatural reduced image even in a region where reversible transformation such as the image boundary is not established. Can be.

  In practice, an error is introduced due to the accuracy of the filter coefficient and the rounding of the pixel value at the time of enlargement / reduction processing, and although it is not completely reversible, it is a relatively small error.

  On the other hand, for the area affected by image edge boundary processing, there may be a method to obtain the filter coefficient that makes the coefficient value reversible for each pixel separately from other areas, but it has the disadvantage of complicated implementation . Therefore, a method that makes it easy to implement enlargement / reduction processing by giving up reversible conversion for areas affected by image edge boundary processing and using a filter coefficient of a certain pattern in the image for both enlargement and reduction. It is desirable to take This is an economical method considering that the area affected by the image edge boundary processing is extremely small with respect to the size of the image.

  An example where N = 10 and M = 25 will be described below. Assuming that the enlargement transformation matrix is A and the reduction transformation matrix is B, the condition BA = I for reversible transformation in the region affected by the image edge boundary processing is as follows, for example.

この場合，変換率が２対５なので変換前後の画素位置の関係から，拡大処理には５セットの係数，縮小には２セットの係数が必要となる。そのため，行列Ｂ，Ａ，Ｉの画像端境界処理が影響しない領域における同一の繰り返しパターンとして行列Ｂ’，Ａ’，Ｉ’と，Ｂ”，Ａ”，Ｉ”の２系統を抜きだし，それぞれ，Ｂ’Ａ’＝Ｉ’，Ｂ”Ａ”＝Ｉ”が成立するＢ’，Ｂ”を求めることで，可逆変換となる縮小処理用フィルタ係数を求めることができる。

In this case, since the conversion rate is 2 to 5, from the relationship between the pixel positions before and after conversion, 5 sets of coefficients are required for enlargement processing and 2 sets of coefficients are required for reduction. For this reason, the matrix B ′, A ′, I ′ and the two systems B ″, A ″, I ″ are extracted as the same repetitive pattern in the region where the image edge boundary processing of the matrices B, A, I is not affected. , B′A ′ = I ′, and B ′, B ″ satisfying B ″ A ″ = I ″ can be obtained, so that a reduction processing filter coefficient that becomes a reversible transformation can be obtained.

  FIG. 1 is a diagram illustrating an apparatus configuration according to Embodiment 1 of the present invention. FIG. 2 is a process flowchart according to the first embodiment of the present invention.

  In FIG. 1, 10 is a decoder (decoder), and 20 is a re-encoder. In the decoder 10, 11 is a decoding unit that decodes an input bit stream, 12 is a 420-422 conversion unit that performs 420-422 conversion on decoded video (4: 2: 0 format video), and 13 is a post-conversion unit The 4: 2: 2 format video is output as a decoder output video, and the video output unit 14 outputs the encoding information obtained from the decoding unit 11 and the 420-422 conversion filter coefficient obtained from the 420-422 conversion unit 12. It is an information output unit.

  In the re-encoder 20, 21 is a video input unit to which a decoder output video output from the video output unit 13 of the decoder 10 is input, and 22 is an encoding output from the information output unit 14 of the decoder 10. An information input unit to which information and a filter coefficient for 420-422 conversion are input, 23 is a reversible filter calculation unit for obtaining a filter coefficient for 422-420 conversion based on the filter coefficient for 420-422 conversion, and 24 is a video input unit 21 422-420 conversion unit that performs 4: 2: 2 format video input from 4: 2: 0 format video and outputs 4: 2: 0 format video, 25 is a re-encoding of 4: 2: 0 format video And a re-encoding unit that outputs an encoded bit stream.

  The re-encoding method according to the first embodiment includes a decoding / enlargement process in the decoder 10 and a reduction process / re-encoding process in the re-encoder 20.

  After decoding the input MPEG-2 bit stream encoded in the 4: 2: 0 format in the decoding unit 11 of the decoder 10 (steps S1 and S2), the 420-422 conversion unit 12 uses 4: 2 : 420-422 conversion of 0 format decoded video is performed (step S3). The 420-422 conversion is a conversion in which only the color difference component of the video is doubled in the vertical direction. In this example, each of the converted pixels is converted by a weighted linear sum of pixel values of four neighboring pixels as follows. The pixel value at the sample position is determined.

y _2n = a × x _n−2 + b × x _n−1 + c × x _n + d × x _{n + 1} (20)
y _{2n + 1} = e × x _n-1 + f × x _n + g × x _{n + 1} + h × x _{n + 2} (21)
Here, n is an integer greater than or equal to 0 and less than N, x is a pixel value before conversion, y is a pixel value after conversion, an index is a vertical coordinate, and N is a number of pixels in the vertical direction before conversion. It is assumed that mirror image inversion processing is performed at the boundary. That is, x ₋₁ = x ₁ , x ₋₂ = x ₂ ,..., X _−n = x _n , x _N = x _N−2 , x _{N + 1} = x _{N−3 ,. xNn-2} . In addition, the pixel value obtained by the above equation is rounded to a value that the pixel value can take, which is an integer from 0 to 255.

  These 420-422 conversion filter coefficients (a, b, c, d) and (e, f, g, h) are expanded based on the FIR filter coefficients having the Nyquist frequency of the image before the expansion as a cutoff frequency. A value obtained by considering the correlation between the previous and subsequent pixel positions and rounding the value to an appropriate accuracy is used.

  The 420-422 conversion unit 12 passes the 4: 2: 2 format video thus obtained to the video output unit 13, and the video output unit 13 outputs this as a decoder output video. At the same time, the information output unit 14 outputs all encoded information including a motion vector other than the DCT coefficient obtained from the decoding unit 11 and a macroblock mode. At the same time, 420-422 conversion filter coefficients (a, b, c, d) and (e, f, g, h) obtained from the 420-422 conversion unit 12 are output (step S4).

  The re-encoder 20 receives the encoded information, the decoder output video (4: 2: 2 format video) output from the video output unit 13, and the filter coefficient for 420-422 conversion (step S5). The video input unit 21 of the re-encoder 20 passes the 4: 2: 2 format video that is the input decoder output video to the 422-420 conversion unit 24.

  The information input unit 22 of the re-encoder 20 confirms from the input encoded information that the pre-encoding has been performed in the 4: 2: 0 format (step S6), and the input 420- The filter coefficients (a, b, c, d) and (e, f, g, h) for 422 conversion are passed to the reversible filter calculation unit 23.

  Based on the 420-422 conversion filter coefficients, the reversible filter calculation unit 23

を満たす，４２２−４２０変換用フィルタ係数（ｐ，ｑ，ｒ，ｓ，ｔ，ｕ，ｖ，ｗ）を求める（ステップＳ７）。

422-420 conversion filter coefficients (p, q, r, s, t, u, v, w) that satisfy the above are obtained (step S7).

  This equation is a simultaneous linear equation with one more unknown than the equation, and since there is a solution with one degree of freedom, for example, a single variable such as s is fixed and a simultaneous 1 such as Gaussian elimination is used. Find the solution by solving the following equation. This is performed for a plurality of s, and the difference from the default filter coefficients (p ′, q ′, r ′, s ′, t ′, u ′, v ′, w ′) for the 422-420 conversion is evaluated. A coefficient with a small difference is selected (step S8).

  For this evaluation, for example, the sum of squares of differences between coefficients is used. The default filter coefficient for 422-420 conversion is a filter coefficient obtained in advance by a normal method, and before and after the reduction based on the FIR filter coefficient using the Nyquist frequency of the reduced image as the cutoff frequency. This is determined in consideration of the correlation between pixel positions.

  The reversible filter calculation unit 23 passes the 422-420 conversion filter coefficients (p, q, r, s, t, u, v, w) determined in this way to the 422-420 conversion unit 24. Using this filter coefficient, the 422-420 conversion unit 24 performs 422-420 conversion on the 4: 2: 2 format video as follows (step S9), and outputs the 4: 2: 0 format video. .

z _n = p × y _2n−3 + q × y _2n−2 + r × y _2n−1 + s × y _2n + t × y _{2n + 1} + u × y _{2n + 2} + v × y _{2n + 3} + w × y _{2n + 4} ( 23)
Here, n is an integer greater than or equal to 0 and less than N, y is a pixel value before conversion, z is a pixel value after conversion, an index is a vertical coordinate, and 2N is a number of pixels in the vertical direction before conversion. Is assumed to be mirror-inverted. _{That, y -1 = y 1, y} -2 = y 2, ..., y -n = y n, y 2N = y 2N-2, y 2N + 1 = y 2N-3, ..., y 2N + n = _Let y _2N-n-2 . In addition, the pixel value obtained by the above equation is rounded to a value that the pixel value can take, which is an integer from 0 to 255. As a result, a 4: 2: 0 format video substantially equal to the 4: 2: 0 format decoded video in the decoder 10 can be obtained.

  In this method, the three lines at the top and bottom of the image are affected by mirror image processing and the image changes. However, since filter coefficients close to the default filter coefficients are used, there is no significant deterioration.

  Further, the re-encoding unit 25 re-encodes the 4: 2: 0 format video obtained in this way according to the encoding information obtained from the information input unit 22 (step S10), and outputs a bit stream. (Step S11). As a result, the decoded video of the re-encoded video stream has almost no deterioration compared to the previous decoded video.

  FIG. 3 is a diagram showing a device configuration according to Embodiment 2 of the present invention. FIG. 4 is a process flowchart according to the second embodiment of the present invention.

  In FIG. 3, 30 is a decoder and 40 is a re-encoder. In the decoder 30, 31 is a decoding unit that decodes an input bit stream, 32 is an enlargement processing unit that performs an enlargement process on the decoded video, 33 is a video output unit that outputs the enlarged video as a decoder output video, and 34 is It is an information output unit that outputs encoded information obtained from the decoding unit 31 and outputs filter coefficients for reduction processing.

  In the re-encoder 40, 41 is a video input unit to which a decoder output video output from the video output unit 33 of the decoder 30 is input, and 42 is an encoding output from the information output unit 34 of the decoder 30. An information input unit 43 to which information and a filter coefficient for reduction processing are input, 43 performs a reduction process on the input video based on the filter coefficient for reduction processing, and outputs a reduced video. A re-encoding unit that performs encoding and outputs a bit stream.

  The example of the re-encoding method shown in the second embodiment includes a decoding / enlargement process in the decoder 30 and a reduction process / re-encoding process in the re-encoder 40. The encoded MPEG-2 bit stream input to the decoder 30 is assumed to be an encoded stream after being reduced in the horizontal direction to 2/5 at the time of encoding.

  After the decoding unit 31 of the decoder 30 decodes the input bit stream (steps S21 and S22), the enlargement processing unit 32 enlarges the decoded video by 5/2 times in the horizontal direction (step S23). In this example, the pixel value at each sample position after conversion is determined by the weighted linear sum of the pixel values of the four neighboring pixels as described below.

_{y 5n = a 1 × x 2n} -2 + a 2 × x 2n-1 + a 3 × x 2n + a 4 × x 2n + 1 (24)
_{y 5n + 1 = b 1 ×} x 2n-1 + b 2 × x 2n + b 3 × x 2n + 1 + b 4 × x 2n + 2 (25)
_{y 5n + 2 = c 1 ×} x 2n-1 + c 2 × x 2n + c 3 × x 2n + 1 + c 4 × x 2n + 2 (26)
_{y 5n + 3 = d 1 ×} x 2n-1 + d 2 × x 2n + d 3 × x 2n + 1 + d 4 × x 2n + 2 (27)
_{y 5n + 4 = e 1 ×} x 2n + e 2 × x 2n-1 + e 3 × x 2n + e 4 × x 2n + 3 (28)
Here, n is an integer greater than or equal to 0 and less than N, x is a pixel value before conversion, y is a pixel value after conversion, an index is a horizontal coordinate, 2N indicates a horizontal pixel number before enlargement, and an image It is assumed that mirror image inversion processing is performed at the boundary. _{That, x -1 = x 1, x} -2 = x 2, ..., x -n = x n, x 2N = x 2N-2, x 2N + 1 = x 2N-3, ..., x 2N + n = _x2N-n-2 . In addition, the pixel value obtained by the above equation is rounded to a value that the pixel value can take, which is an integer from 0 to 255.

  This enlargement processing filter coefficient is obtained by considering the correlation between the pixel positions before and after enlargement based on the FIR filter coefficient having the Nyquist frequency of the image before enlargement as a cutoff frequency, and rounding the value to an appropriate accuracy. Value is used.

  The enlargement processing unit 32 passes the enlarged video obtained in this way to the video output unit 33, and the video output unit 33 outputs it as a decoder output video. At the same time, the information output unit 34 outputs all encoded information including a motion vector other than the DCT coefficient obtained by the decoding unit 31 and a macroblock mode. At the same time, the filter coefficient for reduction processing that is reversible conversion of the enlargement processing is output (step S24).

This reduction processing filter coefficient (s ₁ , s ₂ , s ₃ , s ₄ , s ₅ , s ₆ , s ₇ , s ₈ , s ₉ , s ₁₀ ), (t ₁ , t ₂ , t ₃ , t ₄ , T ₅ , t ₆ , t ₇ , t ₈ , t ₉ , t ₁₀ ) are obtained as follows according to the process shown in FIG.

  The enlargement processing filter coefficient in the case of the enlargement ratio 5/2 is obtained by a normal method (step S241), and the default reduction processing filter coefficient in the case of the reduction ratio 2/5 is obtained by a normal method (step S242). The default reduction filter coefficient is determined in consideration of the correlation between the pixel positions before and after reduction based on the FIR filter coefficient whose cutoff frequency is the Nyquist frequency of the reduced image.

  From the enlargement processing filter coefficients,

を満たす縮小処理用フィルタ係数を求める。

The filter coefficient for reduction processing that satisfies the above is obtained.

These equations are simultaneous linear equations with three unknowns compared to the equation, and there are 3 degrees of freedom, so there are three solutions, for example, fixing three variables such as s ₄ , s ₅ , and s ₆ , A solution is obtained by solving simultaneous linear equations such as Gaussian elimination.

This is performed for a plurality of patterns, and filter coefficients (s ′ ₁ , s ′ ₂ , s ′ ₃ , s ′ ₄ , s ′ ₅ , s ′ ₆ , s ′ ₇ , s ′ ₈ , s for default reduction processing are performed. _'9, s' _10), and _{(t '1, t' 2} , t '3, t' 4, t '5, t' 6, t '7, t' 8, t '9, t' 10) Are evaluated, and the smallest coefficient is selected as a filter coefficient for reduction processing (step S244).

  The enlargement processing filter coefficient and the reduction processing filter coefficient are stored in the decoder 30 (step S245). The information output unit 34 of the decoder 30 outputs the filter coefficient for reduction processing previously obtained in this way together with the encoded information.

  The re-encoder 40 receives the filter coefficient for reduction processing, the decoder output video, and the encoding information (step S25). The information input unit 42 of the re-encoder 40 confirms that the enlargement ratio in the decoder 30 is 5/2 times in the horizontal direction from the encoded image size of the input encoded information and the image size of the decoder output video. After that (step S26), the same input filter coefficient for reduction processing is transferred to the reduction processing unit 43. The reduction processing unit 43 performs 2/5 reduction processing based on the reduction processing filter coefficient as follows (step S27).

_{z 2n = s 1 × y 5n} -4 + s 2 × y 5n-3 + s 3 × y 5n-2 + s 4 × y 5n-1 + s 5 × y 5n + s 6 × y 5n + 1 + s 7 × y 5n + 2 + S ₈ × y _{5n + 3} + s ₉ × y _{5n + 4} + s ₁₀ × y _{5n + 5} (31)
_{z 2n + 1 = t 1 ×} y 5n-1 + t 2 × y 5n + t 3 × y 5n + 1 + t 4 × y 5n + 2 + t 5 × y 5n + 3 + t 6 × y 5n + 4 + t 7 × y 5n _{+5 + t 8 × y 5n +} 6 + t 9 × y 5n + 7 + t 10 × y 5n + 8 (32)
Here, n is an integer from 0 to less than N, y is a pixel value before conversion, z is a pixel value after conversion, an index is a horizontal coordinate, 5N is a horizontal image size before reduction, and at an image boundary, Assume that mirror image inversion processing is performed. _{That, y -1 = y 1, y} -2 = y 2, ..., y -n = y n, y 5N = y 5N-2, y 5N + 1 = y 5N-3, ..., y 5N + n = y _5N-n-2 . In addition, the pixel value obtained by the above equation is rounded to a value that the pixel value can take, an integer from 0 to 255.

  As a result, it is possible to obtain a video substantially equal to the decoded video before enlargement in the decoder 30. In this method, the three lines at the top and bottom of the image are affected by mirror image processing and the image changes. However, since filter coefficients close to the default filter coefficients are used, there is no significant deterioration.

  Further, the re-encoder 40 re-encodes the reduced video obtained in this way according to the input encoding information (step S28), and outputs a bit stream (step S29). As a result, the decoded video of the re-encoded video stream has almost no deterioration compared to the previous decoded video.

It is a figure which shows the apparatus structure which concerns on Example 1 of this invention. It is a processing flowchart concerning Example 1 of the present invention. It is a figure which shows the apparatus structure which concerns on Example 2 of this invention. It is a processing flowchart concerning Example 2 of the present invention. It is a process flowchart which calculates the filter coefficient for a reduction process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10,30 Decoder 11,31 Decoding part 12 420-422 conversion part 13,33 Video output part 14,34 Information output part 20,40 Re-encoder 21,41 Video input part 22,42 Information input part 23 Reversible filter Calculation unit 24 422-420 conversion unit 25, 44 Re-encoding unit 32 Enlargement processing unit 43 Reduction processing unit

Claims (14)

A reduction process used to reduce the resolution when re-encoding an image signal obtained by expanding the resolution of a decoded video obtained by decoding a video encoded stream using an enlargement processing filter after reducing the resolution to the resolution before the enlargement. A resolution conversion filter coefficient determination method for determining a filter coefficient of a filter for an image,
A process of inputting a filter coefficient of a fixed pattern applied to at least a region other than the region affected by the image edge boundary processing and storing it in a storage means , which is a filter coefficient of the enlargement processing filter used for the resolution enlargement processing;
A matrix of 1 column and N rows representing 1-dimensional image data of N pixels before enlargement is X, a matrix of 1 column and M rows representing 1-dimensional image data of M (M> N) pixels after enlargement is Y, and enlargement processing is performed. An N-column, M-row expansion transformation matrix representing Y = AX is represented as A, a 1-column N-row matrix representing one-dimensional image data composed of N pixels after the reduction is represented as Z, and M representing Z = BY representing a reduction process. A matrix of a region portion in which at least image edge boundary processing does not affect at least the filter coefficient of a predetermined pattern of the stored enlargement processing filter, where B is a reduced conversion matrix of N columns and I is a unit matrix of N columns and N rows A process of calculating a reduced transformation matrix B satisfying BA = I from an enlarged transformation matrix A determined by the filter coefficient , including a row of elements in the row direction ,
A resolution conversion filter coefficient determination method, comprising: determining a filter coefficient of a reduction processing filter based on the calculated reduction conversion matrix B. In the resolution conversion filter coefficient determination method according to claim 1,
In the process of calculating the reduced transformation matrix B with BA = I, the product BA of the reduced transformation matrix B and the enlarged transformation matrix A is N columns at least in the region portion where the image edge boundary processing in the enlargement / reduction processing does not affect. In order to obtain a filter coefficient for the reduction process that is equal to the unit matrix I of N rows, the arrangement of the element values in the row direction of the matrix is repeated in the region where the image edge boundary processing of the matrices B, A, I is not affected. The same repeating pattern unit of the portion that appears in the same manner is extracted from the matrices B, A, and I as matrices B ′, A ′, and I ′ , respectively, and a matrix B ′ that satisfies B′A ′ = I ′ is obtained. A resolution conversion filter coefficient determination method, comprising: determining a filter coefficient of a reduction processing filter based thereon. In an image resolution conversion method for reducing an image signal obtained by enlarging the resolution of a decoded video obtained by decoding a video encoded stream using an enlargement processing filter to a resolution before enlargement before re-encoding ,
A step of inputting an image signal with an enlarged resolution using the enlargement processing filter;
3. The resolution of the input image signal is reduced to the resolution before enlargement using a reduction processing filter comprising filter coefficients determined using the resolution conversion filter coefficient determination method according to claim 1 or 2. And a process for converting the image resolution. In an image resolution conversion apparatus for reducing an image signal obtained by enlarging a resolution of a decoded video obtained by decoding a video encoded stream using an enlargement processing filter to a resolution before enlargement before re-encoding ,
Means for inputting an image signal whose resolution is enlarged using the enlargement processing filter;
3. The resolution of the input image signal is reduced to the resolution before enlargement using a reduction processing filter comprising filter coefficients determined using the resolution conversion filter coefficient determination method according to claim 1 or 2. An image resolution conversion apparatus. When re-encoding the video encoded stream by the decoder, the resolution of the decoded video is increased. When the video is re-encoded by the re-encoder, the resolution of the input video is reduced to the pre-enlargement resolution before re-encoding. In the encoding method,
Transmitting information on resolution conversion filter coefficients used for enlarging the resolution of decoded video in the decoder to the re-encoder;
3. The filter coefficient of the reduction processing filter is determined from the resolution conversion filter coefficient information transmitted to the re-encoder using the resolution conversion filter coefficient determination method according to claim 1 or 2. Process,
A process of reducing the resolution of an input video signal to a resolution before enlargement using a reduction filter composed of the determined filter coefficients;
And a process of re-encoding the video signal with reduced resolution. When re-encoding the video encoded stream by the decoder, the resolution of the decoded video is increased. When the video is re-encoded by the re-encoder, the resolution of the input video is reduced to the pre-enlargement resolution before re-encoding. In the encoding method,
A reduction processing filter determined using the resolution conversion filter coefficient determination method according to claim 1 or 2 from information on resolution conversion filter coefficients used for increasing the resolution of decoded video in the decoder. Inputting filter coefficient information and a video signal to the re-encoder;
Using the reduction filter based on the input filter coefficient information to reduce the resolution of the input video signal to the resolution before enlargement;
And a process of re-encoding the video signal with reduced resolution. In a video re-encoding device that inputs a video signal obtained by enlarging the resolution of a decoded video at the time of decoding a video encoded stream and re-encodes after reducing the resolution of the input video to the resolution before enlargement,
Video input means for inputting a video signal to be re-encoded;
Information input means for inputting information on resolution conversion filter coefficients used for enlarging the resolution of the decoded video at the time of decoding the video encoded stream;
Reversible filter calculation means for determining the filter coefficient of the reduction processing filter from the input resolution conversion filter coefficient information using the resolution conversion filter coefficient determination method according to claim 1 or 2 ;
Resolution conversion means for reducing the resolution of the input video signal to the resolution before enlargement using a reduction processing filter comprising the determined filter coefficients;
A video re-encoding apparatus comprising: a re-encoding unit that re-encodes a video signal with reduced resolution. In a video re-encoding device that inputs a video signal obtained by enlarging the resolution of a decoded video at the time of decoding a video encoded stream and re-encodes after reducing the resolution of the input video to the resolution before enlargement,
Video input means for inputting a video signal to be re-encoded;
3. Reduction processing determined using the resolution conversion filter coefficient determination method according to claim 1 or 2 from information on resolution conversion filter coefficients used to increase resolution of decoded video during decoding of the video encoded stream Information input means for inputting the filter coefficient of the filter for use,
Resolution conversion means for reducing the resolution of the input video signal to the resolution before enlargement using the reduction processing filter based on the input filter coefficient information;
A video re-encoding apparatus comprising: a re-encoding unit that re-encodes a video signal with reduced resolution. A resolution conversion filter coefficient determination program for causing a computer to execute the resolution conversion filter coefficient determination method according to claim 1 . An image resolution conversion program for causing a computer to execute the image resolution conversion method according to claim 3 . A video re-encoding program for causing a computer to execute the video re-encoding method according to claim 5 or 6 . A recording medium recording a resolution conversion filter coefficient determination program for causing a computer to execute the resolution conversion filter coefficient determination method according to claim 1 . A recording medium recording an image resolution conversion program for causing a computer to execute the image resolution conversion method according to claim 3 . A recording medium on which a video re-encoding program for causing a computer to execute the video re-encoding method according to claim 5 or 6 is recorded.

Images