JP4258537B2 - Image signal generation method and image signal generation apparatus - Google Patents

Description

The present invention relates to an image signal generation method and an image signal generation apparatus .

In the field of high-efficiency encoding of image signals, various encoding schemes have been devised so far. Of these, video encoding system that combines a motion compensation prediction coding and orthogonal transform coding, the prediction signal generated by motion compensation with motion vector signal of the previous frame, predicting a difference signal between the signal of the current frame By encoding a transform coefficient obtained by extracting the error signal and performing DCT transform (discrete cosine transform) on the prediction error signal, coding with high compression efficiency can be realized. It has been adopted for MPEG1 and MPEG2 video coding, which are international standards, and research and development for future digital broadcasting, CATV, and package media are underway. In digital broadcasting using this encoding method, a high-quality image equivalent to the original image can be received at a bit rate of about 20 mega. However, since most of the image signals are interlaced scanning, there remains a problem that image quality deterioration due to interlace interference such as line flicker and pairing due to this occurs.

On the other hand, as a technique for reducing interlace interference, interlace to non-interlace conversion is known in which a scanning line signal missing in interlace scanning is generated by interpolation processing and displayed as a non-interlace scanning signal. However, the generation of the compensation signal requires motion adaptive signal processing that changes the characteristics according to the motion of the image. However, since motion information is detected by an interlaced scanning signal, there is a problem that a detection error or a false detection occurs, and image deterioration due to this occurs.

Therefore, there is a problem that even if the interference interference reduction technique described above is applied to the decoded image of the digital broadcast, the effect of improving the image quality is small and the high quality digital broadcast feature cannot be fully utilized.

Japanese Patent Laid-Open No. 9-182032

An object of the present invention is to provide an image signal generation method and an image signal generation apparatus that solve the above problems and generate a high-quality and high-definition image.

In order to achieve the above object, for example, it is configured as described in the claims.

According to the present invention, it is possible to realize an image signal encoding / decoding method and apparatus capable of receiving a high-quality and high-definition image free from interlace interference and judder interference, and has a remarkable effect on image quality improvement and improvement. can get.

Motion compensation predictive coding and motion compensation scan conversion processing according to an embodiment of the present invention will be described with reference to the operation schematic diagrams of FIGS.
FIG. 1 shows the operation when the video signal is interlaced and the motion is in the vertical direction.
In the inter-frame predictive encoding in FIG. 6A, for example, the motion vector V (horizontal V _x , horizontal frame _x 1) in one frame period is applied to the signal sequence of the first field and the signal sequence of the second field in interlace scanning . Extract vertical V _y ). It should be noted that the motion vector V is V _x = 2m, V _y = 2n (m = 0, ± 1, ± 2,..., N = 0, ± 1, so as to be consistent with the scan conversion processing for motion compensation described later . Extract with a combination of ± 2, ...).
Then, the difference between the prediction signal generated by motion compensation using the motion vector V and the original signal is detected as a prediction error signal. For example, the motion vector V _0, for detecting a difference signal of the scanning lines a and 1, V ₁ the scanning lines a and 2, V in ₂ scanning lines a and 3 as a prediction error signal.
On the other hand, in the motion compensation scan conversion process, a signal of a scan line (interpolated scan line indicated by a black circle in the figure) missing in interlace scan is generated based on the motion vector V. For example, the first (2) Field of the interpolated scanning line a (a '), the motion vector V ₀ in the second (1) Field of the scanning lines alpha and alpha' from (α + α ') / 2 , V 1 the scanning line In β and β ′) / 2, V ₂ , an interpolation signal of (γ + γ ′) / 2 is generated from the scanning lines γ and γ ′. Then, interlaced to non-interlaced conversion is realized with this interpolation signal.
Incidentally, V _x = 2m relative motion vector V in the predictive coding, V _y = a case without the constraints of 2n is, V _x = 2m of the motion vector V in the scan conversion process of the motion compensation, V _y = 2 n is selected. Based on this motion vector, the above-described interpolation signal is generated.
In the inter-field predictive coding shown in FIG. 5B, a prediction signal generated by extracting a motion vector in one field period from the signal sequence of the first (2) field and performing motion compensation with this motion vector. And the difference from the signal of the second (1) field is detected as a prediction error signal.
In the scan conversion process for motion compensation, a motion vector V whose components satisfy the conditions of V _x = m and V _y = n is selected. Then, based on the selected motion vector, a signal of a scanning line (interpolated scanning line indicated by a black circle in the figure) missing in interlace scanning is generated. For example, the interpolated scanning line a (a ′) in the first (2) field is the scanning line α in the second (1) field scanning line α, V ₁ ′ in the motion vector VV ₀ ′, and the scanning line α in V ₂ ′. Thus, an interpolation signal is generated.
As described above, in one embodiment of the present invention, an interpolation signal close to ideal can be generated by using a motion vector suitable for scan conversion processing for motion compensation. On the other hand, when there is no motion vector that satisfies the conditions of the scan conversion processing for motion compensation, an interpolation signal is generated by the motion adaptive signal processing similar to the prior art.
In the scan conversion process for motion compensation, the accuracy of interpolation depends on the motion vector. For this reason, when the accuracy of the motion vector is poor, it is expected that the image deteriorates. However, this type of deterioration in image quality can be avoided by changing the motion compensation signal processing in a region where the component of the inter-frame difference signal for motion compensation is the same as in the prior art. In one embodiment of the present invention, it is possible to achieve high-quality scan conversion with extremely low interlace interference and high resolution as compared with the prior art.
Some video signals are non-interlaced. For example, cinema images and commercial films can be regarded as non-interlaced scanning signals of 24 frames or 30 frames per second. In order to avoid large-area flicker, the signal having this value is converted into a signal of, for example, 60 frames per second by a frame number conversion process and displayed.
The second is a schematic diagram of predictive coding of motion compensation and scan conversion of motion compensation in an embodiment of the present invention for a video signal of this type of non-interlaced scanning.
In predictive coding, a motion vector V between frames is extracted, and a difference between a previous frame signal and a current frame signal motion-compensated with the motion vector is detected as a prediction error signal. For example, the difference between the scanning lines a and 1 in the motion vector V ₀ and the scanning lines a and 2 in the V ₁ and the scanning lines a and 3 in the V ₂ is detected as a prediction error signal.
On the other hand, in motion compensation scanning conversion processing, signals of interpolation scanning lines indicated by black circles in the figure are generated with the motion vectors shown in the figure. For example, the standard mode converts 30-60 frames, and the cinema mode 24-60. Realize the conversion. That is, in the standard mode, interpolation scanning line a is 'from (alpha + alpha' motion vector V ₀ in scan lines alpha and α) / _2, V 2 the 'from (beta and beta' and scanning lines beta beta) / 2, In V ₄ , an interpolation signal of (γ + γ ′) / 2 is generated from the scanning lines γ and γ ′. On the other hand, in the cinema mode, the interpolated scanning line a is an interpolated signal from the scanning lines α and α ′ to (α + α ′) / 2 in the motion vector V ₀ , and from the scanning lines β and β ′ to (β + β ′) / 2 in V _3. Is generated. The interpolated scanning line a ′ is an interpolated signal from the scanning lines γ and γ ′ to (γ + γ ′) / 2 in the motion vector V ₀ , and from the scanning lines α ′ and α ″ to (α ′ + α ″) / 2 in V _4. Is generated. Then, an interpolation signal close to ideal is generated with these motion vectors suitable for scan conversion processing for motion compensation.
If there is no motion vector suitable for performing the scan conversion process of motion compensation, an interpolation signal is generated using the average value of the signal of the previous frame or the signal of the previous and subsequent frames, as in the conventional case.
The motion compensation scan conversion process is performed on a region where the component of the motion compensation inter-frame difference signal is small. Then, degradation of image quality due to poor motion vector accuracy is avoided. Therefore, according to the embodiment of the present invention, it is possible to achieve scan conversion of frame number conversion with higher resolution and less judder disturbance (disturbance in which the smoothness of motion is impaired) as compared with the prior art.
A first embodiment of the present invention will be described with reference to an overall block diagram shown in FIG. The present embodiment is suitable for encoding by limiting the motion vector used in the motion compensation prediction encoding of the encoding unit to a form suitable for the scan conversion process of motion compensation. In the present embodiment, a case of video coding combining motion compensation prediction coding between frames and orthogonal transform coding of DCT transform (discrete cosine transform) will be described as an example.

In the encoding unit of FIG. 5A, the image signal VS is input to the preprocessing unit 1, digitized, and predetermined image format conversion based on image information PI (information such as image format and encoding mode). To generate an encoded image signal sequence S1. For example, MPEG1
, 2, S1 is a macroblock signal sequence composed of six blocks (four luminance signal blocks and two chrominance signal blocks, and the block size is 8 pixels × 8 lines).

The encoded image signal sequence S1 is transmitted in accordance with a predetermined encoding mode.
DCT transform coding is used for the abbreviated I picture), and interframe prediction coding and DCT transform coding are used for the inter-frame coded frame (abbreviated P picture).
FIG. 4 is a schematic diagram of an encoding mode for an interlaced scanning image signal. Figure (a)
In the independent encoding of the even / odd field, a sequence of encoded frames is constituted by the signal of the first field and the signal of the second field, respectively. Also, in the encoding after the progressive scan conversion in FIG. 5B, the first field signal and the second field signal are combined to form a sequence of encoded frames, and encoding is performed.
On the other hand, in the inter-frame / inter-field adaptive predictive coding shown in FIG. 6C, the inter-frame and inter-field prediction error signals are obtained for a sequence of encoded frames composed of the first field signal and the second field signal. The smaller one is selected and encoded. Hereinafter, the encoding operation will be described by taking the independent encoding of FIG.

The switches 4 and 11 are connected to the terminal a in the I picture encoding mode and connected to the terminal b in the P picture encoding mode to perform the encoding process. That is, the switch 4 uses the signal sequence S1 from the signal sequence S1 to the MC in the subtraction unit 3 when the I picture is the original image encoded image signal sequence S1.
An inter-frame prediction error signal sequence S2 obtained by subtracting the inter-frame motion compensation prediction signal sequence S10 generated by the signal generation unit 13 is output to the signal S3.

The DCT unit 5 applies 8 rows × 8 to the signal of each block (8 pixels × 8 lines) of the signal S3.
A matrix operation with the DCT transformation matrix of the column is performed to generate a transformation coefficient signal series S4. The quantization unit 6 performs a transform coefficient quantization process with predetermined characteristics to generate a coefficient quantized signal sequence S5. Note that the quantization characteristic is controlled according to the buffer retention amount of the buffer unit 16 to ensure a predetermined transmission bit rate.

The inverse quantization unit 8 decodes the transform coefficient signal sequence S6 by inverse quantization processing, the IDCT unit 9 performs a matrix operation with the DCT transform inverse matrix of 8 rows × 8 columns, and an encoded image signal for I picture series,
During P picture, the signal S7 of the inter-frame prediction error signal sequence is decoded. And the addition part 10
Performs addition with the inter-frame motion compensation prediction signal sequence S10, and decodes the encoded image signal sequence in the P picture. The switch 11 has a terminal a for I picture and a terminal b for P picture.
To output the decoded encoded image signal sequence S8. The frame memory unit 12 generates a signal S9 delayed for one frame.

The motion vector detection unit 2 extracts a motion vector MV that is consistent with the scan conversion processing for motion compensation in the decoding unit according to the image information PI. That is, as shown in the scanning line interpolation of FIG. 1 when the image signal is interlaced, the motion vectors V ₀ , V ₁ , V ₂ (
V _x = 2m, to extract the V _y = 2n). When the image signal is non-interlaced scanning, the motion vector shown in the scanning line interpolation of FIG. 2 is extracted. The motion vector is extracted by a block matching method or the like. Then, the MC signal generation unit 13 performs a motion compensation process on the signal S9 with the motion vector MV, and generates an inter-frame motion compensation prediction signal sequence S10.

The VLC units 7 and 14 perform a predetermined variable length encoding process on the coefficient quantized signal sequence S5 and the motion vector MV to generate an encoded image data sequence S11 and an encoded motion vector sequence S12. And in the multiplexing part 15, various data, such as an encoding parameter of image information PI,
A process of multiplexing the encoded image data sequence S11 and the encoded motion vector sequence S12 in a time division manner is performed, and the generated encoded data sequence S13 is input to the buffer unit 16.

A signal is read from the buffer unit 16 at a constant bit rate, and an encoded bit stream signal PCD is generated. Although not explicitly shown in the drawing, this signal PCD is a predetermined channel encoding such as countermeasure against code error or digital modulation according to the transmission medium (for example, digital modulation such as multi-level QAM and multi-level VSB in digital broadcasting). ) And transmit.

In the decoding unit shown in FIG. 5B, the decoded bit stream signal PCD decoded by channel decoding is input to the buffer unit 17. The encoded data sequence S13 read from the buffer unit is separated by the DMPX unit 18 into the encoded image data sequence S11, the encoded motion vector sequence S12, and various data such as the encoding parameter of the image information PI. .

The IVLC units 19 and 20 perform a variable length code decoding process, and decode the coefficient quantized signal sequence S5 and the motion vector MV.

The inverse quantization unit 8 decodes the transform coefficient signal sequence S4 by inverse quantization processing, and the IDCT unit 9
Matrix operation with a DCT transformation inverse matrix of rows × 8 columns is performed. Then, in the decoding at the time of I picture, the encoded image signal sequence is decoded, and in the decoding at the time of P picture, the signal S3 corresponding to the inter-frame prediction error signal sequence is decoded. The adding unit 21 adds the inter-frame motion compensated prediction signal sequence S10 generated by the MC signal generating unit 24, and decodes the encoded image signal sequence S14 for the P picture. The switch 22 is connected to the terminal a for I pictures and to the terminal b for P pictures, and obtains a decoded encoded image signal sequence S15 as an output thereof.

The frame memory unit 23 generates a signal S9 delayed by a period of one frame, and the MC signal generation unit 24 performs a motion compensation process using the motion vector MV on the signal, and performs an interframe motion compensated prediction signal sequence S10. Is generated.

The MC interpolation unit 25 performs a motion compensation scan conversion process as shown in FIG . 1 or FIG. 2 based on the motion vector MV, the image information PI, and the inter-frame prediction error signal sequence of the signal S3. A signal sequence S16 and an interpolation signal sequence S17 are generated.
Details of this operation will be described later.

The post-processing unit 26 performs image format conversion processing, interlace to non-interlace scan conversion or frame number conversion processing, and conversion to an analog signal, and decodes the non-interlace scan image signal VSD. This signal is presented to a non-interlaced scanning image display (not shown in the drawing) to receive a high-quality and high-definition image.

The description of the overall block configuration is thus completed, and the MC interpolation unit 25 will be described based on the embodiments shown in FIGS.

FIG. 5 is a diagram illustrating the first configuration example, which is suitable when the accuracy of the motion vector used in the scan conversion processing for motion compensation is extremely high.

The decoded encoded image signal sequence S15 includes a field memory unit 27-1 and an MCIP unit 2
9 and input. The field memory unit delays a signal for one field period. Then, the main signal series S16 is generated from this output signal. Further, this signal is delayed by one field period in the field memory unit 27-2 to generate a signal S9 delayed by one frame period with respect to the signal S15. A portion surrounded by a dotted line in the figure is the frame memory unit 2 shown in FIG.
It corresponds to 3. Therefore, this portion can be configured to be shared as a frame memory.

The interpolation MV generation unit 28 determines whether the motion compensation scan conversion is interlace to non-interlace scan conversion or frame number conversion based on the image information PI. Then, as shown in FIG. 1 in the former case and in FIG. 2 in the latter case, an interpolation motion vector IMV necessary for generating a motion compensation interpolation signal is generated based on the motion vector MV.
For example, in FIG. 1A, when the motion vector MV is V ₀ , the signal of the scanning line α of the signal S9 and the signal of the scanning line α ′ of the signal S15, and when the motion vector MV is V ₁ , An interpolated motion vector IMV specifying the signal of the scanning line β ′ in S15 is generated.

The MCIP unit 29 extracts signals of the scanning line specified by the interpolation motion vector IMV from the signals S9 and S15, generates a motion compensation interpolation signal using the average value of both signals, and generates an interpolation signal sequence S17. .

Next, this second configuration example is shown in FIGS. This is also suitable when the motion vector accuracy is slightly poor.

In the configuration example diagram shown in FIG. 6, the field memory units 27-1 and 27-2 constitute a frame memory, and the signal of the main signal series S <b> 16 and motion compensation scan conversion processing are performed as in the first configuration example. And a signal S9 delayed for one frame period with respect to this signal. A portion surrounded by a dotted line in the figure is the frame memory unit 23 shown in FIG.
Therefore, the embodiment of FIG. 3 can be configured to share this portion.

The MC interpolation control unit 30 uses the motion vector MV, the image information PI, and the inter-frame difference signal component of motion compensation obtained by the signal S3, the interpolation motion vector IMV necessary for generating a motion compensation interpolation signal, The adaptive coefficients km and 1-km are generated. It is determined from the image information PI whether the motion compensation scan conversion is interlace to non-interlace scan conversion or frame number conversion. Then, as shown in FIG. 1 for the former and FIG. 2 for the latter, an interpolation motion vector IMV necessary for generating an interpolation signal for motion compensation is generated. However, unlike the first configuration example, the interpolation motion vector IMV is generated with the characteristics shown in FIG. The prediction error signal of the signal S3, that is, the absolute value sum Σ | S3 | of the motion compensation inter-frame difference signal component is measured for each block (8 pixels × 8 lines). When this is less than the threshold Th, it is determined that the accuracy of the motion vector is high, and an interpolated motion vector IMV is generated based on the motion vector MV, as in the first configuration example. On the other hand, if Σ | S3 | is equal to or greater than the threshold value Th, it is determined that the accuracy of the motion vector is poor, and 0 is generated as the interpolation motion vector. Further, as shown in FIG. 7B, the motion adaptive coefficient km generates a coefficient value having a characteristic that changes from 0 to 1 in accordance with the value of Σ | S3 |.

The MCIP unit 29 extracts the scanning line signal specified by the interpolation motion vector from the signals S9 and S15, and generates an interpolation signal S18 for motion compensation using the average value of both signals. Note that when the accuracy of the motion vector is poor, IMV = 0, so the signal S18 is equivalent to the interpolation signal in the still mode of the prior art.

The intra-field interpolation unit 31 performs intra-field calculation processing based on the signal S16, and generates a moving image interpolation signal S19 similar to the moving image mode of the prior art with the same field signal.

The coefficient weighting unit 32-1 weights the signal S18 with the motion adaptive coefficient 1-km, and the coefficient adding unit 32-2 weights the signal S19 with the motion adaptive coefficient km. Then, the adder 33 adds both signals to generate an interpolation signal sequence S17.

Next, this third configuration example is shown in FIG. This is also suitable when the accuracy of the motion vector is considerably poor.

In the configuration diagram of FIG. 5A, the field memory units 27-1 and 27-2 constitute a frame memory, and, as in the first and second configuration examples, the signal of the main signal sequence S16 and the motion compensation A signal S15 used for the scan conversion process and a signal S9 obtained by delaying this signal by one frame period are generated. The portion surrounded by a dotted line in FIG. 3 corresponds to the frame memory unit 23 shown in FIG. 3, so that the embodiment shown in FIG. 3 can be configured to share this portion.

The subtracting unit 34 performs subtraction between the signals S15 and S9 to generate a difference signal S20 between frames.

The MC interpolation control unit 35 uses the motion vector MV, the image information PI, the inter-frame difference signal component of the motion compensation of the signal S3, and the inter-frame difference signal S20, the interpolated motion vector IMV, the motion adaptive coefficient km, 1-km is generated. It is determined from the image information PI whether the motion compensation scan conversion is interlace to non-interlace scan conversion or frame number conversion. In the former case, an interpolation motion vector necessary for generating an interpolation signal for motion compensation shown in FIG . 1 and in the latter case is generated with the characteristics shown in FIG. That is, IMV = 0 is generated as an interpolated motion vector for an I picture. On the other hand, the P picture is generated with the magnitude of the difference signal component between frames for motion compensation. That is, the absolute value sum Σ | S3 | of the signal S3 is measured for each block (8 pixels × 8 lines). When this is less than the threshold value Th, it is determined that the accuracy of the motion vector is high, and a corresponding interpolated motion vector is generated based on the motion vector MV. On the other hand, it is determined that the accuracy of the motion vector is poor above the threshold Th, and IMV = 0 is generated as the interpolated motion vector. The motion adaptation coefficient km is a magnitude of Σ | S3 | in the P picture and a magnitude of the absolute value | S20 | of the difference signal between frames in the I picture, and generates a coefficient value that varies in the range of 0 to 1.

The MCIP unit 29 extracts the scanning line signal specified by the interpolation motion vector from the signals S9 and S15, and generates an interpolation signal S18 for motion compensation using the average value of both signals. Note that when the accuracy of the motion vector is poor in the I picture and the P picture, IMV = 0, so that the signal S18 is equivalent to the interpolation signal in the still mode of the prior art.

The intra-field interpolation unit 31 performs intra-field arithmetic processing based on the signal S16, and generates a moving image interpolation signal S19 corresponding to the moving image mode using the same field signal as in the conventional technique.

In the coefficient weighting units 32-1 and 32-2, the motion adaptive coefficient 1− is applied to the signals S18 and S19, respectively.
km, km coefficient values are weighted, and both signals are added by the adder 33 to obtain an interpolation signal sequence S17.
Is generated.

It should be noted that the motion adaptive coefficient km in the second and third configuration examples is preferably realized with a characteristic in which the coefficient value changes stepwise as shown in the figure, but depending on the case, it may be a binary characteristic of 0 or 1. It can also be realized.

Next, a configuration example of the post-processing unit 26 will be described with reference to FIG. The main signal series S16 and the interpolation signal series S17 are input to the image format conversion units 36-1 and 36-2, and an image format conversion process from the encoded image signal series to the format of the image signal in the original scanning form is performed. Then, the image series VMS and VIP composed of the luminance signal and the two color difference signals are decoded. The horizontal time compression units 37-1 and 37-2 use image information PI to perform signal processing for compressing the time axis by half in the horizontal direction in interlaced to non-interlaced scanning conversion, for example, in standard mode in frame number conversion. In the 30 to 60 conversion, signal processing for compressing the time axis by half in the time direction is performed. Then, the time division multiplexing unit 38 multiplexes both signals in a time division manner, and generates an image sequence VP in the form of non-interlaced scanning. The RGB conversion unit 39 performs a predetermined matrix calculation process and converts the image sequence VR into the three primary colors RGB system. Then, the DA converter 40 performs conversion into an analog signal, and decodes the non-interlaced scanning image signal VSD.

The other block units in FIG. 3 may be configured in the same manner as the conventional MPEG 1 and 2 video encoding encoding and decoding units, and thus the description thereof is omitted.

As described above, according to this embodiment, it is possible to realize an image signal encoding / decoding apparatus that receives high-quality and high-definition images without interlace interference or judder interference, and is remarkable in improving image quality. Effects can be achieved.

Next, a second embodiment of the present invention will be described with reference to the overall block diagram shown in FIG. In this embodiment, the decoding unit selects a motion vector suitable for scan conversion processing for motion compensation, and is suitable for decoding a non-interlaced image signal using the selected motion vector. It is. In the present embodiment, a case of video coding conforming to MPEG 1 and 2 combining motion compensation prediction coding between frames and DCT orthogonal transform coding will be described as an example.

In the encoding unit shown in FIG. 5A, the image signal VS is digitized by the preprocessing unit 1 and subjected to predetermined image format conversion to generate an encoded image signal sequence S1.
That is, a macroblock signal sequence composed of six blocks (four luminance signal blocks and two color difference signal blocks, block size is 8 pixels × 8 lines) is generated.

The switches 4 and 11 are connected to the terminal a when the coding mode for performing DCT coding within a frame is I picture, and to the terminal b when the coding mode for performing motion compensation interframe prediction coding and DCT coding is P picture. The DCT unit 5 receives an encoded image signal sequence S1 of the original image for an I picture and an interframe prediction error signal sequence S2 for a P picture as a signal S3. The subtraction unit 3 generates an inter-frame prediction error signal sequence S2 by a subtraction process between the inter-frame motion compensation prediction signal sequence S10 and the encoded image signal sequence S1 generated by the MC signal generation unit 13.

The DCT unit 5 uses 8 blocks × 8 columns of D in units of each block (8 pixels × 8 lines) of the signal S3.
A matrix operation with the CT conversion matrix is performed to generate a conversion coefficient signal series S4. The quantization unit 6
The transform coefficient is quantized to generate a coefficient quantized signal sequence S5. In order to secure a predetermined transmission bit rate, the quantization process is performed by changing the characteristics according to the buffer retention amount of the buffer unit 16.

The inverse quantization unit 8 decodes the transform coefficient signal sequence S6 by the inverse quantization process, and the IDCT unit 9
Matrix operation with a DCT transformation inverse matrix of rows × 8 columns is performed. Then, the signal S7 of the encoded image signal sequence is decoded during the e-picture, and the inter-frame prediction error signal sequence is decoded during the P-picture. Adder 1
0 performs addition processing with the inter-frame motion compensation prediction signal sequence S10, and decodes the encoded image signal sequence in the P picture. The switch 11 is connected to the terminal a for e-picture, and to the terminal b for P-picture. The decoded encoded image signal sequence S8 of the output is input to the frame memory unit 12, and the output is a signal delayed by one frame period. Get S9.

The motion vector detection unit 41 extracts a motion vector MV for one frame period by a block matching method or the like. Unlike the first embodiment, the motion compensation scan conversion process in the decoding unit includes an inconvenient motion vector. The MC signal generation unit 13 performs a motion compensation process on the signal S9 with the motion vector MV, and generates an inter-frame motion compensated prediction signal sequence S10.

The VLC units 7 and 14 perform a predetermined variable length encoding process on the coefficient quantized signal sequence S5 and the motion vector MV, and the encoded image data sequence S11 and the encoded motion vector sequence S1.
2 is generated. The multiplexing unit 15 performs processing for multiplexing the signal series S11, S12, and various data such as the encoding parameters of the image information PI into a predetermined time series, and outputs the encoded data series S13 to the buffer unit 16. input.
Then, it reads out from the buffer unit 16 at a constant bit rate to generate an encoded bit stream signal PCD. The signal PCD is transmitted after being subjected to predetermined channel coding such as code error countermeasures and digital modulation according to the transmission medium, although not explicitly shown in the drawing.

In the decoding unit shown in FIG. 5B, the encoded bit stream signal PCD decoded by channel decoding is input to the buffer unit 17. The encoded data sequence S13 read from the buffer unit is separated by the DMPX unit 18 into an encoded image data sequence S11, an encoded motion vector sequence S12, and various data of the image information PI.

The IVLC units 19 and 20 perform variable length code decoding processing, and perform coefficient quantization signal sequence S5.
Then, the motion vectors MV are respectively decoded.

The inverse quantization unit 8 decodes the transform coefficient signal sequence S4 by the inverse quantization process, and the IDCT unit performs a matrix operation with a DCT transform inverse matrix of 8 rows × 8 columns. Then, the signal S3 corresponding to the encoded image signal sequence for the I picture and the inter-frame prediction error signal sequence for the P picture are decoded. Further, the adding unit 21 adds the inter-frame motion compensated prediction signal sequence S10 generated by the MC signal generating unit 24, and generates an encoded image signal sequence S14 for P picture as an output thereof. The switch 22 is connected to the terminal a for e-picture and to the terminal b for P-picture, and a decoded encoded image signal sequence S15 is obtained at the output.

The frame memory unit 23 generates a signal S9 obtained by delaying the input signal by one frame period.
Then, the MC signal generation unit 24 performs a motion compensation process on the signal S9 using the motion vector MV, and generates an inter-frame motion compensation prediction signal sequence S10.

The MV selection unit 42 performs a process of selecting a motion vector MV that matches a scan conversion process for motion compensation in accordance with the image information PI. That is, when the image signal is interlaced scanning, the motion vectors V ₀ , V ₁ , and V ₂ as shown in the scanning line interpolation of FIG. 1 are used, and when the image signal is non-interlaced scanning, the motion as shown in the scanning line interpolation of FIG. A region for selecting a vector is generated as H, and a region for which a vector is not selected is generated as a selected region signal MCM.

The MC interpolation unit 43 selects the selection motion vector MV ′, the selection region signal MCM, and the image information PI.
Then, the motion compensation scan conversion process shown in FIGS. 1 and 2 is performed on the inter-frame prediction error signal sequence of the signal S3, and the main signal sequence S16 and the interpolation signal sequence S17 are generated. This configuration and operation will be described in detail later.

The post-processing unit 26 performs image format conversion processing, interlace to non-interlace scan conversion and frame number conversion processing using the main signal sequence S16 and the interpolation signal sequence S17, and conversion to analog signals. The image signal VSD of the race scan is decoded.
This signal is displayed on a non-interlaced scanning image display unit (not shown in the drawing) to receive a high-quality and high-definition image.

FIG. 11 is a configuration example diagram of the MC interpolation unit 43. Field memory units 27-1 and 27-
2 is a signal delay for one field period, and a portion surrounded by a dotted line in the figure corresponds to a frame memory, and a main signal series S16 and a signal S1 used for a scan conversion process for motion compensation.
5 and a signal S9 obtained by delaying this signal by one frame period.

The motion detection unit 44 extracts a difference component between frames by subtraction processing between the signals S15 and S9,
Based on this difference component, motion information MI between frames (0 when stationary) is generated.

The MC interpolation control unit 45 generates an interpolation motion vector IMV and motion adaptive coefficients km and 1-km necessary for generating a motion compensation interpolation signal. A schematic diagram of this operation is shown in FIG. Figure (a)
In the first characteristic, the interpolated motion vector is shown in FIG. 1 for interlaced to non-interlaced scan conversion and in FIG. 2 for frame number conversion based on the selected motion vector MV ′ when the selected region signal MCM is H. An IMV necessary for generating the motion compensated interpolation signal shown is generated. On the other hand, when the selection area signal MCM is L, 0 is generated as the interpolation motion vector. The motion adaptive coefficient km generates a coefficient value having a characteristic that changes from 0 to 1 according to the level of motion information MI between frames when km = 0 when the selection region signal MCM is H, and when L. Therefore, with this characteristic,
In the region where the selection region signal MCM is H, the interpolation signal is generated by the motion compensation scan conversion processing, and in the region L, the interpolation signal is generated by the motion adaptation processing similar to the conventional technique.

On the other hand, in the second characteristic of FIG. 5B, the interpolated motion vector is that the selected region signal MCM is H and the prediction error signal of the signal S3, that is, the absolute value sum Σ of the motion compensation inter-frame difference signal component Σ.
When | S ₃ | (for example, a block unit of 8 pixels × 8 lines) is less than the threshold value Th, it is generated based on the selected motion vector MV ′. In other cases, 0 is generated for the interpolation motion vector. The motion adaptive coefficient km is a coefficient value of a characteristic that changes from 0 to 1 corresponding to the value of Σ | S ₃ | in the region where the selection region signal is H, and between frames in the region where the selection region signal is L. The coefficient value of the characteristic which changes from 0 to 1 corresponding to the motion information MI is generated.

Returning to FIG. 11, the MCIP unit 29 extracts the scanning line signals of the signal S9 and the signal S15 specified by the interpolation motion vector IMV, and uses the average value of these signals for the motion compensation interpolation signal S1.
8 is generated. When IMV = 0, the signal S18 generates an interpolation signal equivalent to the still mode in the prior art.

The intra-field interpolation unit 31 performs an intra-field calculation process based on the signal S16, and generates a moving image interpolation signal S19 similar to that in the moving image mode of the prior art using the same field signal.

The coefficient weighting unit 32-1 weights the signal S18 with the motion adaptive coefficient 1-km, and the coefficient weighting unit 32-2 weights the signal S19 with the motion adaptive coefficient km. Then, both signals are added by the adder 33 to generate an interpolation signal series S17.

A portion surrounded by a dotted line in the figure corresponds to the frame memory unit 23 in FIG. For this reason, in the embodiment of FIG. 10, it is also possible to configure this part in a shared form.

As described above, according to the present embodiment, an image signal encoding / decoding device that can be received as a high-quality and high-definition image without interlace interference and judder interference can be obtained, which is remarkable for image quality improvement. The effect is obtained. The configuration of the decoding unit shown in the present embodiment can also be used as the decoding unit in the first embodiment.

In the present invention, the configuration of the decoding unit can be realized in various forms in addition to the first and second embodiments.

  FIG. 13 is an overall block configuration diagram of the third embodiment of the decoding unit.

The encoded bit stream signal PCD subjected to channel decoding is input to the buffer unit 17. Then, the encoded data sequence S13 read from the buffer unit is input to the video decoding unit 46. The video decoding unit corresponds to a portion surrounded by a dotted line in FIGS. 3 and 10, and performs the above-described video decoding process. The decoded encoded image signal sequence S15, the motion vector MV, and the P picture of the signal S3 An inter-frame prediction error signal sequence and image information PI are output.

The image format conversion unit 47 performs an image format conversion process on the encoded image signal sequence, and generates an image signal sequence VFC in the original scanning form.

The MC interpolation unit 48 performs the motion compensation scan conversion process shown in FIG. 1 for interlaced to non-interlaced scan conversion and FIG. 2 for frame number conversion, and generates a main signal sequence S16 and an interpolation signal sequence S17. That is, with the same configuration as in FIGS. 5 to 8, the motion adaptive coefficient km, 1−1 generated by the interpolated motion vector IMV generated based on the motion vector MV and the inter-frame difference signal of motion compensation detected from the signal S3. The interpolation signal sequence S17 is created using km.

The non-interlaced sequence conversion unit 49 divides the main signal sequence S16 and the interpolated signal sequence S17 by 1/2 in the horizontal direction in the case of interlaced to non-interlaced scan conversion based on the image information PI.
In the case of the process of compressing the time axis, for example, in the case of 30 to 60 conversion in the standard mode of frame number conversion, the process of compressing the time axis in half in the time direction is performed.
Then, both compressed signals are multiplexed in time series to generate a non-interlaced scanning signal,
The three primary colors RGB conversion and analog conversion are performed to restore the non-interlaced scanning image signal VSD.

The present embodiment is suitable for video encoding with a motion vector in a form suitable for scan conversion processing for motion compensation in motion compensation predictive coding in the encoder, and there is no interlace interference or judder interference. A decoding device that receives high-quality and high-definition images can be realized.

  FIG. 14 is an overall block configuration diagram of the fourth embodiment of the decoding unit.

The encoded bit stream signal PCD subjected to channel decoding is input to the buffer unit 17. Then, the encoded data sequence S13 read from the buffer unit is input to the video decoding unit 46. This corresponds to a portion surrounded by a dotted line in FIG. 3 and FIG. 10, and the above-described video decoding process is performed to obtain a decoded encoded image signal sequence S15, a motion vector MV, and a signal S3.
Inter-frame prediction error signal sequence (corresponding to inter-frame difference signal for motion compensation)
And image information PI are output.

The MV selector 42 performs a process of selecting a motion vector MV that matches the scan conversion process for motion compensation in accordance with the image information PI. That is, when the image signal is interlaced scanning, the motion vector shown in the scanning line interpolation of FIG. 1 is generated as the selected motion vector MV ′. Further, a selection region signal MCM of H is generated for the region where the motion vector is selected, and L is selected for the region where the motion vector is not selected.

The image format conversion unit 47 performs an image format conversion process for converting an encoded image signal sequence into a signal sequence in the original scanning form, and generates an image signal sequence VFC.

The MC interpolator 50 performs the signal processing shown in FIG. 1 for motion compensation scan conversion processing, that is, interlace to non-interlace scan conversion, and FIG. 2 for frame number conversion, and performs the main signal sequence S16 and the interpolation signal sequence S17. And generate This can be realized by the configuration and characteristics shown in FIGS. Then, the interpolation motion vector IMV and the motion adaptive coefficient km, 1-km
Is used to generate an interpolation signal sequence S17.

The non-interlaced sequence conversion unit 49 performs time axis compression processing of the main signal sequence S16 and the interpolation signal sequence S17. That is, according to the image information PI, in the case of interlaced to non-interlaced scan conversion, 1/2 in the horizontal direction, for example, 30 to 6 in the standard mode of frame number conversion.
In 0 conversion, 1/2 compression is performed in the time direction. Then, both the compressed signals are multiplexed in time series to generate a non-interlaced scanning signal, the three primary colors RGB conversion and analog conversion are performed, and the non-interlaced scanning image signal VSD is decoded.

According to the present embodiment, it is possible to realize a decoding apparatus that receives high-quality and high-definition images without interlace interference and judder interference.

In the description of the embodiment described above, the combination of interframe motion compensated prediction coding and DCT transform coding has been described as an example of video coding. However, the present invention is not limited to this, and various forms of motion compensation are possible. It is clear that the present invention can also be applied to coding by a combination of predictive coding and orthogonal transform coding.

FIG. 3 is a schematic diagram illustrating the principle of predictive coding for motion compensation and scanning line interpolation. FIG. 3 is a schematic diagram illustrating the principle of predictive coding for motion compensation and scanning line interpolation. The whole block block diagram of the 1st Example of this invention. Schematic of encoding mode. The 1st structural example figure of MC interpolation part. The 2nd structural example figure of MC interpolation part. The operation | movement schematic diagram of MC interpolation control part. The 3rd structural example figure of MC interpolation part. The structural example figure of a post-processing part. The whole block block diagram of the 2nd Example of this invention. FIG. 10 is a configuration example diagram of an MC interpolation unit in the second embodiment. The characteristic schematic diagram of MC interpolation control part in the 2nd example. The whole block block diagram of the 3rd Example of the decoding part of this invention. The whole block block diagram of the 4th Example of the decoding part of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Pre-processing part, 2, 41 ... Motion vector detection part, 3,34 ... Subtraction part, 4, 11 , 22 ... Switch, 5 ... DCT part, 6 ... Quantization part, 7, 14 ... VLC part, 8 ... Inverse quantization unit, 9 ... IDCT unit, 10, 21, 33 ... addition unit, 12, 23 ... frame memory unit, 13, 24 ... MC signal generation unit, 15 ... MPX unit, 16, 17 ... buffer unit, 18 ... DMPX unit, 19, 20 ... IVLC unit, 25, 43, 48, 50 ... MC interpolation unit, 26 ... post-processing unit, 27 ... field memory unit, 28 ... interpolation MV generation unit, 29 ... MCIP unit, 30, 35, 45 ... MC interpolation control unit, 31 ... intra-field interpolation unit, 32 ... coefficient weighting unit, 36, 47 ... image format conversion unit, 37 ... horizontal time compression unit, 38 ... time division multiplexing unit, 39 ... RGB conversion unit, 40 ... DA converter, 42 ... MV select , 44 ... motion detection unit, 46 ... video decoder, 49 ... non-interlaced stream conversion unit.

Claims (12)

  An image signal generation method for inputting an encoded image signal by motion prediction encoding, decoding the image signal, and converting the number of frames of the decoded image signal into an image signal,
  Interpolation that is a newly generated frame that is positioned temporally after one frame of a plurality of frames of the image signal before conversion of the number of frames and positioned temporally before one other frame A motion vector that passes through pixels in which an interpolation signal of a generation target frame is generated and connects pixels of a plurality of frames of the image signal before conversion of the number of frames is selected from the encoded image signal,
  The average value of the pixels indicated by the start and end points of the selected motion vector in a plurality of frames of the image signal before the conversion of the number of frames is set as the pixel value of the pixel on which the interpolation signal of the interpolation generation target frame is generated. To generate the interpolation generation target frame and convert the number of frames of the image signal.   The image signal generation method according to claim 1,
  In the conversion of the number of frames of the image signal, one interpolation generation target frame located between two frames of a plurality of frames of the image signal before the conversion of the number of frames is generated, and the two frames An image signal generation method characterized in that the number of frames of an image signal is doubled by setting the one interpolation generation target frame as a frame of an image signal after frame number conversion.   The image signal generation method according to claim 2,
  An image signal generation method, wherein the number of frames of an image signal before frame number conversion is 30 frames per second, and the number of frames of an image signal after frame number conversion is 60 frames per second.   An image signal generation method for inputting an encoded image signal by motion prediction encoding, decoding the image signal, and converting the number of frames of the decoded image signal into an image signal,
  The image signal before the conversion of the number of frames includes the first frame, the second frame, and the third frame,
  Generation of an interpolation signal of a first interpolation generation target frame, which is a newly generated frame that is positioned temporally behind the first frame and positioned temporally before the second frame Is selected from the encoded image signal through the pixels to be performed and connecting the pixels of the first frame and the second frame,
  By setting the average value of the pixels indicated by the start point and the end point of the selected motion vector as the pixel value of the pixel for which the interpolation signal of the first interpolation generation target frame is generated, the first interpolation generation target frame is Generate
  Generation of an interpolation signal of a second interpolation generation target frame, which is a newly generated frame that is positioned behind the second frame and positioned before the third frame. Is selected from the encoded image signal through the pixels to be performed and connecting the pixels of the second frame and the third frame,
  The average value of the pixels indicated by the start point and the end point of the selected motion vector is used as the pixel value of the pixel for which the interpolation signal of the second interpolation generation target frame is generated, thereby obtaining the second interpolation generation target frame. Generate
  A method for generating an image signal, wherein the number of frames of the image signal is converted.   In the image signal generation method described in Claim 4,
  In the conversion of the number of frames of the image signal, a plurality of first interpolation generation target frames that are positioned temporally after the first frame and temporally before the second frame are generated. , Generating a plurality of second interpolation generation target frames that are located behind the second frame and located before the third frame, and the first frame and the first frame 3. The image signal generation method according to claim 3, wherein the third frame, the plurality of generated first interpolation generation target frames, and the plurality of generated second interpolation generation target frames are used as the frames of the image signal after frame number conversion.   In the image signal generation method according to claim 5,
  An image signal generation method, wherein the number of frames of an image signal before frame number conversion is 24 frames per second, and the number of frames of an image signal after frame number conversion is 60 frames per second.   An image signal generation device that receives an encoded image signal by motion prediction encoding, decodes the image signal, and converts the number of frames of the decoded image signal into an image signal,
  Interpolation that is a newly generated frame that is positioned temporally after one frame of a plurality of frames of the image signal before conversion of the number of frames and positioned temporally before one other frame A motion vector selection unit that selects, from the encoded image signal, a motion vector that passes through pixels in which an interpolation signal of a generation target frame is generated and that connects pixels of a plurality of frames of the image signal before conversion of the number of frames. When,
  The average value of the pixels indicated by the start and end points of the selected motion vector in a plurality of frames of the image signal before the conversion of the number of frames is set as the pixel value of the pixel on which the interpolation signal of the interpolation generation target frame is generated. An interpolation signal generation unit that generates the interpolation generation target frame and converts the number of frames of the image signal;
An image signal generation device comprising:   The image signal generation device according to claim 7,
  The conversion of the number of frames of the image signal by the interpolation signal generation unit generates one interpolation generation target frame located between two frames of the plurality of frames of the image signal before the conversion of the number of frames, An image signal generating apparatus characterized in that the number of frames of an image signal is doubled by using the two frames and the one interpolation generation target frame as a frame of an image signal after frame number conversion.   In the image signal generation device according to claim 8,
  An image signal generation apparatus, wherein the number of frames of an image signal before frame number conversion is 30 frames per second, and the number of frames of an image signal after frame number conversion is 60 frames per second.   An image signal generation device that receives an encoded image signal by motion prediction encoding, decodes the image signal, and converts the number of frames of the decoded image signal into an image signal,
  An interpolation signal generator for generating an interpolation frame for conversion of the number of frames;
  A motion vector selection unit that selects a motion vector used for generating an interpolation frame of the interpolation signal generation unit from the encoded image signal;
  The image signal before the conversion of the number of frames includes the first frame, the second frame, and the third frame,
  The first interpolation generation is a newly generated frame in which the motion vector selection unit is positioned temporally after the first frame and temporally before the second frame. A motion vector that passes through the pixel for which the interpolation signal of the target frame is generated and connects the pixel of the first frame and the second frame is selected from the encoded image signal;
  The interpolation signal generation unit sets the average value of the pixels indicated by the start point and the end point of the selected motion vector as the pixel value of the pixel for which the interpolation signal of the first interpolation generation target frame is generated. Generate 1 interpolation generation target frame,
  The second interpolation generation is a newly generated frame in which the motion vector selection unit is positioned temporally behind the second frame and temporally before the third frame. A motion vector that passes through the pixel for which the interpolation signal of the target frame is generated and connects the pixel of the second frame and the third frame is selected from the encoded image signal,
  The interpolation signal generation unit sets the average value of the pixels indicated by the start point and the end point of the selected motion vector as the pixel value of the pixel for which the interpolation signal of the second interpolation generation target frame is generated. 2 interpolation generation target frame is generated,
  An image signal generation device that converts the number of frames of an image signal.   The image signal generation device according to claim 10,
  In the conversion of the number of frames of the image signal by the interpolation signal generation unit, the first interpolation generation that is positioned temporally after the first frame and temporally before the second frame Generating a plurality of target frames, generating a plurality of second interpolation generation target frames positioned temporally after the second frame and positioned temporally before the third frame, One frame, the third frame, the plurality of generated first interpolation generation target frames, and the plurality of generated second interpolation generation target frames are frames of the image signal after frame number conversion. Image signal generator.   The image signal generation device according to claim 11,
  An image signal generating apparatus, wherein the number of frames of an image signal before frame number conversion is 24 frames per second, and the number of frames of an image signal after frame number conversion is 60 frames per second.

Images