JP3189258B2 - Image signal encoding method and image signal encoding device, image signal decoding method and image signal decoding device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for recording a moving image signal on a recording medium such as a magneto-optical disk or magnetic tape.
You can play it back and display it on a display, or use it for video conferencing systems, video phone systems,
An image signal encoding method and an image signal encoding device suitable for use in a case where a moving image signal is transmitted from a transmission side to a reception side via a transmission path and the reception side receives and displays the image signal, the image signal encoding apparatus, and the image signal The present invention relates to a decoding method and an image signal decoding device, and an image signal recording medium.

[0002]

2. Description of the Related Art For example, in a system for transmitting a moving image signal to a remote place, such as a video conference system and a video telephone system, in order to efficiently use a transmission path, line correlation or inter-frame correlation of a video signal is required. Is used to compress and encode an image signal.

[0003] When line correlation is used, an image signal can be compressed by, for example, DCT (discrete cosine transform) processing.

[0004] When the inter-frame correlation is used, it is possible to further compress and encode the image signal. For example, as shown in FIG. 12, when frame images PC1, PC2, and PC3 are generated at times t1, t2, and t3, respectively, the difference between the image signals of frame images PC1 and PC2 is calculated to generate PC12. Further, a difference between the frame images PC2 and PC3 is calculated to generate a PC23. Normally, the images of frames temporally adjacent to each other do not have such a large change. Therefore, when the difference between them is calculated, the difference signal has a small value. Therefore, if the difference signal is encoded, the code amount can be compressed.

[0005] However, if only the difference signal is transmitted, the original image cannot be restored. Therefore, the image of each frame is one of three types of pictures, i-pictures, P-pictures, and B-pictures,
The image signal is compressed and encoded.

That is, as shown in FIG. 13, for example, image signals of 17 frames from F1 to F17 are set as a group of pictures and are set as one unit of processing. And
The image signal of the first frame F1 is coded as an I picture, the second frame F2 is a B picture, and the third frame F3 is a P picture.
Process each. Hereinafter, the fourth and subsequent frames F4
Processes F17 to F17 are alternately processed as B pictures or P pictures.

As an I-picture image signal, an image signal for one frame is transmitted as it is. On the other hand, as an image signal of a P picture, basically, as shown in FIG.
As shown in FIG. 3 (A), a difference from an image signal of an I picture or a P picture which temporally precedes it is transmitted. Further, as an image signal of a B picture, basically, as shown in FIG. 13B, a difference from an average value of both a temporally preceding frame and a succeeding frame is obtained, and the difference is encoded. Become

FIG. 14 shows the principle of a method for encoding a moving picture signal in this manner. As shown in the figure, since the first frame F1 is processed as an I picture, it is transmitted as it is to the transmission path as transmission data F1X (intra-picture encoding). On the other hand, since the second frame F2 is processed as a B picture, the temporally preceding frame F1 and the temporally succeeding frame F3
Of the transmission data F is calculated.
Transmitted as 2X.

[0009] However, there are four types of processing as a B picture in more detail. The first processing is to transmit the data of the original frame F2 as it is as the transmission data F2X (SP1) (intra coding), and is the same processing as in the case of the I picture. The second process is to calculate the difference from the temporally later frame F3 and transmit the difference (SP2) (backward prediction coding). The third process is to transmit a difference (SP3) from the temporally preceding frame F1 (forward prediction coding). Further, the fourth processing is to generate a difference (SP4) between the average value of the temporally preceding frame F1 and the average value of the following frame F3, and transmit this as transmission data F2X (bidirectional predictive coding). .

[0010] Of these four methods, the method that minimizes transmission data is adopted.

When transmitting the difference data, a motion vector x1 (a motion vector between frames F1 and F2) between the image of the frame (prediction image) for which the difference is to be calculated (in the case of forward prediction) Or x2 (frame F
A motion vector between 3 and F2) (for backward prediction) or both x1 and x2 (for bidirectional prediction) are transmitted along with the difference data.

In the frame F3 of the P picture, a difference signal (SP3) from this frame and a motion vector x3 are calculated using the frame F1 temporally preceding as a predicted image, and this is transmitted as transmission data F3X. (Forward prediction coding). Alternatively, the data of the original frame F3 is directly transmitted as transmission data F3X (S3).
P1) (Intra coding). Which method is used, as in the case of the B picture, is selected so that the amount of transmitted data is smaller.

FIG. 15 shows an example of the configuration of an apparatus that encodes and transmits a moving image signal based on the principle described above and decodes it. The encoding device 1 encodes an input video signal and transmits the encoded video signal to a recording medium 3 as a transmission path. The decoding device 2 reproduces the signal recorded on the recording medium 3, and decodes and outputs the signal.

In the coding apparatus 1, the input video signal is input to a pre-processing circuit 11, where a luminance signal and a chrominance signal (a color difference signal in this example) are separated, and an A / D converter 12 , 13 are A / D converted. The video signal converted into a digital signal by A / D conversion by the A / D converters 12 and 13 is supplied to the frame memory 14 and stored therein. The frame memory 14 stores the luminance signal in the luminance signal frame memory 15 and stores the color difference signal in the color difference signal frame memory 16, respectively.

The format conversion circuit 17 converts the signal of the frame format stored in the frame memory 14 into
Convert to block format signal. That is, FIG.
As shown in the figure, the video signal stored in the frame memory 14 is data in a frame format in which V lines are collected for each H dot line. The format conversion circuit 17 converts the signal of one frame into one.
It is divided into M slices in units of 6 lines. Then, each slice is divided into M macroblocks. Each macro block is composed of a luminance signal corresponding to 16 × 16 pixels (dots), and this luminance signal is further divided into blocks Y [1] in units of 8 × 8 dots.
To Y [4]. The 16 × 16 dot luminance signal includes an 8 × 8 dot Cb signal and an 8 × 16 dot Cb signal.
An 8-dot Cr signal is supported.

The data converted into the block format is supplied from the format conversion circuit 17 to the encoder 18, where the data is encoded. The details will be described later with reference to FIG.

The signal encoded by the encoder 18 is output to a transmission path as a bit stream and recorded on the recording medium 3, for example.

The data reproduced from the recording medium 3 is supplied to a decoder 31 of the decoding device 2 and decoded.
The details of the decoder 31 will be described later with reference to FIG.

The data decoded by the decoder 31 is input to a format conversion circuit 32 and converted from a block format to a frame format. The luminance signal in the frame format is supplied to and stored in the luminance signal frame memory 34 of the frame memory 33, and the color difference signal is supplied to and stored in the color difference signal frame memory 35. The luminance signal and the color difference signal read from the luminance signal frame memory 34 and the color difference signal frame memory 35 are D / A converters 36 and 37 respectively.
The signal is A-converted, supplied to the post-processing circuit 38, and synthesized.
Then, the data is output and displayed on a display such as a CRT (not shown).

Next, a configuration example of the encoder 18 will be described with reference to FIG.

Image data to be encoded is input to the motion vector detection circuit 50 in macroblock units.
The motion vector detection circuit 50 converts the image data of each frame into I
Process as a picture, P picture, or B picture. Images of each frame input sequentially are
Which of I, P, and B is processed as a picture is determined in advance (for example, as shown in FIG. 13, a group of pictures composed of frames F1 to F17 is divided into I, B, P, and B, P,..., B, P).

Image data of a frame (for example, frame F1) to be processed as an I picture is obtained from a motion vector detecting circuit 50 by a front original image section 51a of a frame memory 51.
The image data of a frame (for example, frame F2) which is transferred and stored in the
The image data of the frame (for example, frame F3) transferred and stored in 1b and processed as a P picture is transferred and stored in the rear original image unit 51c.

At the next timing, B
When an image of a frame to be processed as a picture (frame F4) or a P picture (frame F5) is input, the image data of the first P picture (frame F3) stored in the rear original image section 51c until then is The image is transferred to the front original image section 51a and the next B picture (frame F
The image data of 4) is stored (overwritten) in the original image section 51b.
Then, the image data of the next P picture (frame F5) is stored (overwritten) in the rear original image section 51c. Such operations are sequentially repeated.

The signal of each picture stored in the frame memory 51 is read therefrom, and the prediction mode switching circuit 52 performs frame prediction mode processing or field prediction mode processing. Further, under the control of the prediction determination circuit 54, the calculation unit 53 performs calculation of intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction. Which of these processes is to be performed is determined according to the prediction error signal (the difference between the reference image to be processed and the predicted image corresponding to the reference image). Therefore, the motion vector detection circuit 50 generates a sum of absolute values (or a sum of squares) of the prediction error signal used for this determination.

Here, the frame prediction mode and the field prediction mode in the prediction mode switching circuit 52 will be described.

When the frame prediction mode is set, the prediction mode switching circuit 52 controls the four luminance blocks Y supplied from the motion vector detection circuit 50.
[1] to Y [4] are output as they are to the operation unit 53 in the subsequent stage. That is, in this case, as shown in FIG. 18A, the data of the line of the odd field and the data of the line of the even field are mixed in each luminance block. In this frame prediction mode, prediction is performed in units of four luminance blocks (macroblocks), and one motion vector corresponds to four luminance blocks.

On the other hand, the prediction mode switching circuit 5
FIG. 18 (A) shows the field prediction mode 2 in the field prediction mode.
As shown in FIG. 18B, a signal input from the motion vector detection circuit 50 in the configuration shown in FIG. 18 is used to convert the luminance blocks Y [1] and Y [2] out of the four luminance blocks into, for example, an odd field. Only by the dots of the line
The other two luminance blocks Y [3] and Y [4] are constituted by the data of the lines of the even field, and
Output to 3. In this case, one motion vector corresponds to two luminance blocks Y [1] and Y [2], and the other two luminance blocks Y [3] and Y [4].
Is associated with one other motion vector.

The motion vector detection circuit 50 outputs to the prediction mode switching circuit 52 the sum of the absolute values of the prediction errors in the frame prediction mode and the sum of the absolute values of the prediction errors in the field prediction mode. Prediction mode switching circuit 5
2 compares the absolute value sum of the prediction errors in the frame prediction mode and the field prediction mode, performs processing corresponding to the prediction mode having a smaller value, and outputs the data to the arithmetic unit 53.

However, such processing is actually performed by the motion vector detection circuit 50. That is, the motion vector detection circuit 50 outputs a signal having a configuration corresponding to the determined mode to the prediction mode switching circuit 52, and the prediction mode switching circuit 52 outputs the signal as it is to the subsequent operation unit 53.

In the frame prediction mode, the color difference signal is supplied to the arithmetic unit 53 in a state where the data of the lines of the odd fields and the data of the lines of the even fields are mixed as shown in FIG. Is done. In the case of the field prediction mode, as shown in FIG. 18B, the upper half (4 lines) of each of the chrominance blocks Cb and Cr corresponds to the odd field corresponding to the luminance block Y [1] and Y [2]. The lower half (4 lines) is a color difference signal of an even field corresponding to the luminance blocks Y [3] and Y [4].

In addition, the motion vector detecting circuit 50 causes the prediction determining circuit 54 to perform intra-picture prediction,
A sum of absolute values of prediction errors for determining whether to perform forward prediction, backward prediction, or bidirectional prediction is generated.

That is, as the sum of the absolute values of the prediction errors of the intra-picture prediction, the sum of the macroblock signals Aij of the reference picture ΣAij
Of the absolute value | ＡAij | of the macroblock signal Aij and the sum Σ | Aij | The sum と し て | Aij− of the absolute value | Aij−Bij | of the difference Aij−Bij | between the signal Aij of the macroblock of the reference image and the signal Bij of the macroblock of the predicted image is used as the absolute value sum of the prediction error of the forward prediction. Bij |
Ask for. In addition, the absolute value sum of the prediction error between the backward prediction and the bidirectional prediction is obtained in the same manner as in the forward prediction (by changing the predicted image to a different predicted image from that in the forward prediction).

The sum of these absolute values is supplied to the prediction judgment circuit 54. The prediction determination circuit 54 selects the smallest absolute value sum of the prediction errors of the forward prediction, the backward prediction, and the bidirectional prediction as the sum of the absolute values of the prediction errors of the inter prediction. Further, the sum of the absolute values of the prediction errors of the inter prediction and the sum of the absolute values of the prediction errors of the intra prediction are compared,
The smaller one is selected, and the mode corresponding to the selected sum of absolute values is selected as the prediction mode. That is, if the sum of absolute values of the prediction errors of the intra prediction is smaller, the intra prediction mode is set. If the sum of the absolute values of the prediction errors in the inter prediction is smaller, the mode in which the corresponding sum of the absolute values is the smallest among the forward prediction, backward prediction, and bidirectional prediction modes is set.

As described above, the motion vector detecting circuit 50
Supplies the macroblock signal of the reference image to the arithmetic unit 53 via the prediction mode switching circuit 52 in a configuration corresponding to the mode selected by the prediction mode switching circuit 52 among the frame or field prediction modes. Of the four prediction modes, the prediction determination circuit 54
, A motion vector between the predicted image corresponding to the prediction mode selected by the above and the reference image is detected, and the variable length coding circuit 5
8 and output to the motion compensation circuit 64. As described above, as the motion vector, the motion vector having the smallest absolute value sum of the corresponding prediction errors is selected.

When the motion vector detection circuit 50 reads out the I-picture image data from the front original image section 51a, the prediction determination circuit 54 sets the intra-frame (picture) intra prediction mode (mode in which motion compensation is not performed) as the prediction mode. ) Is set, and the switch 53d of the calculation unit 53 is switched to the contact a side. As a result, the image data of the I picture
The signal is input to the mode switching circuit 55.

As shown in FIG. 19A or 19B, the DCT mode switching circuit 55 converts the data of the four luminance blocks into a state where the lines of the odd field and the lines of the even field are mixed (frame DCT). Mode) or in a separated state (field DCT mode) and output to the DCT circuit 56.

That is, the DCT mode switching circuit 55
Mixed data of odd field and even field D
The coding efficiency in the case of performing the CT processing is compared with the coding efficiency in the case of performing the DCT processing in a separated state,
Select a mode with good coding efficiency.

For example, the input signal is converted to the signal shown in FIG.
As shown in the figure, the configuration is such that the lines of the odd field and the even field are mixed, and the difference between the signal of the line of the odd field and the signal of the line of the even field which are vertically adjacent to each other is calculated. Sum). Further, as shown in FIG. 19B, the input signal has a configuration in which the lines of the odd field and the even field are separated from each other, and the signal difference between the lines of the odd field adjacent vertically and the line of the even field The difference between the signals is calculated, and the sum (or the sum of squares) of the respective absolute values is obtained. Further, the two (the sum of absolute values) are compared, and the DCT mode corresponding to the smaller value is set. That is, if the former is smaller, the frame DCT mode is set, and if the latter is smaller, the field DCT mode is set.

Then, data having a configuration corresponding to the selected DCT mode is output to the DCT circuit 56, and a DCT flag indicating the selected DCT mode is output to the variable length encoding circuit 58 and the motion compensation circuit 64.

The prediction mode (FIG. 18) in the prediction mode switching circuit 52 and the DCT mode switching circuit 5
As is clear from comparison of the DCT mode in FIG. 5 (FIG. 19), with respect to the luminance block, the data structure in each mode is substantially the same.

If the prediction mode switching circuit 52 selects a frame prediction mode (a mode in which odd lines and even lines are mixed), the DCT mode switching circuit 55 also performs a frame DCT mode (odd lines and even lines are mixed). If the field prediction mode (mode in which the data of the odd field and the data of the even field are separated) is selected in the prediction mode switching circuit 52, the DCT mode switching circuit 55 It is highly likely that the DCT mode (a mode in which the data of the odd field and the data of the even field are separated) is selected.

However, this is not always the case. The prediction mode switching circuit 52 determines the mode so that the sum of the absolute values of the prediction errors is small, and the DCT mode switching circuit 55 performs the encoding. The mode is determined so that the efficiency is good.

The I-picture image data output from the DCT mode switching circuit 55 is input to the DCT circuit 56, where it is subjected to DCT (discrete cosine transform) processing and converted into DCT coefficients. This DCT coefficient is input to the quantization circuit 57, and after being quantized in a quantization step corresponding to the data accumulation amount (buffer accumulation amount) of the transmission buffer 59,
It is input to the variable length coding circuit 58.

The variable length coding circuit 58 includes a quantization circuit 57
The image data (in this case, I-picture data) supplied from the quantization circuit 57 is converted into a variable-length code such as a Huffman code in accordance with the quantization step (scale) supplied thereto and transmitted. Output to the buffer 59.

The quantization step (scale) is set by the quantization circuit 57, and the prediction mode (intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction) is set by the prediction determination circuit 54 in the variable length coding circuit 58. Mode indicating whether the motion vector has been detected), a motion vector from the motion vector detection circuit 50,
A prediction flag (a flag indicating whether the frame prediction mode or the field prediction mode is set) is set by the prediction mode switching circuit 52, and a DCT flag (either the frame DCT mode or the field DCT mode) output from the DCT mode switching circuit 55 is set. Are input, and these are also variable-length coded.

The transmission buffer 59 temporarily stores the input data, and converts the data corresponding to the storage amount into a quantization circuit 57.
Output to

When the remaining data amount increases to the allowable upper limit value, the transmission buffer 59 increases the quantization scale of the quantization circuit 57 by the quantization control signal,
The data amount of the quantized data is reduced. Conversely, when the remaining data amount decreases to the permissible lower limit value, the transmission buffer 59 sends the quantization signal to the quantization circuit 57.
By reducing the quantization scale of, the data amount of the quantized data is increased. In this way, overflow or underflow of the transmission buffer 59 is prevented.

The data stored in the transmission buffer 59 is read at a predetermined timing, output to a transmission path, and recorded on the recording medium 3, for example.

On the other hand, the I picture data output from the quantization circuit 57 is input to the inverse quantization circuit 60 and inversely quantized in accordance with the quantization step supplied from the quantization circuit 57. The output of the inverse quantization circuit 60 is IDCT
(Inverse DCT) is input to a circuit 61 and subjected to inverse DCT processing, and then supplied to a forward prediction image section 63a of a frame memory 63 via an arithmetic unit 62 and stored.

When processing the image data of each sequentially input frame as, for example, pictures of I, B, P, B, P, B,... After the image data of the input frame is processed as an I picture, the image data of the next input frame is processed as a P picture before the image of the next input frame is processed as a B picture. This is because a B picture involves backward prediction and cannot be decoded unless a P picture as a backward predicted image is prepared first.

Then, after the processing of the I picture, the motion vector detecting circuit 50 starts the processing of the picture data of the P picture stored in the rear original picture section 51c. Then, as in the case described above, the sum of the absolute values of the inter-frame differences (prediction errors) in macroblock units is supplied from the motion vector detection circuit 50 to the prediction mode switching circuit 52 and the prediction determination circuit 54. Prediction mode switching circuit 52
And the prediction determination circuit 54 sets a frame / field prediction mode or a prediction mode of intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction in accordance with the absolute value sum of the prediction errors of the macroblocks of the P picture. I do.

When the intra-frame prediction mode is set, the arithmetic unit 53 switches the switch 53d to the contact a as described above. Therefore, this data is stored in the DCT mode switching circuit 55, the DCT
The signal is transmitted to a transmission path via a circuit 56, a quantization circuit 57, a variable length encoding circuit 58, and a transmission buffer 59. This data is supplied to the backward prediction image section 63b of the frame memory 63 via the inverse quantization circuit 60, the IDCT circuit 61, and the calculator 62, and is stored.

In the forward prediction mode, the switch 53d is switched to the contact b, and the image data (in this case, the I picture image) stored in the forward prediction image section 63a of the frame memory 63 is read out. The compensation circuit 64 performs motion compensation corresponding to the motion vector output from the motion vector detection circuit 50. That is, when the setting of the forward prediction mode is instructed by the prediction determination circuit 54, the motion compensation circuit 64 sets the read address of the forward prediction image unit 63a to the position of the macroblock that the motion vector detection circuit 50 is currently outputting. Data is read out from the corresponding position by shifting by the amount corresponding to the motion vector,
Generate predicted image data.

The predicted image data output from the motion compensation circuit 64 is supplied to a computing unit 53a. Arithmetic unit 53a
Subtracts the prediction image data corresponding to the macroblock supplied from the motion compensation circuit 64 from the macroblock data of the reference image supplied from the prediction mode switching circuit 52, and outputs the difference (prediction error). I do. This difference data is supplied to the DCT mode switching circuit 55
T circuit 56, quantization circuit 57, variable length coding circuit 58,
The data is transmitted to the transmission path via the transmission buffer 59. Also,
The difference data is supplied to the inverse quantization circuit 60 and the IDCT circuit 6
1 and is locally decoded and input to the arithmetic unit 62.

The arithmetic unit 62 is also supplied with the same data as the prediction image data supplied to the arithmetic unit 53a. The calculator 62 adds the prediction image data output from the motion compensation circuit 64 to the difference data output from the IDCT circuit 61. As a result, image data of the original (decoded) P picture is obtained. The image data of the P picture is supplied to and stored in the backward prediction image section 63b of the frame memory 63.

After the data of the I picture and the P picture are stored in the forward predicted image section 63a and the backward predicted image section 63b, the motion vector detecting circuit 50 executes the processing of the B picture. The prediction mode switching circuit 52 and the prediction determination circuit 54 correspond to the magnitude of the sum of absolute values of the differences between frames in macroblock units,
A frame / field mode is set, and a prediction mode is set to any of an intra-frame prediction mode, a forward prediction mode, a backward prediction mode, and a bidirectional prediction mode.

As described above, in the intra-frame prediction mode or the forward prediction mode, the switch 53d is switched to the contact point a or b. At this time, the same processing as in the case of the P picture is performed, and the data is transmitted.

On the other hand, when the backward prediction mode or the bidirectional prediction mode is set, the switch 53d is switched to the contact point c or d.

In the backward prediction mode in which the switch 53d is switched to the contact point c, the image data (in this case, the image of the P picture) stored in the backward prediction image section 63b is read out, and the motion compensation circuit 64 As a result, motion compensation is performed corresponding to the motion vector output from the motion vector detection circuit 50. That is, when the setting of the backward prediction mode is instructed by the prediction determination circuit 54,
The data is read by shifting the read address of the backward predicted image section 63b from the position corresponding to the position of the macroblock currently output by the motion vector detection circuit 50 by the amount corresponding to the motion vector, and generates predicted image data.

The predicted image data output from the motion compensation circuit 64 is supplied to a calculator 53b. Arithmetic unit 53b
Subtracts the predicted image data supplied from the motion compensation circuit 64 from the macroblock data of the reference image supplied from the prediction mode switching circuit 52, and outputs the difference. This difference data is supplied to the DCT mode switching circuit 5
5, transmitted to the transmission path via the DCT circuit 56, the quantization circuit 57, the variable length coding circuit 58, and the transmission buffer 59.

In the bidirectional prediction mode in which the switch 53d is switched to the contact point d, the image data (in this case, the I picture image) stored in the forward prediction image section 63a and the data stored in the backward prediction image section 63b. Data (in this case, an image of a P picture) is read out, and the motion compensation circuit 64 outputs the motion vector detection circuit 5
Motion compensation is performed corresponding to the motion vector output by 0.
That is, when the setting of the bidirectional prediction mode is instructed by the prediction determination circuit 54, the motion vector detection circuit 50 outputs the read addresses of the forward predicted image section 63a and the backward predicted image section 63b. The data is read out from the position corresponding to the position of the macroblock in which the motion vector is shifted by an amount corresponding to the motion vector (in this case, two motion vectors for the forward predicted image and the backward predicted image), and predicted image data is generated. I do.

The predicted image data output from the motion compensation circuit 64 is supplied to a computing unit 53c. Arithmetic unit 53c
Subtracts the average value of the predicted image data supplied from the motion compensation circuit 64 from the macroblock data of the reference image supplied from the motion vector detection circuit 50, and outputs the difference. This difference data is transmitted to the transmission path via the DCT mode switching circuit 55, the DCT circuit 56, the quantization circuit 57, the variable length coding circuit 58, and the transmission buffer 59.

The B picture is not stored in the frame memory 63 because it is not regarded as a predicted picture of another picture.

In the frame memory 63, the forward prediction image section 63a and the backward prediction image section 63b are switched between banks as necessary, and a predetermined reference image is
The one stored in one or the other can be switched and output as a forward predicted image or a backward predicted image.

In the above description, the description has been made centering on the luminance block.
8 and the macroblocks shown in FIG. 19 are processed and transmitted. As a motion vector for processing a chrominance block, a motion vector obtained by halving the motion vector of the corresponding luminance block in the vertical and horizontal directions is used.

FIG. 20 is a block diagram showing an example of the structure of the decoder 31 shown in FIG. The encoded image data transmitted via the transmission path (recording medium 3) is received by a receiving circuit (not shown) or reproduced by a reproducing device.
After being temporarily stored in the reception buffer 81, it is supplied to the variable length decoding circuit 82 of the decoding circuit 90. The variable length decoding circuit 82 performs variable length decoding on the data supplied from the reception buffer 81, and outputs the motion vector, the prediction mode, the prediction flag and the DCT flag to the motion compensation circuit 87, and the quantization step to the inverse quantization circuit. 83, and outputs the decoded image data to the inverse quantization circuit 83.

The inverse quantization circuit 83 includes the variable length decoding circuit 8
2 is inversely quantized according to the quantization step also supplied from the variable length decoding circuit 82, and is output to the IDCT circuit 84. The data (DCT coefficient) output from the inverse quantization circuit 83 is output to the IDCT circuit 8
In step 4, the inverse DCT processing is performed and the result is supplied to the arithmetic unit 85.

If the image data supplied from the IDCT circuit 84 is I-picture data, the data is output from the arithmetic unit 85 and is input to the arithmetic unit 85 later (P or B-picture data). Is supplied to and stored in the forward prediction image section 86a of the frame memory 86 in order to generate the prediction image data. This data is output to the format conversion circuit 32 (FIG. 15).

If the image data supplied from the IDCT circuit 84 is P-picture data in which the image data one frame before that is the predicted image data and is data in the forward prediction mode, the forward prediction in the frame memory 86 is performed. The image data (I-picture data) one frame before stored in the image unit 86a is read out, and the motion compensation circuit 8
At 7, the motion compensation corresponding to the motion vector output from the variable length decoding circuit 82 is performed. Then, the arithmetic unit 85 adds the image data (difference data) supplied from the IDCT circuit 84 and outputs the result. The added data, that is, the decoded P picture data is
In order to generate predicted image data of image data (B-picture or P-picture data) input later to the arithmetic unit 85, the predicted data is supplied to the backward predicted image section 86b of the frame memory 86 and stored.

Even in the case of P-picture data, the data in the intra-prediction mode is stored in the backward prediction image section 86b as it is, without being subjected to any particular processing by the computing unit 85, like the I-picture data.

Since this P picture is an image to be displayed next to the next B picture, it is not yet output to the format conversion circuit 32 at this point (as described above, the P picture inputted after the B picture The picture is processed and transmitted before the B picture).

When the image data supplied from the IDCT circuit 84 is B picture data, the image data is stored in the forward prediction image section 86 a of the frame memory 86 in accordance with the prediction mode supplied from the variable length decoding circuit 82. I
The image data of the picture (in the case of the forward prediction mode), the image data of the P picture stored in the backward prediction image section 86b (in the case of the backward prediction mode), or both image data (in the case of the bidirectional prediction mode) The motion vector is read and the motion compensation circuit 87 performs motion compensation corresponding to the motion vector output from the variable length decoding circuit 82,
A prediction image is generated. However, if motion compensation is not required (in the case of the intra-picture prediction mode), no predicted picture is generated.

The data which has been subjected to the motion compensation by the motion compensation circuit 87 in the manner described above
It is added to the output of the T circuit 84. This addition output is output to the format conversion circuit 32.

However, this added output is data of a B picture and is not stored in the frame memory 86 because it is not used for generating a predicted image of another image.

After the picture of the B picture is output, the picture data of the P picture stored in the backward prediction picture section 86b is read out, and the arithmetic unit 85 is output via the motion compensation circuit 87.
Supplied to However, at this time, no motion compensation is performed.

The decoder 31 includes a prediction mode switching circuit 52 in the encoder 18 shown in FIG.
Circuits corresponding to the T-mode switching circuit 55 are not shown, but processing corresponding to these circuits, that is, the configuration in which the signals of the lines of the odd field and the even field are separated is necessary for the original mixed configuration. The process of returning according to
The motion compensation circuit 87 executes.

In the above, the processing of the luminance signal has been described, but the processing of the color difference signal is also performed in the same manner.
However, in this case, the motion vector used for the luminance signal is halved in the vertical and horizontal directions.

By the way, when an image signal is encoded and transmitted as described above, when an image with a higher resolution is to be obtained, the image signal encoding device is configured as shown in FIG. 21, for example. In this example, a high-resolution image is supplied to the high-resolution encoder 18H and encoded. The high-resolution image is down-sampled by 1/2 in the vertical direction and the horizontal direction by the down-sampling circuit 101, and
A resolution image is generated. This 1/4 resolution image is supplied to the low resolution encoder 18L and encoded. The high-resolution encoder 18H and the low-resolution encoder 18L have basically the same configuration as the encoder 18 shown in FIG. However, the high-resolution encoder 18H encodes a high-resolution image signal by weighting the predicted image generated by the low-resolution encoder 18L with a predetermined weighting coefficient.

The low-resolution encoder 18L transmits a motion vector, a DCT coefficient, and the like in addition to mode information such as a prediction mode (macroblock type), a frame / field prediction flag, and a frame / field DCT flag. I have. In addition, since the high-resolution encoder 18H basically operates independently of the low-resolution encoder 18L, the high-resolution encoder 18H also outputs mode information, motion vectors, and DCT coefficients. The high resolution encoder 18H also transmits a weighting coefficient obtained by weighting the predicted image supplied from the low resolution encoder 18L.

FIG. 22 shows a configuration example of an image signal decoding device for decoding the image signal transmitted in this way. As shown in the figure, the mode information, the motion vector, and the DCT coefficient output from the low resolution encoder 18L are supplied to a low resolution decoder 31L, and the low resolution decoder 31L converts the image signal based on the input data. Decode and output an image with a quarter resolution. Further, when decoding the low-resolution image signal, the low-resolution decoder 31L generates a predicted image for decoding the high-resolution image signal, and supplies the predicted image to the high-resolution decoder 31H.

The high-resolution decoder 31H is supplied with mode information, motion vectors, weighting coefficients, and DCT coefficients from the high-resolution encoder 18H. The high-resolution decoder 31H decodes an image signal from the input data and outputs a high-resolution image. The low-resolution decoder 31L and the high-resolution decoder 31H have basically the same configuration as the decoder shown in FIG.

On the decoder side, one of the high-resolution image and the quarter-resolution image can be selected as necessary, and an image of a desired resolution can be viewed.

[0083]

However, FIG.
As shown in FIG. 22 and FIG. 22, in the method of hierarchically encoding and decoding this, since the operations in the respective layers are basically independent, each requires dedicated mode information. However, there is a problem that the overhead increases.

The present invention has been made in view of such a situation, and aims to reduce overhead.

[0085]

SUMMARY OF THE INVENTION An image signal encoding method according to the present invention is provided.
The method is determined when encoding low resolution image signals.
Image information with low resolution
Predicted image of high resolution image signal generated from signal
An image signal obtained by multiplying by a weighting coefficient (1-W);
Weighting the high-resolution image signal obtained by partial decoding
Image obtained by adding the image signal obtained by multiplying
When the weighting coefficient W is changed from the image signal
Select and use the image signal with the smallest prediction error
To encode a high-resolution image signal
Is transmitted along with the low-resolution image signal.
In addition, the weighting information W is added to the high-resolution image signal.
It is characterized by transmission. The mode information is included in the image
Prediction, forward prediction, backward prediction or bidirectional prediction
The information may indicate whether the prediction has been performed. Previous
The mode information is used for frame prediction or field prediction.
The information can be used to indicate which prediction was made.
Wear. The mode information includes a frame DCT or a field DCT.
Information indicating which processing of the DCT
can do. The image signal decoding method of the present invention
The image signal with low resolution is decoded using the mode information, and the
When decoding low resolution image signals, high resolution
A predicted image of the image signal is generated, and a high-resolution image signal is generated.
Image obtained by weighting a predicted image with a weighting coefficient (1-W)
Signal, high-resolution image signal obtained by decoding
Image signal weighted by the
Decoding using the mode information attached to the new image signal
It is characterized by the following. The mode information includes intra prediction, previous
Prediction, backward prediction, or bidirectional prediction
It can be information indicating whether or not it has become. The mode
The information is either frame prediction or field prediction.
The information may indicate whether the prediction has been performed. Previous
The mode information is frame DCT or field DC.
Information indicating which processing of T was performed
Can be. An image signal encoding apparatus according to the present invention converts an image signal
Decomposition means for decomposing into layers with different resolutions and low resolution
Encodes image signals that are
Generates a high-resolution image prediction image from the signal and resolves it
Mode information determined when encoding low-grade image signals
Information transmitted along with the low-resolution image signal.
Encoding means, and a predicted image generated by the first encoding means.
An image signal weighted by a weighting coefficient (1-W);
Weights high-resolution image signals obtained by partial decoding
Image obtained by adding the image signal weighted by the weighting coefficient W
When the weighting coefficient W is changed from the signal,
If you select and use the image signal with the smallest measurement error
In both cases, the first encoding means converts the low-resolution image signal.
Using the mode information determined when encoding, the resolution
Image signal with high resolution and image signal with high resolution,
The mode information is added together with the weighting coefficient W.
And a second encoding means for transmitting the data without any modification.
Features. The image signal decoding device of the present invention
First decoding method for decoding a low image signal using mode information
Encoding means, when decoding a low resolution image signal,
Generation means for generating a predicted image of a high-resolution image signal
And a high-resolution image signal
Image signal weighted by (1-W), obtained by decoding
High-resolution image signal is weighted by weighting coefficient W
The second decoding using the image signal and the mode information
2 decoding means. Of the present invention
The image signal encoding method uses the motion
Vector and detect the solution using the detected motion vector.
A prediction vector that predicts the motion vector of a high-resolution image signal.
The motion vector of the high-resolution image signal.
The predicted vector and the detected high-resolution image signal.
Calculate the difference with the motion vector of the signal and transmit the difference.
And features. A high-resolution image signal is
/ V times and 1 / H times in the horizontal direction to reduce
Signal is detected in low resolution image signal
Multiplied by V in the vertical direction, and
H times to obtain the prediction vector.
Wear. The prediction vector is set as an initial value, and this initial value is
Perform block matching within a predetermined range, and
Obtains the difference vector is small, the prediction vector and the differencing
The motion vector in a high-resolution image signal
Torr. Image signal encoding apparatus of the present invention
Is used to detect the motion vector of a low-resolution image signal.
Output means and the motion vector detected by the detection means.
And predict motion vectors of high resolution image signals
A prediction means for obtaining a prediction vector and a high-resolution image signal
Signal motion vector, and the detected high-resolution image
The motion vector of the image signal and the prediction
Calculation means for calculating a difference vector from the measurement vector.
It is characterized by Image signal decoding method of the present invention
Is the difference vector and the motion vector of the low resolution image signal.
And extract the low-resolution image signal from the extracted motion vector.
Low resolution image extracted by decoding using vector
High resolution from signal motion vector and difference vector
Calculate the motion vector of the image signal and calculate the calculated motion
Decode high resolution image signal using vector
It is characterized by the following. The image signal decoding apparatus of the present invention
A first extraction for extracting a motion vector of an image signal having a low resolution
Output means and a motion vector extracted by the first extraction means
To decode a low resolution image signal using
Encoding means and second extracting means for extracting a difference vector
And a low-resolution image extracted by the first extracting means.
The motion vector of the image signal and the
The motion vector of the high-resolution image signal is
Computing means for computing the vector and computing means
Using the obtained motion vector, a high-resolution image signal
And second decoding means for decoding.
You.

[0086]

As described above, the mode information and the motion vector used for encoding the high-resolution image signal are used for encoding the high-resolution image signal. And the amount of mode information and motion vector data to be transmitted.

[0087]

FIG. 1 is a block diagram showing the configuration of an embodiment of an image signal encoding apparatus according to the present invention, and portions corresponding to those in FIG. 21 are denoted by the same reference numerals. That is,
The basic configuration of this embodiment is the same as that of the conventional case shown in FIG.
H to be encoded.
The high-resolution image is output to the downsampling circuit 1
01 is down-sampled by 縦 in the vertical direction and １ ／ in the horizontal direction to generate a 解像度 resolution image.

The down-sampling circuit 101 includes, as shown in FIG.
It can be constituted by a 2: 1 thinning circuit 132. The low-pass filter 131 limits the frequency band to １ ／, and the thinning circuit 132 thins out every other input pixel data and outputs it. In the case of downsampling in the horizontal direction, for example, data is thinned out every other line.

The 1/4 resolution image thus generated is supplied to the low resolution encoder 18L and encoded. The low-resolution encoder 18L outputs the mode information, the motion vector, and the D
Outputs CT coefficients, predictive image, motion vector, predictive mode (macroblock type), frame /
Field prediction flag and frame / field DC
The T flag is supplied to the high resolution encoder 18H.

The high-resolution encoder 18H encodes a high-resolution image signal using these data. Then, a DCT coefficient generated by encoding the high-resolution image signal is output, and a weighting coefficient obtained by weighting the predicted image is output. Also, the motion vector of the high-resolution image signal is not transferred, and the difference vector from the motion vector of the low-resolution image signal is transferred.

The low-resolution encoder 18L and the high-resolution encoder 18H are configured as shown in FIG. 3 or 4, for example. As is clear from comparison of these figures with the encoder 18 shown in FIG. 17, the low-resolution encoder 18L and the high-resolution encoder 18H are also basically the same as those shown in FIG.
The configuration is the same as that of the encoder 18 shown in FIG. Therefore, in these figures, the portions corresponding to those in FIG. 17 will be described by adding the code L in the low resolution encoder 18L and adding the code H in the high resolution encoder 18H.

The prediction mode switching circuit 52L, the prediction determination circuit 54L, or the DCT of the low resolution encoder 18L
The frame / field prediction flag, the prediction mode (macroblock type), or the frame / field DCT flag output by the mode switching circuit 55L respectively corresponds to the prediction mode switching circuit 52H, the prediction determination circuit 54H, or the DCT mode switching of the high resolution encoder 18H. Each is supplied to the circuit 55H.

That is, the prediction mode switching circuit 52, the prediction determination circuit 54L, and the prediction mode switching circuit 52 of the low resolution encoder 18L.
The CT mode switching circuit 55L performs the same operation as the case described with reference to FIG. 17, but the prediction mode switching circuit 52H, the prediction determination circuit 54H, or the DCT mode switching circuit 55H of the high resolution encoder 18H.
Perform operations dependent on corresponding circuits on the low resolution encoder 18L side, and do not perform independent determination processing. This enables faster processing.

The motion vector output from the motion vector detection circuit 50L of the low resolution encoder 18L is supplied to the scaling circuit 151 of the high resolution encoder 18H. The scaling circuit 151 doubles the motion vector of the input low-resolution image signal in the vertical direction and also doubles the motion vector in the horizontal direction. This magnification corresponds to the magnification in the downsampling circuit 101 shown in FIG. That is, in the downsampling circuit 101, when the signal is multiplied by 1 / V in the vertical direction and 1 / H in the horizontal direction, the scaling circuit 151
It is multiplied by V in the vertical direction and H times in the horizontal direction.

The motion vector enlarged by the scaling circuit 151 to the magnification of the corresponding layer is input to the motion vector detection circuit 50H.

The motion vector detection circuit 50L of the low resolution encoder 18L detects a motion vector of a low resolution image signal in units of macro blocks. That is, as shown in FIG. 5, the 1/4 resolution image is divided into N slices every 16 lines, for example. Each slice is 16x1
A macroblock is divided into units of 6 pixels, and a motion vector is detected using the macroblock as a unit. The range in which the motion vector is detected is a range of ± 16 pixels in the vertical and horizontal directions.

As shown in FIG. 5, a 1/4 resolution image is an image generated by downsampling a high resolution image by 1/2 in the horizontal and vertical directions.
When a resolution image is composed of N slices, a high resolution image is composed of 2N slices. And
The number of pixels in the horizontal direction included in each slice of the high-resolution image is twice that of the slice of the quarter-resolution image. Therefore, one macroblock of the ４ resolution image corresponds to four macroblocks adjacent in the vertical and horizontal directions of the high resolution image.

The motion vector detection circuit 50H of the high resolution encoder 18H detects the motion vectors Q _{1 to} Q ₄ for each of these four macroblocks. The motion vector P of the corresponding macro block detected by the detection circuit 50L is used.

That is, the motion vector P is doubled in the vertical and horizontal directions to obtain a motion vector 2P. Then, using this motion vector 2P as an initial value, block matching is performed in a range of ± 2 pixels in each of the vertical direction and the horizontal direction around the initial value, and the vector Δ1 in which the residual becomes the smallest is obtained.
To Δ4. Then, a vector obtained by adding the initial value vector 2P and the difference vector Δ1 to [Delta] 4, a motion vector Q ₁ to Q ₄ of each macro block.

That is, as shown in FIG. 6, the motion vectors Qi of the four macroblocks have an initial value vector of 2P,
When the difference vector is Δi, it is expressed by the following equation. Qi = 2P + Δi

One macroblock of a low-resolution image signal and four macroblocks of a high-resolution image signal are:
Since they correspond to each other, the vectors 2P and Qi should have approximate values. That is, the value of Δi is extremely small. Accordingly, the range for searching for a motion vector may be extremely narrow (in this embodiment, the range is ± 2 pixels). As a result, the motion vector Qi can be detected very quickly.

The motion vector detection circuit 50H converts the motion vector Qi thus detected into a motion compensation circuit 64H.
To supply. The motion compensation circuit 64H performs motion compensation using this motion vector.

On the other hand, the output of the arithmetic unit 62L of the low resolution encoder 18L is used as a predicted image of a high resolution image signal.
Upsampling circuit 1 of high resolution encoder 18H
52 is input.

The up-sampling circuit 152 can be constituted by an interpolation circuit 141, for example, as shown in FIG. This interpolation circuit 141 is, for example, as shown in FIG.
As shown in (1), the data of the non-existing line is generated by halving the values of the data located on the upper and lower lines, respectively, and then adding (averaging). When the down-sampling is performed by the down-sampling circuit 101, the spatial frequency is not widened by the up-sampling because the band is limited by the low-pass filter 131 (FIG. 2), but the resolution is doubled. Back to).

The data up-sampled by the up-sampling circuit 152 in this way is input to the weighting circuit 153, and is weighted by (1-W). That is, the weighting circuit 153 multiplies the data supplied from the upsampling circuit 152 by the weighting coefficient (1-W) set by the weighting coefficient circuit 154, and outputs the result to the calculators 156a to 156c.

On the other hand, the data that has been motion-compensated by the motion compensation circuit 64H is input to the weighting circuit 155 and weighted by the weighting coefficient W. That is, the weighting circuit 15
5 multiplies the coefficient W output from the weighting coefficient circuit 154 by the data input from the motion compensation circuit 64H, and outputs the result to the calculators 156a to 156c. Arithmetic unit 156a
To 156c add the data supplied from the weighting circuit 153 and the data supplied from the weighting circuit 155, and output the result to the computing units 53aH to 53cH.

For example, as weighting coefficients W, 0, 1 /
When 4, 2/4, 3/4, and 1 are set in the weighting circuit 155, the weighting circuit 153
/ 4, 2/4, 1/4, 0 are set. The weighting circuits 155 and 153 multiply the input predicted image signal by five types of coefficients, respectively, and output five types of predicted image signals to the calculators 156a to 156c. Arithmetic unit 156a
156c generate five types of predicted image signals by adding corresponding ones of the five types of weighted predicted image signals. Then, a prediction error signal in the case of adopting each of the five types is generated, a signal having the smallest prediction error signal is selected as a final prediction error signal, and output to the computing units 53aH to 53cH. This enables more efficient compression.

The entire encoding operation in the low-resolution encoder 18L or the high-resolution encoder 18H is basically the same as in the conventional case shown in FIG.
Although the description is omitted, in the present embodiment, the low-resolution encoder 18L outputs a DCT coefficient obtained as a result of encoding the image signal, and also outputs the motion vector detection circuit 50L.
The motion vector detected by L is also transmitted. Furthermore, mode information such as a frame / field prediction flag output from the prediction mode switching circuit 52L, a prediction mode output from the prediction determination circuit 54L, and a frame / field DCT flag output from the DCT mode switching circuit 55L are transmitted together.

On the other hand, the high resolution encoder 18H
Transmits a DCT coefficient obtained by encoding a high-resolution image signal and information on a weighting coefficient W set in the weighting coefficient circuit 154. Further, the motion vector Qi detected by the motion vector detection circuit 50H is not transmitted, but the above-described difference vector Δi is transmitted. For this reason,
A weighting coefficient W output from the weighting coefficient circuit 154;
Difference vector Δ output by motion vector detection circuit 50H
i is supplied to the variable length coding circuit 58H. However, unlike the case of the low resolution encoder 18L, the prediction mode switching circuit 52H and the prediction determination circuit 5
The frame / field prediction flag, prediction mode, or frame / field DCT flag used in the 4H or DCT mode switching circuit 55H is not transmitted.

FIG. 7 shows a format for combining and transmitting the data output from the low-resolution encoder 18L and the high-resolution encoder 18H. As shown in the figure, data of one slice of the first slice (slice 1) of 1/4 resolution is transmitted first, and then data of two slices of high resolution (slice 1 and slice 2) corresponding thereto are transmitted. Is transmitted. In the same manner, data of one slice of の resolution and data of two slices of high resolution corresponding thereto are sequentially transmitted in the same manner.

[0111] A header is arranged at the head of one slice of data of 1/4 resolution, and data is arranged next to the macro block unit. Data corresponding to each macroblock is arranged in the order of mode information, motion vector, and DCT coefficient data. As the mode information, for example, a table obtained as a result of combining the above-described prediction mode (macroblock type), a frame / field prediction flag, a frame / field DCT flag, and the like is prepared on the encoder side and the decoder side. It is transmitted as data specifying the position.

In each slice of a high-resolution image signal, a header is arranged at the head of the slice, and data is arranged in units of macro blocks. In each macroblock, a weighting coefficient W, a difference vector Δi, D
Each data is arranged in the order of the CT coefficient data.

FIG. 8 is a block diagram showing the configuration of an embodiment of an image signal decoding device for decoding the image signal transmitted from the image signal encoding device shown in FIG. The signal transmitted according to the format shown in FIG. 7 is separated, and the data output from the low resolution encoder 18L is supplied to the low resolution decoder 31L,
The data output by 8H is supplied to the high resolution decoder 31H. The low-resolution decoder 31L decodes a low-resolution image from the input data and outputs it. Also, at this time, a predicted image, mode information and a motion vector are detected,
Output to the high resolution decoder 31H. The high-resolution decoder 31H decodes and outputs a high-resolution image using these data.

FIG. 9 shows a configuration example of the low-resolution decoder 31L and the high-resolution decoder 31H. These decoders are basically configured similarly to the circuit shown in FIG. The numbers corresponding to the respective elements shown in FIG. 20 are indicated by adding L to the elements on the low resolution decoder 31L side and adding H to the elements on the high resolution decoder 31H side.

The low resolution decoder 31L receives the input D
The CT coefficient is decoded according to the mode information and the motion vector, and is output as a 1/4 resolution image. The operation is the same as that described with reference to FIG.
The variable length decoding circuit (IV) of the low resolution decoder 31L
LC) 82L separates mode information consisting of a frame / field prediction flag, a prediction mode, and a frame / field DCT flag, and separates this into motion compensation circuit 87L.
, And also to the motion compensation circuit 87H of the high resolution encoder 18H. The motion compensation circuit 87H
Motion compensation is performed according to these input data.

Further, the variable length decoding circuit 82L detects the motion vector, outputs the motion vector to the motion compensation circuit 87L, and outputs the motion vector to the scaling circuit 164 of the high resolution encoder 18H. As in the case of the scaling circuit 151 of the high-resolution encoder 18H, the scaling circuit 164 doubles the motion vector P of the input low-resolution image signal in the vertical and horizontal directions, respectively, and outputs the result to the calculator 165.

The arithmetic unit 165 is also supplied with the difference vector Δi separated by the variable length decoding circuit (IVLC) 82H. The arithmetic unit 165 adds the difference vector Δi to a vector 2P obtained by doubling the motion vector P of the low-resolution image signal supplied from the scaling circuit 164, and obtains the motion vector Qi of the high-resolution image signal.
Ask for. This motion vector Qi is calculated by the motion compensation circuit 87
H. The motion compensation circuit 87H performs motion compensation according to the motion vector Qi and the mode information output from the variable length decoding circuit 82L of the low resolution decoder 31L.

The variable length decoding circuit 82H detects a weighting coefficient W from the input data,
2 and 163. The weighting circuit 162 is supplied with the image signal output from the arithmetic unit 85L of the low resolution decoder 31L via the upsampling circuit 161. This up-sampling circuit 161 is formed by, for example, an interpolation circuit 141 as shown in FIG. 10, similarly to the up-sampling circuit 152 in FIG.

The weighting circuit 162 multiplies the data up-sampled by the up-sampling circuit 161 by a weighting coefficient (1-W) and outputs the result to the arithmetic unit 160.

The operation unit 160 also includes a motion compensation circuit 87.
The signal output from H is supplied after being multiplied by the weighting coefficient W by the weighting circuit 163. The arithmetic unit 160 outputs the output of the weighting circuit 162 and the weighting circuit 1
63 and the final output of the prediction error signal.
Output to the arithmetic unit 85H. The computing unit 85H adds this prediction error signal to the output of the IDCT circuit 84H, and decodes the original high-resolution image. This high resolution decoder 31H
20 is basically the same as the operation in FIG. 20, and a description thereof will be omitted.

In the above-described embodiment, band division is performed by DCT to orthogonally transform data of a block of n × n pixels (n = 8 in the above embodiment). May be used to perform subband division. Further, the present invention can be applied to a case where octave division is performed by wave red transform, or a case where encoding is performed by performing predetermined conversion or division on input two-dimensional image data.

Further, the encoded audio signal and the synchronizing signal are multiplexed with the encoded video signal bit stream, an error correction code is added, and a predetermined modulation is applied. The laser beam can be modulated by the modulation signal and recorded as pits or marks on the disk. Further, a stamper is formed using this disk as a master disk, and a large number of duplicate disks (for example, optical disks) can be formed from the stamper. In this case, the decoder will reproduce data from the duplicate disc.

[0123]

According to the image signal encoding method of the present invention, a high-resolution image signal is encoded by using mode information obtained when encoding a low-resolution image signal. Since the information is transmitted together with the low-resolution image signal and the mode information of the high-resolution image signal is not transmitted, the overhead can be reduced.

According to the image signal decoding method of the present invention, a high-resolution image signal is decoded by using mode information associated with a low-resolution image signal. It is possible to reliably decode a high-resolution image signal from the signal.

According to the image signal encoding method according to the eleventh aspect and the image signal encoding apparatus according to the fourteenth aspect,
Instead of directly transmitting the motion vector of the high-resolution image signal, the difference between the motion vector predicted from the low-resolution image signal and the motion vector of the high-resolution image signal is transmitted. It is possible to reduce it.

According to the image signal decoding method of the present invention, a motion vector of a low-resolution image signal is predicted from a motion vector of a low-resolution image signal. The motion vector of the high-resolution image signal is calculated from the difference between the motion vector obtained and the motion vector of the high-resolution image signal. It becomes possible to decrypt.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an embodiment of an image signal encoding device according to the present invention.

FIG. 2 is a block diagram illustrating a configuration example of a downsampling circuit 101 of FIG. 1;

FIG. 3 is a block diagram illustrating a configuration of a low-resolution encoder 18L of FIG. 1;

FIG. 4 is a block diagram showing a configuration of a high-resolution encoder 18H of FIG.

FIG. 5 is a diagram illustrating motion vectors of low-resolution and high-resolution image signals.

FIG. 6 is a diagram illustrating the principle of detecting a motion vector of a high-resolution image signal.

FIG. 7 is a diagram of a transmission format of a low-resolution and high-resolution image signal.

FIG. 8 is a block diagram illustrating a configuration of an embodiment of an image signal decoding device according to the present invention.

FIG. 9 is a block diagram showing a configuration of a low-resolution decoder 31L and a high-resolution decoder 31H of FIG.

FIG. 10 is a circuit diagram showing the upsampling circuit 152 of FIG. 4 and FIG. 9;
3 is a block diagram illustrating a configuration example of an upsampling circuit 161 of FIG.

11 is a diagram illustrating an interpolation operation of the interpolation circuit 141 in FIG.

FIG. 12 is a diagram illustrating the principle of high-efficiency coding.

FIG. 13 is a view for explaining types of pictures when compressing image data.

FIG. 14 is a diagram illustrating the principle of encoding a moving image signal.

FIG. 15 is a block diagram illustrating a configuration example of a conventional image signal encoding device and a conventional decoding device.

16 is a diagram illustrating the format conversion operation of the format conversion circuit 17 in FIG.

17 is a block diagram illustrating a configuration example of an encoder 18 in FIG.

18 is a diagram illustrating the operation of the prediction mode switching circuit 52 in FIG.

19 is a diagram illustrating the operation of the DCT mode switching circuit 55 of FIG.

20 is a block diagram illustrating a configuration example of a decoder 31 in FIG.

FIG. 21 is a block diagram illustrating a configuration of an example of a conventional image signal encoding device that encodes image signals of different hierarchies.

FIG. 22 is a block diagram illustrating a configuration of an example of a conventional image signal decoding device that decodes image signals of hierarchies having different resolutions.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Encoding device 2 Decoding device 3 Recording medium 12,13 A / D converter 14 Frame memory 15 Luminance signal frame memory 16 Color difference signal frame memory 17 Format conversion circuit 18 Encoder 18L Low resolution encoder 18H High resolution encoder 31 Decoder 31L Low Resolution decoder 31H High resolution decoder 32 Format conversion circuit 33 Frame memory 34 Luminance signal frame memory 35 Color difference signal frame memory 36, 37 D / A converter 50, 50L, 50H Motion vector detection circuit 51, 51L, 51H Frame memory 52, 52L , 52H prediction mode switching circuit 53, 53L, 53H calculation unit 54, 54L, 54H prediction determination circuit 55, 55L, 55H DCT mode switching circuit 56, 56L, 56H DCT circuit 57, 57L, 57H Quantization circuit 58, 58L, 58H Variable length coding circuit 59, 59L, 59H Transmission buffer 60, 60L, 60H Inverse quantization circuit 61, 61L, 61H IDCT circuit 62, 62L, 62H Arithmetic unit 63, 63L, 63H Frame memories 64, 64L, 64H Motion compensation circuits 81, 81L, 81H Receiving buffers 82, 82L, 82H Variable length decoding circuits 83, 83L, 83H Inverse quantization circuits 84, 84L, 84H IDCT circuits 85, 85L, 85H arithmetic unit 86, 86L, 86H frame memory 87, 87L, 87H motion compensation circuit 101 downsampling circuit 131 low-pass filter 132 thinning circuit 141 interpolation circuit 151 scaling circuit 152 upsampling circuit 153, 155 weighting circuit 1 4 weighting coefficient circuits 156a, 156b, 156c calculator 161 up-sampling circuit 162, 163 weighting circuit 164 the scaling circuit 165 calculator

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-4-177992 (JP, A) Yasuo Katayama, "Outline and Standardization Trends of MPEG", Interface, CQ Publishing Company, August 1, 1992, No. 1 Vol. 18, No. 8, p. 124−146 (58) Fields surveyed (Int.Cl. ⁷ , DB name) H04N ^7/ 24-7/68 H04N 1/41-1/419

Claims (16)

(57) [Claims] An image signal is decomposed into layers having different resolutions, a low-resolution image signal is encoded, a predicted image of a high-resolution image signal is generated from a low-resolution image signal, and the predicted image is used. In the image signal encoding method for encoding a high resolution image signal and transmitting the image signal together with the low resolution image signal, the mode information determined when encoding the low resolution image signal is used , and the low resolution image is used. signal
Weights the predicted image of the high-resolution image signal generated from
Image signal obtained by multiplying by the
Weights the high-resolution image signal obtained by decoding
The image signal obtained by adding the image signal obtained by multiplying by the coefficient W
When the weighting coefficient W is changed from
A high resolution image signal is encoded by selecting and using an image signal with the smallest prediction error , and the mode information is transmitted along with the low resolution image signal, and the weighting information W is converted to the resolution. High
An image signal encoding method characterized in that the image signal is transmitted together with a new image signal. 2. The image signal encoding method according to claim 1, wherein the mode information is information indicating which of intra prediction, forward prediction, backward prediction, and bidirectional prediction has been performed. . 3. The mode information according to claim 1, wherein the mode information is information indicating which of a frame prediction and a field prediction is performed.
Or the image signal encoding method according to 2. 4. The image signal encoding method according to claim 1, wherein said mode information is information indicating which processing of frame DCT or field DCT has been performed. 5. Decomposing an image signal into hierarchies having different resolutions, encoding a low-resolution image signal, and generating a predicted image of a high-resolution image signal from the low-resolution image signal; a low image signal resolution with use mode information determined when coding, the prediction
Image signal obtained by multiplying an image by a weighting coefficient (1-W)
And the high resolution image signal obtained by local decoding
Add the image signal obtained by multiplying the weight by the weighting coefficient W
The weighting coefficient W is changed from the obtained image signals.
The image signal with the smallest prediction error
And use this in, encodes a high-resolution image signal, the mode information, while transmitting in association with the low resolution image signal, the weighting information W of the resolution high
A video signal decoding method for transmitting an image signal having a high resolution and transmitting a high-resolution image signal without attaching the mode information, the image signal having a low resolution using the mode information. When decoding a low-resolution image signal, a predicted image of a high-resolution image signal is generated, and the high-resolution image signal is weighted by a weighting coefficient of the predicted image.
Image signal weighted by (1-W), obtained by decoding
High-resolution image signal is weighted by weighting coefficient W
Image signals, and an image signal decoding method characterized in that decoding by using the mode information associated with the low resolution image signal. 6. The image signal decoding method according to claim 5, wherein the mode information is information indicating which of intra prediction, forward prediction, backward prediction and bidirectional prediction has been performed. . 7. The mode information according to claim 5, wherein the mode information is information indicating which of a frame prediction and a field prediction is performed.
Or the image signal decoding method according to 6. 8. The image signal decoding method according to claim 5, wherein the mode information is information indicating whether processing of frame DCT or field DCT has been performed. 9. A decomposing means for decomposing an image signal into hierarchies having different resolutions, encoding a low-resolution image signal, and generating a predicted image of a high-resolution image signal from the low-resolution image signal; First encoding means for transmitting the mode information determined when encoding the low image signal along with the low resolution image signal; and weighting the predicted image generated by the first encoding means .
The image signal weighted by the correction coefficient (1-W)
Weighting coefficient for high resolution image signal obtained by decoding
Of the image signal obtained by adding the image signal weighted by W
From the middle, when the weighting coefficient W is changed,
When selecting and using the image signal with the smallest error
In addition, the first encoding unit uses the mode information determined when encoding a low-resolution image signal,
A second encoding unit that encodes a high-resolution image signal and transmits the high-resolution image signal without accompanying the weighting coefficient W and the mode information. Signal encoding device. 10. decomposes the image signal into different levels of resolution, as well as encode the low image signal resolution, and generates a predicted image of a high resolution image signals from the low resolution image signal, the prediction image, the Weighting coefficient of predicted image (1-W)
And the solution obtained by local decoding
An image obtained by weighting an image signal having a high
The weighted image signal from the image signal obtained by adding the
When the prediction coefficient W is changed, the prediction error is minimized.
That with use selects an image signal, by using the mode information determined when encoding a low resolution image signal, it encodes the high image signal resolution, the mode information, a low resolution image signal And transmits the high-resolution image signal with the weighting coefficient W.
An image signal decoding apparatus for decoding an image signal transmitted without accompanying the mode information , wherein the first decoding means decodes an image signal having a low resolution using the mode information; when decoding a low image signals, and generating means for generating a predicted image of a high resolution image signal, a high image signal resolution, weights the prediction image coefficients
Image signal weighted by (1-W), obtained by decoding
High-resolution image signal is weighted by weighting coefficient W
An image signal decoding device comprising: an image signal; and second decoding means for decoding using the mode information. 11. An image signal is decomposed into hierarchies having different resolutions, a low-resolution image signal is encoded, and a predicted image of a high-resolution image signal is generated from the low-resolution image signal. In the image signal encoding method for encoding a high-resolution image signal and transmitting the image signal together with the low-resolution image signal, a motion vector of the low-resolution image signal is detected, and a high-resolution image signal is detected by using the detected motion vector. Obtaining a prediction vector for predicting a motion vector of the image signal; detecting a motion vector of the high-resolution image signal; calculating a difference between the prediction vector and the detected motion vector of the high-resolution image signal; Image signal encoding method. 12. An image signal having a high resolution is set to 1 in the vertical direction.
/ V times and 1 / H times in the horizontal direction to generate a low-resolution image signal, the motion vector detected in the low-resolution image signal is multiplied by V in the vertical direction and The image signal encoding method according to claim 11, wherein the prediction vector is obtained by multiplying by H. 13. Using the prediction vector as an initial value, performing block matching in a predetermined range around the initial value to obtain a difference vector with the smallest residual, and calculating the sum of the prediction vector and the difference vector as a resolution 13. The image signal encoding method according to claim 12, wherein the motion vector is a motion vector in a high image signal. 14. An image signal is decomposed into layers having different resolutions, a low-resolution image signal is encoded, and a predicted image of a high-resolution image signal is generated from the low-resolution image signal. An image signal encoding method for encoding a high-resolution image signal and transmitting the image signal together with a low-resolution image signal, comprising: detecting means for detecting a motion vector of the low-resolution image signal; and detecting the motion vector detected by the detecting means. make use of,
Prediction means for obtaining a prediction vector for predicting a motion vector of a high-resolution image signal; detecting a motion vector of the high-resolution image signal; and detecting the motion vector of the detected high-resolution image signal. And a calculating means for calculating a difference vector from the predicted vector. 15. A low-resolution image signal and its associated motion vector, and a high-resolution image signal and its associated motion vector of a high-resolution image signal predicted from the low-resolution image signal. An image signal decoding method for decoding an image signal including a difference vector between a predicted vector of the image signal and a motion vector of a high-resolution image signal, comprising: extracting the difference vector and the motion vector of the low-resolution image signal; The image signal is decoded using the extracted motion vector, and the motion vector of the high-resolution image signal is calculated from the extracted motion vector of the low-resolution image signal and the difference vector. An image signal decoding method comprising decoding an image signal having a high resolution using the method. 16. A low-resolution image signal and its associated motion vector, and a high-resolution image signal and its associated motion vector of a high-resolution image signal predicted from the low-resolution image signal. An image signal decoding device that decodes an image signal including a difference vector between the predicted vector of the image signal and the motion vector of the high-resolution image signal, wherein: a first extraction unit that extracts a motion vector of the low-resolution image signal; A first decoding unit for decoding a low-resolution image signal using the motion vector extracted by the first extraction unit; a second extraction unit for extracting the difference vector; and the first extraction unit. From the motion vector of the low resolution image signal extracted by the above and the difference vector extracted by the second extracting means, a high resolution image is obtained. Image processing means for calculating a motion vector of a signal, and second decoding means for decoding a high-resolution image signal using the motion vector calculated by the calculation means. Signal decoding device.