JP3164110B2 - Encoding device and method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for recording a moving image signal on a recording medium such as a magneto-optical disk or a magnetic tape and reproducing and displaying the signal on a display or the like. The present invention relates to an encoding apparatus and method suitable for transmitting a moving image signal from a transmitting side to a receiving side via a transmission path, and receiving and displaying the moving image signal on a receiving side. is there.

[0002]

2. Description of the Related Art For example, in a system for transmitting a moving image signal to a remote place, such as a video conference system and a video telephone system, in order to efficiently use a transmission path, line correlation or inter-frame correlation of a video signal is required. Is used to compress and encode an image signal.

In the case of utilizing the line correlation, the image signal can be compressed by, for example, DCT (discrete cosine transform) processing.

[0004] When the inter-frame correlation is used, it is possible to further compress and encode the image signal. For example, as shown in FIG. 6A, time t = t1, t2, t3
In FIG. 6, when the frame images PC1, PC2, and PC3 are respectively generated, the difference between the image signals of the frame images PC1 and PC2 is calculated, and as shown in FIG.
C12 is generated, and the frame image PC2 of FIG.
The image PC23 of FIG. 6B is generated by calculating the difference between PC3 and PC3. Normally, the images of frames temporally adjacent to each other do not have such a large change. Therefore, when the difference between them is calculated, the difference signal has a small value. That is, in the image PC 12 shown in FIG.
The difference between the image signals of the frame images PC1 and PC2 of
The signal of the portion of the image PC12 shown in FIG. 6B indicated by oblique lines is obtained. In the image PC23 shown in FIG. 6B, the difference between the image signals of the frame images PC2 and PC3 in FIG. The signal of the portion of the image PC23 shown in FIG. 6B indicated by oblique lines is obtained. Therefore, if the difference signal is encoded, the code amount can be compressed.

However, the transmission of only the difference signal cannot restore the original image. Therefore, an image of each frame is defined as an I picture (Intra-coded picture), a P picture (forward predictive coded picture) or a B picture (quantitative predictive coded picture: Bidirectionally-codedp).
picture), and the image signal is compressed and encoded.

That is, as shown in, for example, FIGS. 7A and 7B, image signals of 17 frames from F1 to F17 are set as a group of pictures, and are set as one unit of processing. Then, the image signal of the first frame F1 is coded as an I picture, the second frame F2 is processed as a B picture, and the third frame F3 is processed as a P picture. Hereinafter, the fourth and subsequent frames F4 to F17 are alternately processed as B pictures or P pictures.

As an I-picture image signal, an image signal for one frame is transmitted as it is. On the other hand, as an image signal of a P picture, basically, as shown in FIG.
As shown in A of FIG. 7, a difference from an image signal of an I picture or a P picture that precedes it is encoded and transmitted. Further, as an image signal of a B picture, basically, as shown in FIG. 7B, a difference from an average value of both temporally preceding and succeeding frames is obtained, and the difference is encoded. To transmit.

FIGS. 8A and 8B show the principle of a method for encoding a moving picture signal in this manner. FIG. 8A schematically shows frame data of a moving image video signal, and FIG. 8B schematically shows frame data to be transmitted. As shown in FIG. 8, since the first frame F1 is processed as an I picture, that is, a non-interpolated frame,
The data is directly transmitted as transmission data F1X (transmission non-interpolated frame data) to the transmission path (intra-picture encoding). On the other hand, since the second frame F2 is processed as a B picture, that is, an interpolation frame, the second frame F2 is temporally preceding and the second frame F3 is temporally succeeding.
The difference from the average value of the (non-interpolated frames of inter-frame coding) is calculated, and the difference is transmitted as transmission data (transmission interpolation frame data) F2X.

[0009] However, there are four types of processing as a B picture in more detail. In the first process, the data of the original frame F2 is represented by a dashed arrow SP in the figure.
As shown by 1, the data is transmitted as transmission data F2X as it is (intra coding), and the same processing as in the case of an I picture is performed. The second process is to calculate a difference from the temporally later frame F3, and to use a dashed arrow SP2 in the figure.
The difference is transmitted as shown by (back prediction coding). The third process is to transmit a difference from the temporally preceding frame F1 as indicated by a broken arrow SP3 in the drawing (forward prediction coding). Further, the fourth processing is
As shown by a broken arrow SP4 in the figure, a difference between the average value of the temporally preceding frame F1 and the average value of the following frame F3 is generated and transmitted as transmission data F2X (bidirectional predictive coding).

[0010] Of these four methods, the method that minimizes transmission data is adopted.

When transmitting the difference data, a motion vector x1 (frame F1 in the case of forward prediction) between the image of the frame (prediction image) for which the difference is to be calculated.
Motion vector between F2 and F2) or motion vector x
2 (the motion vector between frames F3 and F2 in the case of backward prediction) or both motion vectors x1 and x2 (in the case of bidirectional prediction) are transmitted together with the difference data.

Further, a frame F3 of a P picture (a non-interpolated frame of inter-frame coding) has a difference signal (indicated by a broken-line arrow SP3) from the frame F1, using a temporally preceding frame F1 as a predicted image. Motion vector x3
Is calculated and transmitted as transmission data F3X (forward prediction coding). Alternatively, the original frame F3
Is transmitted as it is as transmission data F3X (indicated by a broken line arrow SP1) (intra coding). This P
In a picture, which method is used for transmission is the same as in the case of a B picture, and a method in which less data is transmitted is selected.

The frame F4 of the B picture and the frame F5 of the P picture are processed in the same manner as described above, and transmission data F4X, F5X, motion vectors x4, x5, x6 and the like are obtained.

FIG. 9 shows an example of the configuration of an apparatus that encodes and transmits a moving image signal based on the above-described principle and decodes it. The encoding device 1 encodes an input video signal, and transmits and records the video signal on a recording medium 3 as a transmission path. The decoding device 2 reproduces the signal recorded on the recording medium 3, and decodes and outputs the signal.

First, in the encoding device 1, the video signal VD input via the input terminal 10 is converted into a pre-processing circuit 11.
, Where the luminance and chrominance signals (in this case,
A / D converters 12 and 1 are separated from each other.
3 is A / D converted. The video signal converted into a digital signal by A / D conversion by the A / D converters 12 and 13 is
The data is supplied to the frame memory 14 and stored. In the frame memory 14, the luminance signal is stored in the luminance signal frame memory 15, and the color difference signals are stored in the color difference signal frame memory 1.
6 respectively.

The format conversion circuit 17 converts the signal of the frame format stored in the frame memory 14 into
Convert to block format signal. That is, FIG.
(A), the video signal stored in the frame memory 14 is data in a frame format in which V lines of H dots are collected per line. The format conversion circuit 17 divides the signal of one frame into N slices in units of 16 lines. Then, each slice is divided into M macroblocks, as shown in FIG. Each macro block is configured by a luminance signal corresponding to 16 × 16 pixels (dots), as shown in FIG. 10C, and the luminance signal is expressed as shown in FIG. 8 more
Blocks Y [1] to Y [4] in units of × 8 dots
It is divided into The 16 × 16 dot luminance signal includes an 8 × 8 dot Cb signal and an 8 × 8 dot Cb signal.
r signals are supported.

The data thus converted into the block format is supplied from the format conversion circuit 17 to the encoder 18, where the data is encoded. The details will be described later with reference to FIG.

The signal encoded by the encoder 18 is output to a transmission path as a bit stream and recorded on the recording medium 3, for example.

The data reproduced from the recording medium 3 is supplied to the decoder 31 of the decoding device 2 and decoded.
The details of the decoder 31 will be described later with reference to FIG.

The data decoded by the decoder 31 is input to a format conversion circuit 32 and converted from a block format to a frame format. The luminance signal in the frame format is supplied to and stored in the luminance signal frame memory 34 of the frame memory 33, and the color difference signal is supplied to and stored in the color difference signal frame memory 35. The luminance signal and the color difference signal read from the luminance signal frame memory 34 and the color difference signal frame memory 35 are D / A converters 36 and 37 respectively.
The signal is A-converted, supplied to the post-processing circuit 38, and synthesized.
This output video signal is output from the output terminal 30 to a display such as a CRT (not shown) and displayed.

Next, a configuration example of the encoder 18 will be described with reference to FIG.

The image data to be coded supplied through the input terminal 49 is input to the motion vector detection circuit 50 in macroblock units. The motion vector detection circuit 50 processes the image data of each frame as an I picture, a P picture, or a B picture according to a predetermined sequence set in advance. Whether the image of each frame that is sequentially input is processed as one of I, P, and B pictures is determined in advance (for example, as shown in FIG. 7, frames F1 to F1
7 are I, B,
P, B, P,..., B, P).

The image data of a frame (for example, frame F1) processed as an I picture is obtained from a motion vector detecting circuit 50 by a front original image section 51a of a frame memory 51.
The image data of a frame (for example, frame F2) that is transferred and stored in a B picture (for example, frame F2) is transferred and stored in an original image section (reference original image section) 51b, and is processed as a P picture (for example, frame F3). The image data of ()) is transferred and stored in the rear original image section 51c.

At the next timing, B
When an image of a frame to be processed as a picture (for example, the frame F4) or a P picture (the frame F5) is input, the image of the first P picture (frame F3) stored in the rear original image unit 51c up to that time The data is transferred to the front original image section 51a, and the image data of the next B picture (frame F4) is stored (overwritten) in the original image section 51b, and the next P picture (frame F5)
Is stored (overwritten) in the rear original image section 51c.
Is done. Such operations are sequentially repeated.

The signal of each picture stored in the frame memory 51 is read therefrom, and the prediction mode switching circuit 52 performs frame prediction mode processing or field prediction mode processing. Further, under the control of the prediction determination circuit 54, the calculation unit 53 performs calculation of intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction. Which of these processes is to be performed is determined according to the prediction error signal (the difference between the reference image to be processed and the predicted image corresponding to the reference image). Therefore, the motion vector detection circuit 50 generates a sum of absolute values (or a sum of squares) of the prediction error signal used for this determination.

Here, the frame prediction mode and the field prediction mode in the prediction mode switching circuit 52 will be described.

When the frame prediction mode is set, the prediction mode switching circuit 52 controls the four luminance blocks Y supplied from the motion vector detection circuit 50.
[1] to Y [4] are output as they are to the operation unit 53 in the subsequent stage. That is, in this case, as shown in FIG. 12A, the data of the odd field lines and the data of the even field lines are mixed in each luminance block. The solid line in each macroblock in FIG. 12 indicates line data of an odd field (line of the first field), and the broken line indicates data of line of an even field (line of the second field). “a” and “b” indicate units of motion compensation. In this frame prediction mode, prediction is performed in units of four luminance blocks (macroblocks), and one motion vector corresponds to four luminance blocks.

On the other hand, the prediction mode switching circuit 5
FIG. 12 shows a case where the field prediction mode is set.
The signal input from the motion vector detection circuit 50 in the configuration shown in FIG. 12A is converted into luminance blocks Y [1] and Y [2] out of the four luminance blocks as shown in FIG. The other two luminance blocks Y [3] and Y [4] are constituted by the data of the even-numbered field lines, and output to the calculation unit 53. In this case, one motion vector corresponds to two luminance blocks Y [1] and Y [2], and the other two luminance blocks Y [3] and Y [3]
Another motion vector is associated with [4].

Referring to the configuration of FIG. 11, the motion vector detecting circuit 50 calculates a sum of absolute values of prediction errors in the frame prediction mode and a sum of absolute values of prediction errors in the field prediction mode by using a prediction mode switching circuit 52. Output to The prediction mode switching circuit 52 compares the sum of absolute values of prediction errors in the frame prediction mode and the field prediction mode, performs the above-described processing corresponding to the prediction mode having a small value, and outputs data to the calculation unit 53.

However, such processing is actually performed by the motion vector detection circuit 50. That is, the motion vector detection circuit 50 outputs a signal having a configuration corresponding to the determined mode to the prediction mode switching circuit 52, and the prediction mode switching circuit 52 outputs the signal as it is to the subsequent operation unit 53.

Note that, in the frame prediction mode, the color difference signal is supplied to the arithmetic unit 53 in a state where the data of the odd field lines and the data of the even field lines are mixed as shown in FIG. You. In the case of the field prediction mode, as shown in FIG. 12B, the upper half (4 lines) of each of the chrominance blocks Cb and Cr includes the chrominance of the odd field corresponding to the luminance block Y [1] and Y [2]. And the lower half (4 lines) is a luminance block Y
These are color difference signals of even fields corresponding to [3] and Y [4].

In addition, the motion vector detection circuit 50 causes the prediction determination circuit 54 to perform intra-picture prediction,
A sum of absolute values of prediction errors for determining whether to perform forward prediction, backward prediction, or bidirectional prediction is generated.

That is, the sum of the signals Aij of the macroblocks of the reference image ΣAij
Of the absolute value | Aij | of the macroblock signal Aij and the sum Σ | Aij | of the absolute value | Aij | The sum of the absolute value | Aij−Bij | of the difference (Aij−Bij) between the signal Aij of the macroblock of the reference image and the signal Bij of the macroblock of the predicted image is used as the absolute value sum of the prediction error of the forward prediction.
| Aij−Bij |. In addition, the absolute value sum of the prediction error between the backward prediction and the bidirectional prediction is obtained in the same manner as in the forward prediction (by changing the predicted image to a different predicted image from that in the forward prediction).

The sum of these absolute values is supplied to the prediction judgment circuit 54. The prediction determination circuit 54 selects the smallest absolute value sum of the prediction errors of the forward prediction, the backward prediction, and the bidirectional prediction as the sum of the absolute values of the prediction errors of the inter prediction. Further, the sum of the absolute values of the prediction errors of the inter prediction and the sum of the absolute values of the prediction errors of the intra prediction are compared, and the smaller one is selected, and the mode corresponding to the selected absolute value sum is set as the prediction mode. select. That is, if the sum of absolute values of the prediction errors of the intra prediction is smaller, the intra prediction mode is set. If the sum of the absolute values of the prediction errors in the inter prediction is smaller, the mode in which the corresponding sum of the absolute values is the smallest among the forward prediction, backward prediction, and bidirectional prediction modes is set.

As described above, the motion vector detection circuit 50
The macroblock signal of the reference image is transmitted through the prediction mode switching circuit 52 in the configuration shown in FIG. 12 corresponding to the mode selected by the prediction mode switching circuit 52 among the frame or field prediction modes. A motion vector between the predicted image and the reference image corresponding to the prediction mode selected by the prediction determination circuit 54 among the four prediction modes is detected, and a variable length encoding circuit described later is supplied to the arithmetic unit 53. 58 and motion compensation circuit 64
Output to As described above, as the motion vector, the motion vector having the smallest absolute value sum of the corresponding prediction errors is selected.

When the motion vector detection circuit 50 reads out the image data of the I picture from the front original image section 51a, the prediction determination circuit 54 sets the intra-frame (image) prediction mode (the mode in which no motion compensation is performed) as the prediction mode. ) Is set, and the switch 53d of the calculation unit 53 is switched to the contact a side. As a result, the image data of the I picture
The signal is input to the mode switching circuit 55.

As shown in FIG. 13A or 13B, the DCT mode switching circuit 55 converts the data of the four luminance blocks into a state where the lines of the odd field and the lines of the even field are mixed (frame DCT mode). Alternatively, the signal is output to the DCT circuit 56 in one of the separated states (field DCT mode).

That is, the DCT mode switching circuit 55
Mixed data of odd field and even field D
The coding efficiency in the case of performing the CT processing is compared with the coding efficiency in the case of performing the DCT processing in a separated state,
Select a mode with good coding efficiency.

For example, as shown in FIG. 13A, the input signal has a configuration in which the lines of the odd field and the even field are mixed, and the signal of the odd field line and the signal of the even field line which are vertically adjacent to each other. Is calculated, and the sum (or sum of squares) of the absolute values is calculated. Further, as shown in FIG. 13B, the input signal has a configuration in which the lines of the odd field and the even field are separated, and the signal difference between the lines of the odd field adjacent vertically and the line of the even field are separated. Are calculated, and the sum (or sum of squares) of the absolute values is calculated. Further, the two (the sum of absolute values) are compared, and the DCT mode corresponding to the smaller value is set. That is, if the former is smaller, the frame DCT mode is set, and if the latter is smaller, the field DCT mode is set.

Then, data having a configuration corresponding to the selected DCT mode is output to the DCT circuit 56, and a DCT flag indicating the selected DCT mode is output to the variable length coding circuit 58 and the motion compensation circuit 64.

As is apparent from a comparison between the prediction mode (see FIG. 12) in the prediction mode switching circuit 52 and the DCT mode (see FIG. 13) in the DCT mode switching circuit 55, each mode of the luminance block is different. Are substantially the same.

When the prediction mode switching circuit 52 selects a frame prediction mode (a mode in which odd lines and even lines are mixed), the DCT mode switching circuit 55 also performs a frame DCT mode (odd lines and even lines are mixed). When the field prediction mode (mode in which the data of the odd field and the data of the even field are separated) is selected in the prediction mode switching circuit 52, the DCT mode switching circuit 55 It is highly likely that the DCT mode (a mode in which the data of the odd field and the data of the even field are separated) is selected.

However, this is not always the case. In the prediction mode switching circuit 52, the mode is determined so that the absolute value sum of the prediction errors becomes small. The mode is determined so that the efficiency is good.

The image data of the I picture output from the DCT mode switching circuit 55 is input to the DCT circuit 56, where it is subjected to DCT (discrete cosine transform) processing and converted into DCT coefficients. This DCT coefficient is input to the quantization circuit 57, and after being quantized in a quantization step corresponding to the data accumulation amount (buffer accumulation amount) of the transmission buffer 59,
It is input to the variable length coding circuit 58.

The variable length coding circuit 58 includes a quantization circuit 57
The image data (in this case, I-picture data) supplied from the quantization circuit 57 is converted into, for example, Huffman (Huffm) in accordance with the supplied quantization step (scale).
an) is converted to a variable-length code such as
Output to

The quantization step (scale) is set by the quantization circuit 57 and the prediction mode (intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction) is set by the prediction determination circuit 54 in the variable length coding circuit 58. Mode indicating whether the motion vector has been detected), a motion vector from the motion vector detection circuit 50,
A prediction flag (a flag indicating whether the frame prediction mode or the field prediction mode has been set) from the prediction mode switching circuit 52 and a DCT flag (either the frame DCT mode or the field DCT mode) output from the DCT mode switching circuit 55 are set. Are input, and these are also variable-length coded.

The transmission buffer 59 temporarily stores the input data, and converts the data corresponding to the storage amount into a quantization circuit 57.
Output to

When the remaining data amount increases to the allowable upper limit value, the transmission buffer 59 increases the quantization scale of the quantization circuit 57 by the quantization control signal,
The data amount of the quantized data is reduced. Conversely, when the remaining data amount decreases to the permissible lower limit value, the transmission buffer 59 sends the quantization signal to the quantization circuit 57.
By reducing the quantization scale of, the data amount of the quantized data is increased. In this way, overflow or underflow of the transmission buffer 59 is prevented.

The data stored in the transmission buffer 59 is read at a predetermined timing, and
Is output to the transmission path via the communication medium 3 and recorded on the recording medium 3, for example.

On the other hand, the I picture data output from the quantization circuit 57 is input to the inverse quantization circuit 60 and inversely quantized in accordance with the quantization step supplied from the quantization circuit 57. The output of the inverse quantization circuit 60 is IDCT
(Inverse DCT) is input to a circuit 61 and subjected to inverse DCT processing, and then supplied to a forward prediction image section 63a of a frame memory 63 via an arithmetic unit 62 and stored.

By the way, the motion vector detection circuit 50 converts the image data of each frame inputted sequentially into
For example, as described above, I, B, P, B, P, B,.
When each image is processed as a picture, after the image data of the first input frame is processed as an I picture, before the image of the next input frame is processed as a B picture, The image data of the frame is processed as a P picture. This is because a B picture involves backward prediction and cannot be decoded unless a P picture as a backward predicted image is prepared first.

Therefore, the motion vector detection circuit 50 starts the processing of the picture data of the P picture stored in the rear original picture section 51c after the processing of the I picture. Then, as in the case described above, the absolute value sum of the inter-frame difference (prediction error) in macroblock units is supplied from the motion vector detection circuit 50 to the prediction mode switching circuit 52 and the prediction determination circuit 54. Prediction mode switching circuit 52
And the prediction determination circuit 54 sets a frame / field prediction mode or a prediction mode of intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction in accordance with the absolute value sum of the prediction errors of the macroblocks of the P picture. I do.

When the intra-frame prediction mode is set, the arithmetic unit 53 switches the switch 53d to the contact a as described above. Therefore, this data is stored in the DCT mode switching circuit 55, the DCT
The signal is transmitted to a transmission path via a circuit 56, a quantization circuit 57, a variable length encoding circuit 58, and a transmission buffer 59. This data is supplied to the backward prediction image section 63b of the frame memory 63 via the inverse quantization circuit 60, the IDCT circuit 61, and the calculator 62, and is stored.

On the other hand, in the forward prediction mode, the switch 53
d is switched to the contact b, and the image (I-picture image in this case) data stored in the forward prediction image section 63a of the frame memory 63 is read out. Are subjected to motion compensation in accordance with the output motion vector. That is, when the setting of the forward prediction mode is instructed by the prediction determination circuit 54, the motion compensation circuit 64
The data is read by shifting the read address of a from the position corresponding to the position of the macro block which is currently being output by the motion vector detection circuit 50 by an amount corresponding to the motion vector, and generates predicted image data.

The predicted image data output from the motion compensation circuit 64 is supplied to a calculator 53a. Arithmetic unit 53a
Subtracts the prediction image data corresponding to the macroblock supplied from the motion compensation circuit 64 from the macroblock data of the reference image supplied from the prediction mode switching circuit 52, and outputs the difference (prediction error). I do. This difference data is supplied to the DCT mode switching circuit 55
T circuit 56, quantization circuit 57, variable length coding circuit 58,
The data is transmitted to the transmission path via the transmission buffer 59. Also,
The difference data is supplied to the inverse quantization circuit 60 and the IDCT circuit 6
1 and is locally decoded and input to the arithmetic unit 62.

The arithmetic unit 62 is also supplied with the same data as the predicted image data supplied to the arithmetic unit 53a. The calculator 62 adds the prediction image data output from the motion compensation circuit 64 to the difference data output from the IDCT circuit 61. As a result, image data of the original (decoded) P picture is obtained. The image data of the P picture is supplied to and stored in the backward prediction image section 63b of the frame memory 63.

After the data of the I picture and the P picture are stored in the forward predicted image section 63a and the backward predicted image section 63b, respectively, the motion vector detection circuit 50 then executes the processing of the B picture. The prediction mode switching circuit 52 and the prediction determination circuit 54 correspond to the magnitude of the sum of absolute values of the differences between frames in macroblock units,
A frame / field mode is set, and a prediction mode is set to any of an intra-frame prediction mode, a forward prediction mode, a backward prediction mode, and a bidirectional prediction mode.

As described above, in the intra prediction mode or the forward prediction mode, the switch 53d is switched to the contact point a or b. At this time, the same processing as in the case of the P picture is performed, and the data is transmitted.

On the other hand, when the backward prediction mode or the bidirectional prediction mode is set, the switch 53d is switched to the contact point c or d.

In the backward prediction mode in which the switch 53d is switched to the contact point c, the image (in this case, the P picture) data stored in the backward prediction image section 63b is read out, and the motion compensation circuit 64 As a result, motion compensation is performed corresponding to the motion vector output from the motion vector detection circuit 50. That is, when the setting of the backward prediction mode is instructed by the prediction determination circuit 54,
The data is read by shifting the read address of the backward predicted image section 63b from the position corresponding to the position of the macroblock currently output by the motion vector detection circuit 50 by the amount corresponding to the motion vector, and generates predicted image data.

The predicted image data output from the motion compensation circuit 64 is supplied to a calculator 53b. Arithmetic unit 53b
Subtracts the predicted image data supplied from the motion compensation circuit 64 from the macroblock data of the reference image supplied from the prediction mode switching circuit 52, and outputs the difference. This difference data is supplied to the DCT mode switching circuit 5
5, transmitted to the transmission path via the DCT circuit 56, the quantization circuit 57, the variable length coding circuit 58, and the transmission buffer 59.

In the bidirectional prediction mode in which the switch 53d is switched to the contact point d, the image data (in this case, the I-picture image) stored in the forward prediction image section 63a and the image data stored in the backward prediction image section 63b. Data (in this case, an image of a P picture) is read out, and the motion compensation circuit 64 outputs the motion vector detection circuit 5
Motion compensation is performed corresponding to the motion vector output by 0.
That is, when the setting of the bidirectional prediction mode is instructed by the prediction determination circuit 54, the motion vector detection circuit 50 outputs the read addresses of the forward predicted image section 63a and the backward predicted image section 63b. The data is read out from the position corresponding to the position of the macroblock in which the motion vector is shifted by an amount corresponding to the motion vector (in this case, two motion vectors for the forward predicted image and the backward predicted image), and predicted image data is generated. I do.

The predicted image data output from the motion compensation circuit 64 is supplied to a calculator 53c. Arithmetic unit 53c
Subtracts the average value of the predicted image data supplied from the motion compensation circuit 64 from the macroblock data of the reference image supplied from the motion vector detection circuit 50, and outputs the difference. This difference data is transmitted to the transmission path via the DCT mode switching circuit 55, the DCT circuit 56, the quantization circuit 57, the variable length coding circuit 58, and the transmission buffer 59.

The picture of the B picture is not stored in the frame memory 63 because it is not regarded as a predicted picture of another picture.

In the frame memory 63, the forward predictive image section 63a and the backward predictive image section 63b are switched between banks as required, and a predetermined reference image is
The image stored in one or the other can be switched and output as a forward predicted image or a backward predicted image.

In the above description, the description has been made centering on the luminance block.
2 and the macroblocks shown in FIG. 13 are processed and transmitted. As a motion vector for processing a chrominance block, a motion vector obtained by halving the motion vector of the corresponding luminance block in the vertical and horizontal directions is used.

FIG. 14 is a block diagram showing an example of the structure of the decoder 31 shown in FIG. The encoded image data transmitted via the transmission path (recording medium 3) is received by a receiving circuit (not shown) or reproduced by a reproducing device.
After being temporarily stored in the reception buffer 81 via the input terminal 80, it is supplied to the variable length decoding circuit 82 of the decoding circuit 90. The variable length decoding circuit 82 performs variable length decoding on the data supplied from the reception buffer 81 and converts the motion vector, the prediction mode, the prediction flag and the DCT flag into the motion compensation circuit 8.
7, and outputs the quantization step to the inverse quantization circuit 83, and outputs the decoded image data to the inverse quantization circuit 83.

The inverse quantization circuit 83 includes the variable length decoding circuit 8
2 is inversely quantized according to the quantization step also supplied from the variable length decoding circuit 82, and is output to the IDCT circuit 84. The data (DCT coefficient) output from the inverse quantization circuit 83 is output to the IDCT circuit 8
In step 4, the inverse DCT processing is performed and the result is supplied to the arithmetic unit 85.

When the image data supplied from the IDCT circuit 84 is I-picture data, the data is output from the arithmetic unit 85 and is input to the arithmetic unit 85 later (P or B picture data). Is supplied to and stored in the forward prediction image section 86a of the frame memory 86 in order to generate the prediction image data. This data is output to the format conversion circuit 32 (FIG. 9).

When the image data supplied from the IDCT circuit 84 is P-picture data in which the image data one frame before that is predicted image data and is data in the forward prediction mode, the forward prediction in the frame memory 86 is performed. The image data (I-picture data) one frame before stored in the image unit 86a is read out, and the motion compensation circuit 8
At 7, the motion compensation corresponding to the motion vector output from the variable length decoding circuit 82 is performed. Then, the arithmetic unit 85 adds the image data (difference data) supplied from the IDCT circuit 84 and outputs the result. The added data, that is, the decoded P picture data is
In order to generate predicted image data of image data (B-picture or P-picture data) input later to the arithmetic unit 85, the predicted data is supplied to the backward predicted image section 86b of the frame memory 86 and stored.

Even in the case of P-picture data, the data in the intra-prediction mode is stored in the backward prediction image section 86b without being subjected to any particular processing by the computing unit 85, like the I-picture data.

Since this P picture is an image to be displayed next to the next B picture, it is not yet output to the format conversion circuit 32 at this point (as described above, the P picture inputted after the B picture The picture is processed and transmitted before the B picture).

When the image data supplied from the IDCT circuit 84 is B picture data, the image data is stored in the forward prediction image section 86 a of the frame memory 86 in accordance with the prediction mode supplied from the variable length decoding circuit 82. I
The image data of the picture (in the case of the forward prediction mode), the image data of the P picture stored in the backward prediction image section 86b (in the case of the backward prediction mode), or both image data (in the case of the bidirectional prediction mode) The motion vector is read and the motion compensation circuit 87 performs motion compensation corresponding to the motion vector output from the variable length decoding circuit 82,
A prediction image is generated. However, if motion compensation is not required (in the case of the intra-picture prediction mode), no predicted picture is generated.

The data subjected to the motion compensation by the motion compensation circuit 87 in this manner is output to the arithmetic unit 85 by the IDC.
It is added to the output of the T circuit 84. This addition output is output to the format conversion circuit 32 via the output terminal 91.

However, this added output is data of a B picture and is not used for generating a predicted image of another image, so that it is not stored in the frame memory 86.

After the picture of the B picture is output, the picture data of the P picture stored in the backward prediction picture section 86b is read out, and the arithmetic unit 85 is provided via the motion compensation circuit 87.
Supplied to However, at this time, no motion compensation is performed.

The decoder 31 includes a prediction mode switching circuit 52 in the encoder 18 shown in FIG.
Circuits corresponding to the T-mode switching circuit 55 are not shown, but processing corresponding to these circuits, that is, a configuration in which the signals of the odd-field and even-field lines are separated is necessary for the original mixed configuration. The process of returning according to
The motion compensation circuit 87 executes.

In the above, the processing of the luminance signal has been described, but the processing of the color difference signal is also performed in the same manner.
However, in this case, the motion vector used for the luminance signal is halved in the vertical and horizontal directions.

Here, the quality of the image transmitted in the above-described image signal encoding / decoding is controlled as shown in FIG. For example, the SNR (SN ratio) of an image is
Control is performed in accordance with the so-called picture types representing the types of coding such as the above-described intra-picture coding, forward prediction coding, bidirectional prediction coding, and the like. Transmitted with poor quality. In other words, it is possible to obtain better image quality if all images can be transmitted at a high transmission rate.However, there is a case where a transmission rate higher than a certain value cannot be selected due to the characteristics of the transmission path. As a visual characteristic, a transmission method as shown in FIG. 15 described above is adopted by utilizing the characteristic that the image quality is better when the image quality is vibrated as shown in FIG. 15 than when all the image qualities are averaged. Thus, a higher quality image is obtained at a certain fixed transmission rate. Therefore, in the configuration of FIG. 12, the transmission rate control is performed by the quantizer 57 so as to achieve this image quality.

[0080]

FIG. 16 shows a case where the above-mentioned image signal encoding / decoding devices are connected in tandem in series.

In FIG. 16, an analog video signal is supplied to an input terminal 200 as an input signal a.
The signal is sent to the image signal encoding device 201. In the image signal encoding device 201, an input signal a, which is an analog video signal, is converted into a digital video signal by an A / D converter 211, and this signal is encoded by an encoding circuit 212 as described above. The output (encoded digital video signal) of the encoding circuit 212 is supplied to the image signal decoding device 2 in the next stage.
02.

In the image signal decoding device 202, the supplied encoded digital video signal is decoded by the decoding circuit 213, converted to an analog signal by the D / A converter 214, and output. The output signal b, which is an analog video signal, is sent to an image signal encoding device 203 similar to the image signal encoding device 201 in the next stage.

Similarly, the image signal encoding device 20
The encoded digital video signal from the image signal decoding device 20 is the same as the image signal decoding device 202 described above.
The output signal c, which is an analog video signal from the image signal decoding device 204, is output via a terminal 205 or sent to a further connected image signal encoding device or the like.

Here, the analog video signal from the first stage encoding / decoding device having the tandem connection configuration, that is, the output signal b from the image signal decoding device 202, and 2
FIG. 17 shows changes in the SNR (SN ratio) of an image in the analog video signal from the encoding / decoding device in the second stage, that is, the output signal c from the image signal decoding device 204.

In FIG. 17, it can be seen that the output signal c has a significantly lower SNR than the output signal b. This is because the picture type applied in the first stage encoding / decoding device is different from the picture type applied in the second stage encoding / decoding device. That is, in the first stage encoding / decoding device, B
An image encoded as a picture is encoded /
When the decoding apparatus encodes a picture as, for example, a P-picture, the picture quality is vibrated according to the picture type as described above, so that the picture quality is greatly deteriorated.

Further, since the picture quality deterioration caused by the tandem connection is caused by the difference of the picture type between the stages, the picture quality is deteriorated by digitally connecting the codecs (between the encoding / decoding devices). Also occurs in the case where

FIG. 18 shows a connection state in the case of this digital connection.

In FIG. 18, an analog video signal input via a terminal 300 is converted into digital data by an A / D converter 301 and supplied to an image signal encoding device 302. This image signal encoding device 302
Then, the digital video data is sent to an encoding circuit 312 via a digital interface 311 and encoded (compressed) by the encoding circuit 312 to be converted into a digital video bit stream.

The digital video signal (compressed signal bit stream) from the encoding circuit 312 is sent to an image signal decoding device 303, and the image signal decoding device 3
The digital video signal is decoded by a decoding circuit 313 in the circuit 03 and is output as a digital video signal (output signal b) via a digital interface 314.

This digital video signal is supplied to an image signal encoder 304 similar to the image signal encoder 302 described above.
Sent to Hereinafter, similarly, the image signal encoding device 30
4 is further decoded by an image signal decoding device 305 similar to the above-described image signal decoding device 303. Is converted into an analog video signal (output signal c) by the D / A converter 306, and then output from the terminal 307.

In the digital connection shown in FIG.
The same problem as in FIG. 17 described above occurs.

Therefore, the present invention has been made in view of such a situation, and when a tandem connection is made,
An object of the present invention is to provide an encoding device and an encoding method that realize encoding with little deterioration in image quality.

[0093]

An encoding apparatus according to the present invention comprises:
A coding device for coding video data, means for extracting, from the video data, information indicating a picture type specified in a past coding process performed on the video data, Encoding means for encoding data; and control for controlling the encoding means based on the information indicating the picture type, so that the video data is encoded with the same picture type as in the past encoding processing. With the provision of the means, the above-described problem is solved.

[0094] The encoding method of the present invention is an encoding method for encoding video data. The encoding method is a method for encoding a picture type specified in a past encoding process applied to the video data from the video data. Extracting information indicating the picture type, based on the information indicating the picture type, using the same picture type as the past encoding process,
The above-mentioned problem is solved by comprising an encoding step of encoding the video data.

That is, according to the present invention, information indicating a picture type specified in a past encoding process performed on video data is extracted, and the encoding process is controlled based on the information indicating the picture type. Thereby, for example, when a multi-stage tandem connection in which the encoding and decoding processes are performed in a multi-stage configuration is performed, the encoding type in the first stage encoding and the encoding type in the next stage encoding can be matched. .

[0096]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will be described with reference to the drawings.

First, a case of analog connection will be described as a first embodiment.

The image signal transmission device (image signal encoding device / decoding device) of the first embodiment to which the encoding device and method of the present invention are applied is connected in series in tandem (in this embodiment, two stages are connected). 1 is shown in FIG.

The image signal transmission apparatus according to the present embodiment encodes / decodes an image according to any of the I-, P-, and B-picture types and transmits the image. The image signal decoding device 121, the encoding device 122, and the decoding device 123 are connected in series in tandem, and the image signal encoding devices 120 and 12 are connected.
2 (in this case, a bit stream of a digital video signal for performing A / D conversion in each of the encoding devices 120 and 122). The picture types at the time of encoding in the image signal encoding devices 120 and 122 are detected by decoding circuits 103 and 110, and the D / A converters 104 and 109 convert the picture types into analog video signals after the D / A conversion. Are multiplexed by the multiplexing circuits 105 and 111, and the image signal encoding device 122 converts the ID signal representing the picture type multiplexed into the analog video signal (output signal b) by the separation circuit 106. The coding circuit 108 separates the analog video signal into an intra-picture code according to the separated picture type. , In which so as to encode the forward predictive coding, the bidirectional predictive coding. The encoding circuit 102 of the first-stage image signal encoding device 120 encodes the A / D converted output of the analog video signal (input signal a) based on the picture type (ID signal) from the terminal 118. It has become.

That is, in FIG.
At 0, the analog video signal is supplied as an input signal a and sent to the image signal encoding device 120. In the image signal encoding device 120, the input signal a, which is an analog video signal, is converted into a digital video signal by the A / D converter 101, and this signal is encoded by the encoding circuit 102.
At the time of encoding by the encoding circuit 102, the terminal 118
Performs any of intra-picture coding, forward prediction coding, and bidirectional prediction coding based on an ID signal indicating a picture type supplied via. The output of the encoding circuit 102 becomes the output signal of the image signal encoding device 120, and the encoded bit stream of the digital video signal is sent to the image signal decoding device 121 at the next stage. Note that the encoding circuit 120 can be configured to be capable of encoding even if the picture type is not supplied.
The encoding circuit 120 in this case can be realized by having the same configuration as that of FIG.

In the image signal decoding device 121, the supplied encoded digital video signal is decoded by the decoding circuit 103, and the decoding circuit 103 detects the ID signal indicating the picture type. The multiplexing circuit 105 of the image signal decoding device 121 includes an analog video signal obtained by converting the digital video signal decoded by the decoding circuit 103 by the D / A converter 104 and an ID of the picture type. And a signal. The multiplexing circuit 105 multiplexes and outputs the ID signal during a vertical blanking period of the analog video signal. The output of the multiplexing circuit 105 is output as an analog video signal (output signal b) from the image signal decoding device 121.

The output signal b from the image signal decoding device 121 is sent to the next-stage image signal encoding device 122. The output signal b supplied to the image signal encoding device 122 is supplied to the separation circuit 106. The separation circuit 106 separates the analog video signal and the ID signal, and converts the analog video signal into the A / D converter 1.
At 07, the ID signal is sent to the encoding circuit 108. In the encoding circuit 108, the A / D converter 10
The video signal converted to a digital signal in
Encoding is performed based on the picture type corresponding to the D signal.
The output of the encoding circuit 108 is the output (bit stream of the digital video signal) of the image signal encoding device 122.

The bit stream from the image signal encoding device 122 is further transmitted to the next-stage image signal decoding device 12.
3 is supplied. In the image signal decoding device 123,
The supplied encoded digital video signal is decoded by the decoding circuit 110, and an ID signal representing the picture type is detected. The image signal decoding device 12
The digital video signal decoded by the decoding circuit 110 is supplied to the D / A converter 109 by the multiplexing circuit 111 of FIG.
The analog video signal converted by the above and the picture type ID signal are supplied. The multiplexing circuit 111 multiplexes and outputs the ID signal during the vertical blanking period of the analog video signal. The output of the multiplexing circuit 111 is output to the image signal decoding device 12.
3 is output as an analog video signal (output signal c), and further sent to a subsequent configuration via a terminal 119.

That is, in the tandem connection of FIG. 1, the picture type (ID signal) is set to the first codec (the image signal encoding device 120 and the decoding device 12).
1 is encoded to match the picture type of the second-stage codec (encoding / decoding by the image signal encoding device 122 and the decoding device 123). It has become so.

The SNR of the video output signal b of the first stage and the video output signal c of the second stage in the above-described configuration of FIG.
(SN ratio) is shown in FIG. From FIG. 2, as described above, by matching the picture types of the first stage codec and the second stage codec, there is no discrepancy in the picture types between the first stage and the second stage. Even if the image quality is vibrated according to the type,
It is possible to minimize the deterioration of the image due to the tandem connection.

Next, as a second embodiment of the present invention,
The case of digital connection is described below.

FIG. 3 shows a connection state in the case where an image signal encoding / decoding device as an image signal transmitting device according to the second embodiment is digitally connected in tandem in series.

That is, in FIG.
In the case of 0, an analog video signal is supplied as an input signal a, and this analog video signal is converted by the A / D converter 141 into a digital video signal. This digital video signal is sent to the image signal encoding device 142. The image signal encoding device 142 sends the digital video signal to the encoding circuit 152 via the digital interface 151. This encoding circuit 152 has a terminal 14
An ID signal representing the picture type is also provided via 8
Converting the digital video signal based on the ID signal into
The coding is performed by any of intra-picture coding, forward prediction coding, and bidirectional prediction coding. The output of the encoding circuit 152 becomes the output signal (compressed signal bit stream) of the image signal encoding device 142, and the compressed signal bit stream is sent to the next-stage image signal decoding device 143. Note that, also in the configuration of FIG. 3, the image signal encoding device 142 can be configured to perform encoding without using a picture type, as in the first embodiment described above.

In the image signal decoding device 143, the digital video signal of the supplied compressed signal is decoded by the decoding circuit 1
53, and I representing the picture type
The D signal is detected. The multiplexing circuit 155 of the image signal decoding device 143 is supplied with the digital video signal decoded by the decoding circuit 153 via the digital interface 154 and the picture type ID signal. The multiplexing circuit 155 multiplexes the ID signal as a flag for each image unit in the format of the digital video signal. The multiplexing circuit 15
5 is output as a digital video signal (output signal b) from the image signal decoding device 143.

The output signal b from the image signal decoding device 143 is sent to the next-stage image signal encoding device 144. The output signal b supplied to the image signal encoding device 144 is supplied to a separation circuit 156. The separation circuit 156 separates the digital video signal from the ID signal, and sends the digital video signal to the digital interface 157 and the ID signal to the encoding circuit 158. The encoding circuit 158 encodes the digital video signal via the digital interface 157 based on a picture type corresponding to the ID signal. The output of the encoding circuit 158 is the output (bit stream of the compressed signal) of the image signal encoding device 144.

The bit stream from the image signal encoding device 144 is further transmitted to the next image signal decoding device 14.
5 is supplied. In the image signal decoding device 145,
The decoding circuit 1 decodes the supplied bit stream of the compressed signal.
60, and I representing the picture type
The D signal is detected. The multiplexing circuit 161 of the image signal decoding device 145 is supplied with the digital video signal decoded by the decoding circuit 160 via the digital interface 159 and the picture type ID signal. The multiplexing circuit 161 multiplexes and outputs the ID signal as a flag in the format of the digital video signal. The output of the multiplexing circuit 161 is further converted by a D / A converter 146 into an analog video signal (output signal c), which is sent to a subsequent configuration via a terminal 147.

Also in the digital connection shown in FIG. 3, the picture type (ID signal) is the same as the picture type of the first codec (encoding / decoding in the image signal encoding device 142 and the decoding device 143) and The codec at the stage (the image signal encoding device 144 and the decoding device 145)
(Encoding / decoding at the same time) in order to match the picture type.

The SNR of the video output signal b of the first stage and the video output signal c of the second stage in the above-described configuration of FIG.
As shown in FIG. 2 described above, there is no difference in picture type between the first stage and the second stage. Therefore, even if the image quality is vibrated according to the picture type as described above, It is possible to minimize the deterioration of the image due to the tandem connection.

Finally, FIG. 4 shows a more specific configuration of the encoding circuit in each image signal encoding device in the first and second embodiments described above. In FIG. 4,
The same components as those in FIG. 11 described above are denoted by the same reference numerals, and a detailed description thereof will be omitted.

In FIG. 4, an ID signal indicating a picture type is supplied from a terminal 70, and this ID signal is
The data is sent to the motion vector detection circuit 50, the prediction determination circuit 54, and the variable-length coding circuit 58, and the components are subjected to processing according to the picture type.

FIG. 5 shows a more specific configuration of the decoding circuit in each image signal decoding device in the first and second embodiments. In FIG. 5, the same components as those in FIG. 14 described above are denoted by the same reference numerals, and detailed description thereof is omitted.

In FIG. 5, variable length decoding circuit 82
Detects the picture type in addition to the above-described prediction mode, motion vector, frame / field prediction flag, and frame / field DCT flag, and sends a signal representing the picture type to the motion compensation circuit 87. The motion compensation circuit 87 performs motion compensation according to the picture type. The ID signal indicating the detected picture type is output from the terminal 92 to the subsequent stage.

As described above, according to the present embodiment, in a multi-stage tandem connection, a signal representing the image encoding type is multiplexed with the output image signal of the first stage and output, and this is input to the next stage. By separating the signal into a signal representing the encoding type and an image signal, and performing encoding according to the encoding type, the encoding type in the first-stage decoding and the encoding type in the next-stage encoding Can be matched. As a result, it is possible to minimize the image quality deterioration in the tandem connection of the codecs. Also, in the case of digital connection, the encoding type in the first stage decoding and the encoding type in the next stage encoding are transmitted by multiplexing and transmitting the encoding type to the image data between each stage as digital data. Can be matched, and similarly, image quality degradation can be minimized.
As described above, the present embodiment is an effective method for a multi-stage tandem connection of two or more stages.

[0119]

As described above, according to the present invention,
From the video data, information indicating a picture type specified in a past encoding process performed on the video data is extracted, and based on the information indicating the picture type,
By encoding video data with the same picture type as in the past encoding process, it is possible to achieve encoding with little deterioration in image quality in the case of tandem connection.

[Brief description of the drawings]

FIG. 1 is a block circuit diagram showing a tandem connection (in the case of analog connection) of a video codec according to a first embodiment of the present invention.

FIG. 2 is a diagram for explaining a decrease in SNR in a tandem connection in consideration of a picture type according to the embodiment of the present invention.

FIG. 3 is a block circuit diagram showing a tandem connection (in the case of digital) of a video codec according to a second embodiment of the present invention.

FIG. 4 is a block circuit diagram showing a specific configuration of an encoding circuit in the image signal encoding device according to the embodiment of the present invention.

FIG. 5 is a block circuit diagram showing a specific configuration of a decoding circuit in the image signal decoding device according to the embodiment of the present invention.

FIG. 6 is a diagram illustrating the principle of high-efficiency coding.

FIG. 7 is a view for explaining types of pictures when compressing image data.

FIG. 8 is a diagram illustrating the principle of encoding a moving image signal.

FIG. 9 is a block diagram illustrating a configuration example of a conventional image signal encoding device and a conventional decoding device.

FIG. 10 is a diagram for explaining the format conversion operation of the format conversion circuit 17 in FIG. 9;

11 is a block diagram illustrating a configuration example of an encoder 18 in FIG.

12 is a diagram illustrating the operation of the prediction mode switching circuit 52 of FIG.

13 is a diagram illustrating the operation of the DCT mode switching circuit 55 of FIG.

14 is a block diagram illustrating a configuration example of a decoder 31 in FIG.

FIG. 15 is a diagram illustrating rate control according to a picture type.

FIG. 16 is a block circuit diagram showing a state of tandem connection (in the case of analog connection) of a conventional video codec.

FIG. 17 is a diagram for explaining a decrease in SNR in a tandem connection that does not consider a conventional picture type.

FIG. 18 is a block circuit diagram showing a state of tandem connection (in the case of digital connection) of a conventional video codec.

[Explanation of symbols]

1 encoding device, 2 decoding device, 3 recording medium,
12, 13 A / D converter, 14 frame memory, 15 luminance signal frame memory, 16 color difference signal frame memory, 17 format conversion circuit, 1
8 encoder, 31 decoder, 32 format conversion circuit, 33 frame memory, 34 luminance signal frame memory, 35 color difference signal frame memory, 36,37 D / A converter, 50 motion vector detection circuit, 51 frame memory, 52 prediction mode switching Circuit, 53 operation unit, 54 prediction judgment circuit, 55 DCT mode switching circuit, 56 D
CT circuit, 57 quantization circuit, 58 variable length coding circuit, 59 transmission buffer, 60 inverse quantization circuit,
61 IDCT circuit, 62 arithmetic unit, 63 frame memory, 64 motion compensation circuit, 81 reception buffer, 82 variable length decoding circuit, 83 inverse quantization circuit, 84 IDCT circuit, 85 arithmetic unit, 86
Frame memory, 87 motion compensation circuit, 101, 1
07,141 A / D converter, 102,108,15
2,158 encoding circuit, 103,110,153,
160 decoding circuit, 104, 109, 146 D / A
Converter, 105, 111, 155, 161 multiplexing circuit, 106, 156 demultiplexing circuit, 120, 122,
142, 144 image signal encoding device, 121, 12
3,143,145 image signal decoding device;
154,157,159 Digital interface

Claims (2)

(57) [Claims] 1. An encoding device for encoding video data, comprising: means for extracting, from the video data, information indicating a picture type designated in a past encoding process performed on the video data. Encoding means for encoding the video data; and encoding means for encoding the video data with the same picture type as the past encoding processing based on the information indicating the picture type. And a control means for controlling the encoding. 2. An encoding method for encoding video data, wherein information indicating a picture type specified in a past encoding process applied to the video data is extracted from the video data. And an encoding step of encoding the video data with the same picture type as the past encoding process based on the information indicating the picture type.