JPH11205803A - Encoding device and encoding method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device generating encoded data capable of reproducing images of high quality on a decoder side even at low delay mode by quantizing an image data part and a differential data part based on respective quantization step sizes and generating encoded data. SOLUTION: When actual generated code quantity data B(j) for respective macro-blocks are inputted to the GC calculation part 81A of a quantization control part 80 from a VLC part 15A, a feedback-type video encoder 20 calculates the actual generated code quantity of P-pictures allocated to an intra-slice part and an inter-slice part by the GC calculation part 81A. The GC calculation part 81A obtains GC data Xi showing the difficulty of the picture of the intra-slice part, obtains GC data Xp showing the difficulty of the image of the inter-slice part and supplies GC data Xi and Xp to a target code quantity calculation part 82A.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[Table of Contents] The present invention will be described in the following order.

TECHNICAL FIELD [0002] Prior Art (FIGS. 8 to 11) Problems to be Solved by the Invention Means for Solving the Problems Embodiments of the Invention (1) Overall Configuration (FIGS. 1 and 2) 2) Configuration of Video Encoder (FIGS. 3 to 5) (3) Operation and Effect (4) Other Embodiments (FIGS. 6 and 7) Effects of the Invention

[0003]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an encoding apparatus and an encoding method.
The present invention is suitably applied to an encoding device and an encoding method for performing compression encoding by a Picture Experts Group (Picture Experts Group) method.

[0004]

2. Description of the Related Art In recent years, various compression encoding methods have been proposed as methods for reducing the amount of video and audio information, and a typical one is MPEG2 (Moving Picture Experts).
There is a method called GroupPhase 2).

When compressing and encoding video data according to the MPEG2 system, as shown in FIG. 8, the encoder 1 on the transmitting side converts each frame image of numbers 0 to 11 into an intra-frame encoded image (hereinafter referred to as an encoded image). Which of the three image types, i.e., an I-picture, an inter-frame forward predictive coded image (hereinafter, referred to as a P-picture), and a bidirectional predictive coded image (hereinafter, referred to as a B-picture) After specifying whether to process as the image type, the image type of the frame image (I-picture, P-picture, and B-picture)
, The frame images are rearranged in the order of encoding (hereinafter, this is called reordering), and each frame image is subjected to encoding processing in that order and transmitted.

[0006] The decoder 2 on the receiving side decodes the frame image encoded by the encoder 1 and then reorders the frame image to return to the original order to display the reproduced image. In this case, in the encoder 1,
Since the encoding process is performed after the reordering, the frame image of the number 2 must be encoded before the frame image of the number 0 is encoded. , Which is called a reordering delay).

Also, since the decoder 2 performs reordering after decoding, the frame image of number 2 must be decoded before the frame image of number 0 is decoded and displayed. The reordering day-lay is generated by the minute. As described above, since the encoder 1 and the decoder 2 perform reordering on both sides, a reordering delay for three frames occurs between the time when the image data is encoded and the time when the reproduced image is displayed.

When transmitting coded data that has been compressed and coded according to the MPEG2 system, the coded data transmitted from the compression coding apparatus on the transmission side is converted to a video STD (System Target Decoder) buffer (STD) on the reception side. So-called V
It is stored in BV (Video Buffer Verifier) buffer for each picture.

That is, as shown in FIG. 9, a VBV buffer has a fixed buffer size (capacity), and encoded data is sequentially stored for each picture. In this case, each of the encoded data of the I picture, the P picture, and the B picture is
At a fixed transmission rate, the data is extracted by the decoder at the decode timing at the time (one frame period) stored in the VBV buffer. Here, the I-picture requires a large amount of time to be stored in the VBV buffer because the amount of encoded data is large, and the B-picture has a small amount of encoded data compared to the I-picture, so that it takes a short time. Stored in VBV buffer.

[0010] At this time, the compression encoding apparatus on the transmitting side buffers the VBV buffer so that overflow and underflow do not occur when the encoded data is stored in the VBV buffer on the decoder side and when the encoded data is extracted. It is necessary to control (rate control) the generated code amount of the coded data generated based on the occupancy. In this case, since the I-picture necessary for updating the screen has a large generated code amount (FIG. 9), it requires a long transmission time for the image data, and this time is delayed.

When a delay or a reordering delay due to such a transmission time occurs, when performing real-time transmission requiring real-time performance such as a videophone call or a video conference, the encoded data transmitted from the transmission side is required. There is a problem that a time lag occurs between the receiving side and the display of the reproduced image. In order to reduce such a delay, the MPEG-2 system reduces the delay time to 150 [ms] or less by LOW DELAY CODING.
The method called is prepared by the standard.

In this low-delay coding, only P-pictures are used without using B-pictures and I-pictures with a large amount of generated codes, which cause reordering delays, and the P-pictures are intra-slices consisting of several slices. And an inter-slice consisting of all remaining slices, and encodes without reordering.

Here, the intra slice is an image portion in which the image data of the slice portion is intra-coded, and the inter slice is the image portion of the slice portion and the reference image data of the same area in the previous frame image. Is the image part to be encoded.

For example, as shown in FIG. 10, in low-delay coding, all frame images of numbers 0 to 11 are P-pictures, and a frame image of number 0 in a frame size of 45 macroblocks in width and 30 macroblocks in height. From the upper row, an area corresponding to 2 vertical macroblocks and 45 horizontal macroblocks is set as an intra slice I0, and all other areas are set as an inter slice P0.

The next frame image of number 1 is number 0
The intra-slice I1 is set in a region having the same area at a position following the intra-slice I0 of the frame image of the other frame image, and all the others become the inter-slice P1. Hereinafter, similarly, an intra slice and an inter slice are set for each frame image, and the intra slice I11 and the inter slice P
11 is set.

In low-delay coding, the intra slices I1 to I11 of each frame image are encoded as transmission data as they are, and the other interslices P1 to P11 are encoded with the reference image in the same area of the previous frame image. Encoding is performed based on the difference data, and the encoding is repeatedly performed on the frame image of number 0 to the frame image of number 11, whereby the image data of the entire screen in one P-picture is encoded.

In this case, since the image data sizes of the intra slices I1 to I11 in each frame image are all uniform, and naturally the image data size of the inter slice is also uniform, the generated code amount for each frame image is almost constant. Fixed rate.

As a result, as shown in FIG. 11, all the frame images (P pictures) become encoded data of the same generated code amount, and the transitions of the encoded data when stored in the VBV buffer and when extracted are all changed. It will be the same. As a result, the compression encoding apparatus on the transmission side can easily control the generated code amount of the encoded data without causing underflow and overflow of the VBV buffer on the decoder side, thereby enabling the I-picture having a large generated code amount. The resulting delay and reordering delay can be eliminated, and thus the reproduced image can be displayed by the decoder without delay.

[0019]

By the way, in the compression coding apparatus having such a configuration, the intra slices I1 to I1 are used.
Regarding I11, it is encoded as transmission data as it is,
The other interslices P1 to P11 are coded based on the difference data from the reference image of the same area in the previous frame image, so that the intra slices I1 to I1 are used.
The actual amount of generated code when the image data portion of one is compression-encoded is large, and the actual amount of generated code when the image data portion of the inter slices P1 to P11 is compression-coded is small.

However, although the generated code amount of the entire picture is specified, the intra slices I1 to I11
Further, the generated code amount allocated to each of the inter slices P1 to P11 is not specified. That is, even for an image portion in which the amount of generated codes when encoding is performed as in the intra slices I1 to I11, the inter slice P
The generated code amount is evenly allocated to the image data portion in which the generated code amount does not become too large when encoded as in 1 to P11.

Therefore, an intra slice I having a large data amount
Inter-slices P1 to P11 in which the amount of generated codes allocated to 1 to I11 is small and the amount of data is small
In some cases, the amount of generated codes allocated to the image becomes large, and in such a case, there is a problem that an image as a whole picture is distorted.

The present invention has been made in view of the above points, and has an encoding apparatus and an encoding method capable of generating encoded data capable of reproducing a high-quality image on a decoder side even in a low-delay mode. It is intended to propose.

[0023]

According to the present invention, in order to solve the above-mentioned problem, an image data portion consisting of a predetermined region and a difference between the image data of the remaining region and the image data in the same region of the reference frame corresponding to the remaining region are obtained. When quantizing and encoding the frame image allocated to the difference data portion, the actual generated code amount in the image data portion of the frame image and the actual generated code amount in the difference data portion are calculated. Based on the actual generated code amount in the difference data portion, the target generated code amount of the image data portion and the difference data portion in the next frame image is calculated, and the actual generated code amount for the target generated code amount of the image data portion is calculated. Based on the ratio of the actual amount of generated code to the target Then, the quantization step size for encoding the image data portion of the next frame image to the same code amount as the target generated code amount of the image data portion, and the difference data portion of the next frame image as the target generated code of the difference data portion By individually calculating the quantization step size to be encoded to the same code amount as the amount, and quantizing the image data portion and the difference data portion based on the respective quantization step sizes to generate encoded data, The encoded data can be generated by quantizing the image data portion and the differential data portion in the next frame image so that the target generated code amounts of the data portion and the differential data portion are obtained.

A frame image divided into an image data portion composed of a predetermined region and a difference data portion composed of a difference between the image data of the remaining region and the image data in the same region of the corresponding reference frame is quantized and encoded. In this case, it is determined that the pattern of the next frame image to be encoded and the pattern of the immediately preceding frame image have changed, and the pattern of the next frame image to be encoded is changed to the pattern of the immediately preceding frame image. If it changes, the initial buffer capacity of the virtual buffer assuming the transition of the coded data stored and output in the buffer provided in the decoder is initialized, and the next frame to be coded based on the initial buffer capacity is initialized. The quantization step sizes for the image data portion and the difference data portion in the image are individually calculated, and the image data portion and the difference data portion are calculated. By quantifying the portion based on each quantization step size to generate coded data, when the pattern of the next frame image to be coded changes due to scene change, it becomes the previous frame image. Instead of quantizing the frame image to be encoded next based on the quantization step size calculated based on the quantization step size, the optimal generated code amount for the image data portion and the difference data portion in the frame image to be encoded next is calculated. Encoded data can be generated.

[0025]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below in detail with reference to the drawings.

(1) Overall Configuration As shown in FIG. 1, the data transmission device 2 includes a video encoder 20, an audio encoder 24, a subtitle encoder 28, a multiplexing system 12, and a control system 42.

The multiplexing system 12 includes input FIFO memories 32A, 32B, 32C, a first switch circuit 34, a second switch circuit 36, an output FIFO memory 38, and a SCSI (Small Computer System Interface) interface. It comprises a circuit 40.

The control system 42 includes a data size counting interface circuit (data size IF) 30A, 30B,
30C, a RAM 430, an Ethernet interface circuit (Ethernet interface: ENIF circuit) 420, a serial interface circuit (Serial Interface: SIF circuit) 422, a CPU 424, a processing RAM 426, and a control data RAM 428 are interconnected via a CPU bus. Has a configuration connected to

The CPU 424 sets a target generated code amount for the video encoder 20 and, based on the generated code amount of the actually generated video encoded data, sets the target generated code amount after the compression encoding. Control is performed so as to be equal to the code amount. By the setting of the target generated code amount by the CPU 424 and the control of the video encoder 20, each component of the data transmission device 2 compresses and encodes the image data input from the external device by the MPEG2 system, and generates the image data substantially equal to the target generated code amount. The coded data of the code amount is generated, and the data size IF 30A and the F
The data is sent to the IFO memory 32A.

The audio encoder 24 compresses and encodes audio data input from an external device in accordance with the MPEG2 method to generate audio encoded data, and outputs the data size IF 30B and the FIFO memory 3 of the data transmitting device 2.
2B.

The subtitle encoder 28 encodes the subtitle data input from the external device that generates the subtitle data by linear quantization processing and fixed-length encoding processing, and as the subtitle encoded data, the data size IF 30C of the data transmitting apparatus 2. And FIFO
The data is sent to the memory 32C.

Here, in the multiplexing system 12, the FIFO memories 32A, 32B and 32C store the video encoder 2
0, buffers video coded data, audio coded data and subtitle coded data inputted from the audio encoder 24 and the subtitle encoder 28, and sends them out to the input terminals a, b, c of the switch circuit 34.

The switch circuit 34 selects one of the input terminals a, b, c, and d according to the control of the multiplexing system 12 via a control signal, and encodes the encoded data inputted to each of these input terminals. (Elementary stream) is selected and multiplexed, and transmitted to the input terminal b of the switch circuit 36. The switch circuit 34 has input terminals a, b, c, and d when there is no encoded data to be input to any of the input terminals, or when performing a stuffing process.
, And outputs predetermined blank data (continuous theoretical value 1 or 0).

The switch circuit 36 selects one of the input terminals a and b according to the control of the multiplexing system 12 through the control signal, and outputs the encoded data input from the switch circuit 34 to the input terminal b. Either or input terminal a for processing R
The private data stream (header data) input from the AM 426 is selected and multiplexed, and sent to the FIFO memory 38 and the SCSIIF circuit 40.

The FIFO memory 38 includes a switch circuit 36
Buffers the multiplexed data stream and sends it out as a transport stream to a data receiving device 60 described later via a predetermined communication line.

The SCSIIF circuit 40 includes the switch circuit 3
6 outputs the multiplexed data stream to a hard disk device (HDD) or a recording device (not shown) of a magneto-optical disk device (MOD) for recording.

In the control system 42, the data size IF3
0A, 30B, and 30C are video encoders 2 respectively.
0, the data size of each frame of the video coded data, audio coded data and subtitle coded data input from the audio encoder 24 and the subtitle encoder 28 is counted, and C is counted via the CPU bus.
Send it to PU424.

The counting of the data size is performed by a counter built in the data size IF 30A, 30B, 30C.

The ENIF circuit 420 receives private data input via a LAN (not shown) such as an Ethernet and the like, and receives the private data via a CPU bus.
Is sent to The SIF circuit 422 receives serial private data input from, for example, a computer and sends it to the CPU 424.

The CPU 424 is composed of, for example, a microprocessor, a ROM for storing programs, and peripheral circuits thereof, and has a data size IF 30A, 30B, 30B.
C, the data size input from the ENIF circuit 420 and the SIF circuit 422 is stored in the processing RAM 426,
Based on the remaining storage capacity (remaining buffer capacity) of the IFO memories 32A, 32B, 32C, etc., the data size stored in the processing RAM 426, and the image frame signal supplied from the video encoder 20, the multiplexing order of the encoded data is determined. A multiplexing method such as multiplexing timing adjustment and scheduling is planned, and the multiplexing operation of the switch circuits 34 and 36 is controlled via the CPU bus according to the planned multiplexing method. In this embodiment, the CPU 424 calculates a multiplexing plan for each video frame based on the image frame signal supplied from the video encoder 20.

The control data RAM 428 has a CPU
It stores control data related to the processing of 424,
The CPU 424 creates header data on the processing RAM 426 in accordance with scheduling using the control data stored in the control data RAM 428, and sends it to the input terminal a of the switch circuit 36.

A part of the header data may be created based on the private data input through the ENIF circuit 420 or the SIF circuit 422 and stored in the control data RAM 428.

Here, in the data transmission device 2, the PCR (P
rogram Clock Reference: Program time reference value)
Adaptation fields and PES (Pac
ketized Elementary Stream)
4 and the values are stored in the processing RAM 426. Incidentally, the adaptation field is a field to which additional information relating to an individual stream or information of a dynamic state change such as a stuffing byte (invalid data byte) can be assigned.

The information stored in the processing RAM 426 in this manner is transmitted to the switch circuit 36 by the switch circuit 3.
4 is inserted into each coded data inputted at a predetermined timing through the PTS 4, thereby forming a PES packet and dividing the data into predetermined lengths and multiplexing the TS (Transport Stroke).
eam) Packetization is performed.

On the other hand, FIG.
FIG. 1 shows the configuration of a data receiving device 60 that receives a transport stream output from FIG. 1; a packet separating circuit 53 of a separating unit 55 receives the transmitted transport stream and converts the transport stream from the transport stream; The port packet header is separated, and stuffing bytes are removed to reproduce the PES packet.

Subsequently, the packet separating circuit 53 of the separating section 55
Removes the PES packet header from the reproduced PES packet and outputs it to the input terminal of the switch circuit 54.
The packet separating circuit 53 controls the connection between the input terminal and the output terminal of the switch circuit 54.

The switch circuit 54 includes a packet separating circuit 5
In accordance with the control of (3), compressed video coded data is output from the output terminal a to the video STD buffer 58, and compressed audio coded data is output from the output terminal b to the audio STD buffer 59. To separate video encoded data and audio encoded data.

The video STD buffer 58 buffers video encoded data input from the switch circuit 54 and supplies one frame of video encoded data to the video decoder 61 at a predetermined timing. The video decoder 61 uses the M
The data is decompressed and decoded by the PEG2 method, and the decoding result is displayed on a monitor (not shown). The video STD buffer 58
Are monitored by the video encoder 20 for the transition of the accumulation state of the encoded data supplied by the video encoder 20 of the data transmitting apparatus, and are controlled so as not to overflow and underflow.

The audio STD buffer 59 buffers the audio coded data inputted from the switch circuit 54 and supplies one frame of audio coded data to the audio decoder 62 at a predetermined timing. The audio decoder 62 performs decompression decoding by the MPEG2 method performed in the audio encoder 24, and outputs the decoded audio via a speaker (not shown).

(2) Configuration of Video Encoder Here, the video encoder 20 is a feedback type data encoding device, which is entirely adapted to perform encoding by a low-delay coding method using P-pictures. As shown in FIG. 3, the video encoder 20
The image data D1A supplied from the outside is converted into a preprocessing unit 11A.
To enter. The pre-processing unit 11A converts each frame image (all P pictures in this case) of the sequentially input image data D1A into a macro block constituted by a luminance signal of 16 pixels × 16 lines and a color difference signal corresponding to the luminance signal. This is supplied to the arithmetic circuit 12A and the motion vector detector 23A as macroblock data D2A.

In this case, the preprocessing unit 11A divides all of the intra slices I1 and the inter slices P1 in the frame image of the number 0 (FIG. 10) into macro blocks, and repeats this until the frame image of the number 11.

The motion vector detecting section 23A calculates a motion vector of each macro block of the macro block data D2A based on the macro block data D2A and the reference image data D14A stored in the frame memory 21A. The signal is sent to the motion compensator 22A as 18A.

The arithmetic circuit 12A, for the macroblock data D2A supplied from the preprocessing unit 11A, determines the image type of each macroblock of the macroblock data D2A (intra mode or interslice P1 for intra slices I1 to I11). For P11 to P11, motion compensation in the intra mode or the forward prediction mode based on the forward prediction mode is performed.

Here, the intra mode is a method in which a frame image to be encoded is directly used as transmission data, and the forward prediction mode is a method in which a prediction residual between the frame image to be encoded and a past reference image is used. Is the transmission data. By the way, in the video encoder 20 in this case, since encoding is performed using only P-pictures and divided into intra-slices I1 to I11 and inter-slices P1 to P11, the encoding is performed in the intra mode and the forward prediction mode. Only motion compensation is performed.

First, the case where the macroblock data D2A is one of the intra slices I1 to I11 will be described. In this case, the macro block data D2
A is processed in the intra mode. That is, the arithmetic circuit 12A converts the macro block of the macro block data D2A into the DCT (Discrete
(Cosine Transform, discrete cosine transform) section 13A. The DCT unit 13A applies DC to the operation data D3A.
DCT coefficients are obtained by performing T conversion processing,
The data is sent to the quantization unit 14A as CT coefficient data D4A.

The quantization section 14A includes a generated code amount control section 80
The quantization processing is performed on the DCT coefficient data D4A based on the quantized index data Q (j + 1) supplied from A, and the VLC unit 1 generates the quantized DCT coefficient data D5A.
5A and the inverse quantization unit 18A. Here, the quantization unit 14A adjusts the quantization step size in the quantization process according to the quantization index data Q (j + 1) supplied from the generated code amount control unit 80A, thereby generating a code to be generated. It is made to control the amount.

The quantized DC transmitted to the inverse quantization unit 18A
The T coefficient data D5A undergoes an inverse quantization process using the same quantization step size as the quantization unit 14A, and is sent to the inverse DCT unit 19A as DCT coefficient data D11A. Then, the DCT coefficient data D11A undergoes inverse DCT processing in the inverse DCT section 19A, is sent to the arithmetic circuit 20A as arithmetic data D12A, and is stored in the frame memory 21A as reference image data D13A.

Next, the case where the macroblock data D2A is one of the inter slices P1 to P11 will be described. In this case, the arithmetic circuit 12A performs a motion compensation process in the forward prediction mode on the macroblock data D2A.

The arithmetic circuit 12A has the macro block data D
For 2A, a subtraction process is performed using the forward prediction image data D17A supplied from the motion compensation unit 22A. The forward prediction image data D17A is calculated by performing motion compensation on the reference image data D13A stored in the frame memory 21A according to the motion vector data D18A.

That is, in the forward prediction mode, the motion compensator 22A reads the reference image data D13A by shifting the read address of the frame memory 21A in accordance with the motion vector data D18A, and uses this as the forward prediction image data D17A. And arithmetic circuit 20
A. The arithmetic circuit 12A subtracts the forward prediction image data D17A from the macroblock data D2A to obtain difference data as a prediction residual, and this is used as arithmetic data D3
A is transmitted to the DCT unit 13A.

The operation circuit 20A includes a motion compensator 22.
The forward prediction image data D17A is supplied from A, and the arithmetic circuit 20A locally reproduces the reference image data D13A by adding the forward prediction image data D17A to the arithmetic data D12A, and stores it in the frame memory 21A.

Thus, the image data D1A input to the video encoder 20 undergoes the motion compensation prediction process, the DCT process, and the quantization process to obtain the quantized DCT coefficient data D5.
A is supplied to the VLC unit 15A. VLC unit 15A
Performs variable-length coding on the quantized DCT coefficient data D5A based on a predetermined conversion table, and converts the resulting variable-length coded data D6A into a buffer 141A.
And the generated code amount data B (j) indicating the number of coding generated bits for each macro block are generated by a generated code amount control unit 80A of the quantization control unit 80 and a GC (Global Comple
xity) is sent to the calculation unit 81A.

The GC calculation section 81A calculates the generated code amount data B
(j) are sequentially accumulated for each macro block, and when all the generated code amount data B (j) for one picture are accumulated, the generated code amount data B (j) for all macro blocks is cumulatively added. Thus, the generated code amount for one picture is calculated.

The GC calculation section 81A calculates the following equation:

[0065]

(Equation 1)

By calculating the product of the generated code amount of the intra slice portion of one picture and the average value of the quantization step size in the intra slice portion, the difficulty of the image in the intra slice portion is calculated. hereinafter simply seek GC data X _i representing the called a GC), and supplies it to the target code amount calculating section 82A of this.

At the same time, the GC calculator 81A calculates the following equation:

[0068]

(Equation 2)

By calculating the product of the generated code amount of the inter-slice portion of one picture and the average value of the quantization step size in the inter-slice portion, the GC data X _p in the inter-slice portion is calculated. Then, this is supplied to the target code amount calculation unit 82A.

The target code amount calculation section 82A is provided with the GC calculation section 8
The following equation based on GC data X _i supplied from 1A

[0071]

(Equation 3)

Is used to calculate the target generated code amount data T _pi of the intra slice portion in the next picture, and the GC data X _p supplied from the GC calculation section 81A.
Based on

[0073]

(Equation 4)

The target generated code amount data T _pp of the interslice portion in the next picture is calculated using the above, and the calculated target generated code amount data T _pi and T _pp are transmitted to the generated code amount control unit 80A. .

Incidentally, the generated code amount control section 80A constantly monitors the accumulation state of the variable length encoded data D6A stored in the buffer section 141A, and based on the occupation amount information indicating the accumulation state, the quantization step size is determined. Has been made to determine.

When the generated code amount data B (j) of the macro block actually generated is larger than the target generated code amount data T _pi of the intra slice portion, the generated code amount control section 80A reduces the generated code amount. In order to increase the generated code amount, if the actual generated code amount data B (j) is smaller than the target generated code amount data _Tpi , the quantization step size is decreased. It has been made to be.

Further, the generated code amount control unit 80A similarly outputs the target generated code amount data T
_If the generated code amount data B (j) of the macro block actually generated is larger than _pp , the quantization step size is increased to reduce the generated code amount, and the actual generated code amount data B (j) is larger than the target generated code amount data T _pp. When the generated code amount data B (j) is small, the quantization step size is reduced to increase the generated code amount.

That is, the generated code amount control unit 80A assumes the transition of the storage state of the variable length coded data D6A stored in the VBV buffer 58 (FIG. 2) provided on the decoder side, as shown in FIG. Thus, the buffer occupancy d (j) of the virtual buffer in the j-th macroblock is

[0079]

(Equation 5)

And the buffer occupancy d (j + 1) of the virtual buffer in the (j + 1) th macroblock.
Is

[0081]

(Equation 6)

By subtracting equation (6) from equation (5), the buffer occupancy d (j + 1) of the virtual buffer in the (j + 1) th macroblock is calculated by the following equation.

[0083]

(Equation 7)

Can be modified as follows.

Subsequently, since the macro block in the picture is divided into an intra slice portion and an inter slice portion, the generated code amount control section 80A determines whether the macro block of the intra slice portion and the inter slice portion are divided as shown in FIG. The target generated code amounts T _pi and T _pp to be assigned to each macro block are individually set. In this case, in the graph, when the count number of the macroblock is between 0 and s and between t and end,

[0086]

(Equation 8)

By substituting the target generated code amount T _pp for the buffer occupation amount d (j
+1) is obtained, and when the number of macroblocks is between st and t, the following equation is obtained.

[0088]

(Equation 9)

By substituting the target generated code amount T _pi for the intra-slice portion, the buffer occupation amount d (j
+1) is obtained.

Therefore, the generated code amount control unit 80A calculates the buffer occupation amount d (j + 1) in the intra-slice part and the inter-slice part by the following equation.

[0091]

(Equation 10)

By substituting into macro block (j
+1), and supplies the quantized index data Q (j + 1) to the quantization unit 14A.

The quantization section 14A determines a quantization step size corresponding to an intra slice or an inter slice in the next macroblock based on the quantization index data Q (j + 1), and determines the quantization step size. Accordingly, the DCT coefficient data D4A is quantized.

As a result, the quantization unit 14A calculates the intra-slice part and inter-slice of the next picture calculated based on the actual generated code amount data B (j) in the intra-slice part and inter-slice part of the previous picture. the I connexion DCT coefficient data D4A to optimum quantization step size Te convex to the target generated code amount T _pp and T _pi can be quantized in portion.

Thus, in the quantization section 14A, in accordance with the data occupation amount of the buffer section 141A, the buffer section 14A
1A can be quantized so as not to overflow or underflow, and the VBV buffer 58 on the decoder side.
It is possible to generate quantized DCT coefficient data D5A that has been quantized so that (FIG. 2) does not overflow or underflow.

The buffer section 141A stores and accumulates variable-length coded data D6A for each macro block supplied from the VLC section 15A, and stores the variable-length coded data D6A for one picture in units of a picture. The data is output as fixed-length encoded data D7A. Here, the generated code amount of the fixed-length coded data D7A output from the buffer unit 141A is substantially the same for each frame image (picture) because all the data is obtained by coding P pictures.

(3) Operation and Effect In the above configuration, the feedback type video encoder 20 outputs the actual generated code amount data B (j) for each macroblock from the VLC unit 15A to the G of the quantization control unit 80.
When input to the C calculator 81A, the GC calculator 81A
, The actual amount of generated codes of the P-pictures distributed to the intra slice portion and the inter slice portion is calculated.

The GC calculation section 81A obtains GC data X _i indicating the difficulty of the image in the intra slice portion, and also obtains G data indicating the difficulty of the image in the inter slice portion.
C data X _p is obtained, and these GC data X _i and X _p
Is supplied to the target code amount calculation unit 82A.

The target code amount calculating unit 82A calculates the target data amount based on the GC data X _i indicating the difficulty of the image of the intra slice part and the GC data X _p indicating the difficulty of the image of the inter slice part in the immediately preceding picture. Next, optimal target code amount data T _pi and T _pp are calculated which are optimal for performing coding without image degradation on the intra-slice portion and the inter-slice portion of the P picture to be coded. It is supplied to the control unit 80A.

The generated code amount control unit 80A determines that the code amount of the actual generated code amount data B (j) for each macroblock sent from the VLC unit 15A is smaller than the target generated code amount data T _pi and T _pp. If the number of bits is large, the quantization step size is increased so as to approach the target generated code amount data T _pi and T _pp , and if the code amount of the actual generated code amount data B (j) for each macroblock is small, a target generated code amount data T _pi and T reduce the quantization step size so as to approach _pp quantization index data Q (j + 1)
To the quantization unit 14A.

The quantizing section 14A performs the DC operation on the intra slice portion based on the quantized index data Q (j + 1).
T coefficient data D4A and DCT of the inter slice part
By performing quantization using the optimal quantization step size corresponding to the coefficient data D4A, encoding can be performed with an optimal generated code amount for each intra slice portion or inter slice portion in the P-picture.

As a result, the fixed-length coded data D7A output via the buffer section 141A becomes coded data coded with an optimum generated code amount for each of the intra-slice portion and the inter-slice portion. Yes,
Thus, when the decoder decodes the fixed-length coded data D7A, it is possible to display a high-quality image with reduced image quality deterioration.

According to the above configuration, the feed-back type video encoder 20 encodes each image data in the low-delay encoding method with a generated code amount optimal for each of the intra-slice portion and the inter-slice portion of the P-picture. With this arrangement, a high-quality image can be reproduced on the decoder side.

(4) Other Embodiments In the above-described embodiment, the case where the feedback type video encoder 20 (FIG. 3) is used as the encoding device has been described. However, the present invention is not limited to this. The configuration is not limited to that shown in FIG.
The video encoder 10 of the feedforward type shown in FIG.

In this case, in FIG. 6 where parts corresponding to those in FIG. 3 are assigned the same reference numerals, the feedforward video encoder 10 includes a quantization control unit 80 (FIG. 3) instead of the quantization control unit 80 (FIG. 3). And a ME (Motion Estimate) calculated by the motion vector detecting unit 23A.
The residual information ME1 is input to the quantization control unit 100.

The feedforward video encoder 10 performs the feedback quantization control similarly to the normal feedback video encoder 20. However, the feedforward video encoder 20 performs the encoding process of the immediately preceding picture and the next picture. If the picture of the next picture changes greatly (so-called scene change), the macroblock data D of the next picture
Based on 2A, ME residual information ME1 is calculated by the motion vector detecting unit 23A.

Here, the ME residual information ME1 is calculated in units of pictures, and is the total value of the luminance difference values in the previous and next pictures. Therefore, when the ME residual information ME1 indicates a large value, it indicates that the picture of the immediately preceding picture is greatly different from the picture of the next picture. That is, the fact that the patterns are different means that the quantized index data Q (j + 1) generated based on the target generated code amount data T _pi and T _pp calculated using the image data D1A of the immediately preceding picture. Therefore, it is not appropriate to determine the quantization step size of the quantization unit 14A.

For this reason, the video encoder 10 stops the feedback control when the picture of the picture to be encoded next greatly changes while performing the normal feedback control of the quantization. The initial buffer capacity d (0) of the virtual buffer is initialized based on the ME residual information ME1 supplied from the vector detection unit 23A, and the quantum is calculated for each of the intra slice and the inter slice based on the new initial buffer capacity d (0). The index data Q (j + 1) is newly calculated.

That is, in the video encoder 10, the generated code amount control section 100A controls the motion vector detection section 23
When the average value avg of the ME residual information originally held by the generated code amount control unit 100A is subtracted from the ME residual information ME1 supplied from A, and the subtraction result exceeds a predetermined threshold D, It recognizes that the frame image to be encoded is a new pattern frame image with scene change.

At this time, the following equation is obtained.

[0111]

[Equation 11]

X representing the degree of difficulty (GC) of an image in units of picture represented by is directly proportional to the ME residual information ME1, and at this time, the difficulty X of the image in unit of picture and the ME residual information ME1 and the following equation

[0113]

(Equation 12)

When the relations are equal as shown in the following, the quantized index data Q of the entire picture is expressed by the following equation.

[0115]

(Equation 13)

The initial buffer capacity d (0) of the virtual buffer at this time is expressed by the following equation.

[0117]

[Equation 14]

Is calculated. Generated code amount control unit 1
In the case of 00A, the quantized index data Q of the entire picture is obtained by substituting the initial buffer capacity d (0) calculated by the equation (14) into the equation (13) again.

After that, the generated code amount control unit 100A
The following equation is used to newly change the average value avg used to determine whether or not a scene change has occurred between the previous picture and the next picture based on the residual information ME1.

[0120]

(Equation 15)

Thus, the new average value avg is recalculated and updated.

Further, generated code amount control section 100A uses the newly calculated initial buffer capacity d (0) to assume the transition of variable length coded data D6A stored in VBV buffer 58 provided on the decoder side. By doing this, the buffer occupancy d (j) of the virtual buffer in the j-th macroblock is

[0123]

(Equation 16)

The buffer occupancy d (j + 1) of the virtual buffer in the (j + 1) th macroblock is expressed by the following equation.

[0125]

[Equation 17]

By subtracting equation (17) from equation (16), the buffer occupancy d (j + 1) of the virtual buffer in the j + 1-th macroblock is expressed by the following equation.

[0127]

(Equation 18)

It can be modified as

Subsequently, since the macroblock in the picture is divided into an intra-slice portion and an inter-slice portion, the generated code amount control section 100A assigns the target generation to be assigned to the macro block of the intra-slice portion and the macro block of the inter-slice portion. Code amount T _PI and T _PP
Are set individually. Also in this case, as shown in the graph (FIG. 5), the count number of the macroblock is 0 to 0.
When between s and t ~ end,

[0130]

[Equation 19]

When the buffer occupancy d (j + 1) is obtained and the count of macroblocks is between s and t, the following equation is obtained.

[0132]

(Equation 20)

The buffer occupation amount d (j + 1) represented by the following expression is obtained.

Accordingly, the generated code amount control unit 100A calculates the buffer occupation amount d (j + 1) calculated for each macro block by the following equation.

[0135]

(Equation 21)

The quantization index value Q (j + 1) of the macroblock (j + 1) in the intra-slice portion is calculated by substituting the value into the quantization index value Q (j + 1), and the quantization index value of the macroblock (j + 1) in the interslice portion is calculated. Value Q (j +
1) is calculated and supplied to the quantization unit 14A.

The quantization section 14A determines a quantization step size corresponding to an intra slice or an inter slice in the next macroblock based on the quantization index value Q (j + 1), and determines the quantization step size based on the quantization step size. Then, the DCT coefficient data D4A is quantized.

As described above, when the picture to be coded is significantly different from the previous picture, the quantization control unit 100A determines the virtual buffer based on the ME residual information ME1 obtained from the picture to be coded. The initial buffer capacity d (0) is newly calculated, and based on the recalculated initial buffer capacity d (0), the optimum quantized index data Q (j + 1) corresponding to each intra slice and inter slice is obtained. Is determined by the quantization unit 14A, rather than determining the quantization step size based on the previous picture having a different picture, but the optimum quantization step size suitable for the picture to be encoded from now on. Is determined.

Next, the generated code amount controller 100A newly calculates the initial buffer capacity d (0) of the virtual buffer based on the ME residual information ME1, and newly calculates the average value avg of the ME residual information ME1. The virtual buffer update processing procedure up to the update will be described with reference to the flowchart of FIG.

That is, the generated code amount control unit 100A
It enters from the start step of T1 and moves to step SP1.
In step SP1, the generated code amount control unit 100A
Upon receiving the ME residual information ME1 supplied from the motion vector detecting section 23A, the process proceeds to step SP2.

In step SP2, the generated code amount control unit 100A subtracts the average value avg of the ME residual information from the supplied ME residual information ME1, and determines whether the subtracted value is larger than a predetermined threshold D. judge.

Here, if a negative result is obtained, this means that the subtraction value is smaller than the predetermined threshold value D, and there is not much difference between the picture in the current picture and the picture in the immediately preceding picture, that is, when the scene change occurs. In this case, the generated code amount control unit 100A proceeds to step SP4.

On the other hand, if an affirmative result is obtained in step SP2, this means that the subtraction value is larger than the predetermined threshold value D, that is, a scene change has occurred. 100A moves to step SP3.

In step SP3, the generated code amount control section 100A determines the equations (11), (12), (13) and (14).
Based on the formula, the initial buffer capacity d (0) of the virtual buffer is calculated, the virtual buffer is updated, and the routine proceeds to step SP4.

In step SP4, the generated code amount control unit 100A calculates and updates the average value avg of the ME residual information based on the equation (15) in preparation for the next picture to be supplied, and returns to step SP1 again. Then, the above-described processing is repeated for the next ME residual information ME1.

[0146] Thus, the video encoder 10 performs normal feedback-type quantization control to determine the optimal quantization step size for each of the intra slices and the inter slices, and performs the quantization control. If a scene change with a picture that is significantly different from the previous picture occurs, instead of using the quantized index data Q (j + 1) calculated based on the previous picture, encode it from now on. The initial buffer capacity d of the virtual buffer based on the ME residual information ME1 of the picture
(0) is updated and new quantized index data Q (j +
By recalculating 1), even when scene change occurs, the quantization control can be performed by determining the optimal quantization step size for each of the intra slices and the inter slices.

In the above-described embodiment, all the frame images of numbers 0 to 11 are P-pictures as the low-delay coding, and the frame images of the frame size of 45 horizontal macroblocks and 30 vertical macroblocks are used. Although the description has been given of the case where the area corresponding to two vertical macroblocks and 45 horizontal macroblocks from the upper stage is set as one intra slice portion and all the others are set as inter slice portions, the present invention is not limited to this. May be formed in an area of other various sizes, such as an area for one vertical macroblock and 45 horizontal macroblocks.

Further, in the above-described embodiment, a case has been described in which the present invention is applied to video encoders 20 and 10 as encoding devices that perform compression encoding according to the MPEG system. The present invention is not limited to this, and the present invention may be applied to an encoding device using various other image compression methods.

[0149]

As described above, according to the present invention, an image is calculated based on a quantization step size calculated so as to have a target generated code amount corresponding to the image data portion and the difference data portion in the next frame image. By quantizing the data portion and the difference data portion to generate encoded data, a high-quality reproduced image can be output at the decoder.

Further, according to the present invention, when the picture of the next frame image to be coded greatly changes due to scene change, the next step is performed based on the quantization step size calculated based on the previous frame image. Rather than quantizing the frame image to be encoded, the decoder generates the encoded data with the optimal generated code amount for the image data portion and the difference data portion in the next frame image to be encoded. Can output a high-quality reproduced image.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a data transmission device according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of a data receiving device.

FIG. 3 is a block diagram showing a configuration of a video encoder.

FIG. 4 is a graph showing the buffer occupancy of a virtual buffer.

FIG. 5 is a graph showing the buffer occupancy of a virtual buffer for each intra slice and inter slice.

FIG. 6 is a block diagram showing a configuration of a video encoder according to another embodiment.

FIG. 7 is a flowchart showing a virtual buffer update processing procedure of a video encoder according to another embodiment.

FIG. 8 is a schematic diagram showing a conventional typical MPEG reordering delay.

FIG. 9 is a schematic diagram showing transition of encoded data in a VBV buffer.

FIG. 10 is a schematic diagram illustrating daylay in a low daylight mode.

FIG. 11 is a schematic diagram showing a transition of encoded data in a VBV buffer when all are P pictures.

[Explanation of symbols]

14A: Quantizing unit, 15A: VLC unit, 10, 20
... Video encoders, 80, 100 ... Quantization control units, 80A, 100A ... Generated code amount control units, 81A ...
... GC calculator, 82A ... Target code amount calculator.

Claims (10)

[Claims] A quantized frame image divided into an image data portion composed of a predetermined region and a difference data portion composed of a difference between the image data in the remaining region and the image data in the same region of the corresponding reference frame. An encoding device for encoding, comprising: a generated code amount calculating unit configured to calculate an actual generated code amount in the image data portion of the frame image and an actual generated code amount in the difference data portion; and the image data portion and the difference. Target code amount calculating means for calculating a target generated code amount of the image data portion and the difference data portion in a next frame image based on each of the actual generated code amounts in the data portion; The ratio of the actual generated code amount to the target generated code amount and the target generation of the difference data portion The quantization step size for encoding the image data portion of the next frame image to the same code amount as the target generated code amount of the image data portion based on the ratio of the actual generated code amount to the code amount, and the next frame image Quantization value calculating means for individually calculating a quantization step size for coding the difference data portion of the difference data portion into the same code amount as the target generated code amount of the difference data portion, and converting the image data portion and the difference data portion to the above. A coding means for generating coded data by performing quantization based on each quantization step size. 2. The image processing apparatus according to claim 1, wherein the frame images are all inter-frame forward prediction coded images, and the image data portion is an intra slice and the difference data portion is an inter slice. An encoding device according to claim 1. 3. The quantization value calculating means does not break down the buffer while assuming a transition of the accumulation state of the coded data stored in the buffer provided in the decoding means for decoding the coded data. 2. The encoding apparatus according to claim 1, wherein the quantization step size is determined. 4. Quantizing a frame image divided into an image data portion composed of a predetermined region and a difference data portion composed of a difference between the image data of the remaining region and the image data in the same region of the corresponding reference frame. In the encoding method for encoding, an actual generated code amount in the image data portion of the frame image and an actual generated code amount in the difference data portion are calculated. Based on the actual generated code amount, a target generated code amount of the image data portion and the difference data portion in the next frame image is calculated, and a ratio of the actual generated code amount to the target generated code amount of the image data portion is calculated. And the ratio of the actual generated code amount to the target generated code amount of the difference data portion. The quantization step size for encoding the image data portion of the next frame image to the same code amount as the target generated code amount of the image data portion, and the difference data portion of the next frame image is represented by the target generated code of the difference data portion. Quantization step sizes to be coded to the same code amount as the amount are individually calculated, and coded data is generated by quantizing the image data portion and the difference data portion based on the respective quantization step sizes. Encoding method. 5. The frame image according to claim 4, wherein all of the frame images are inter-frame forward prediction coded images, and the image data portion is an intra slice and the difference data portion is an inter slice. Encoding method. 6. A quantized frame image divided into an image data portion composed of a predetermined region and a difference data portion composed of a difference between the image data of the remaining region and the image data in the same region of the corresponding reference frame. In the encoding apparatus for encoding, an image determining means for determining that the pattern of the frame image to be encoded next and the pattern of the immediately preceding frame image have changed; When the picture of the previous frame image has changed, the initial buffer capacity of the virtual buffer assuming the transition of the encoded data stored and output in the buffer provided in the decoder is initialized, and the initial buffer is changed. Quantization step size for the image data portion and the difference data portion in the frame image to be coded next based on the capacity Quantizing value calculating means for individually calculating, and quantizing means for generating coded data by quantizing the image data portion and the difference data portion based on the respective quantization step sizes. An encoding device characterized by the above-mentioned. 7. The frame image according to claim 6, wherein all of the frame images are inter-frame forward prediction encoded images, and the image data portion is an intra slice and the difference data portion is an inter slice. An encoding device according to claim 1. 8. The quantization step size calculating means assuming a transition of the accumulation state of the coded data stored in the buffer provided in the decoder so as not to break the buffer. The encoding apparatus according to claim 6, wherein 9. Quantizing a frame image divided into an image data portion composed of a predetermined region and a difference data portion composed of a difference between the image data of the remaining region and the image data in the same region of the corresponding reference frame. In the encoding apparatus for encoding, it is determined that the picture of the next frame image to be encoded and the picture of the immediately preceding frame image have changed, and the picture of the next frame image to be encoded is changed to the previous one. In the case of a change with respect to the picture of the frame image, the initial buffer capacity of the virtual buffer which assumes the transition of the encoded data stored and output in the buffer provided in the decoder is initialized, and based on the initial buffer capacity. The quantization step sizes for the image data portion and the difference data portion in the next frame image to be encoded are individually set. Out, encoding method and generating encoded data by quantizing the basis the image data portion and said differential data portion to each of the quantization step size. 10. The frame image according to claim 9, wherein all of the frame images are inter-frame forward predictive coded images, wherein the image data portion is an intra slice and the difference data portion is an inter slice. Encoding method.